Knuckle installation detection and identification method based on visual image identification

Abstract

The invention discloses a knuckle installation detection identification method based on visual image identification, and the method comprises the steps: obtaining a plurality of images at unqualified occurrence moments, and carrying out the preprocessing of the images, and obtaining a gray image; extracting a plurality of image features from the grayscale image, and screening out key features by using a maximum correlation minimum redundancy algorithm; constructing a Gaussian mixture model, performing training learning by using the key features and the error categories, and establishing a combined model of the Gaussian mixture model and a semi-supervised Gaussian mixture model; and acquiring a monitoring image of a monitoring area in real time, inputting key features of the monitoring image into the joint model, and judging the early warning level of the monitoring area. According to the method, the key features are screened to remove the features with small correlation to the unqualified early warning, the calculation efficiency is improved, and the accuracy of model calculation can be effectively improved by establishing the combined model based on the Gaussian mixture model and the semi-supervised Gaussian mixture model.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile assembly, and particularly to a steering knuckle installation detection and recognition method based on visual image recognition. Background Art

[0002] At present, the unqualified alarm of the steering knuckle is generally realized by using an unqualified detector, but there are also great limitations. Due to various reasons, the detector will have false alarms and missed alarms. The false alarm rate is relatively high, which will bring some inconveniences to the monitoring personnel, but the situation of missed alarms is more serious. Once unqualified occurs, the function of the failed detector is zero. In order to make up for the deficiencies of the unqualified detector, an image-based unqualified monitoring system has emerged and has received great attention at home and abroad. Summary of the Invention

[0003] The purpose of the present invention is to overcome the disadvantages of the existing technology and provide a steering knuckle installation detection and recognition method based on visual image recognition. By acquiring images at multiple unqualified occurrence times and performing preprocessing to obtain grayscale images, key features of the grayscale images are screened to remove features with little relevance to unqualified early warning. By establishing a joint model based on the Gaussian mixture model and the semi-supervised Gaussian mixture model, the qualification of the steering knuckle installation in the real-time detection area is calculated.

[0004] The technical solution of the present invention is as follows:

[0005] A steering knuckle installation detection and recognition method based on visual image recognition, comprising:

[0006] Acquiring images at multiple unqualified occurrence times and performing preprocessing to obtain grayscale images;

[0007] Extracting multiple image features from the grayscale images and screening out key features by using the maximum correlation minimum redundancy algorithm;

[0008] Constructing a Gaussian mixture model and using the key features and error categories for training and learning to establish a joint model of the Gaussian mixture model and the semi-supervised Gaussian mixture model;

[0009] Real-time acquiring monitoring images of the monitoring area and inputting the key features of the monitoring images into the joint model to judge the early warning level of the monitoring area.

[0010] Preferably, the preprocessing includes:

[0011] Traversing and calculating the grayscale value of each pixel point of the image;

[0012] Selecting a sliding window with a length of L, where L = 2N + 1 and N is a positive integer;

[0013] Traverse each pixel, calculate the gray values within the L×L neighborhood of each pixel point and sort them, and find the median value;

[0014] Replace the gray value of this pixel with the median value.

[0015] Preferably, the extraction of the smoke morphological features specifically includes the following steps:

[0016] By labeling the non-connected particle regions in the gray image, the number of particles can be obtained;

[0017] Statistically calculate the pixels within the cross-sectional area of the smoke particles projected on the horizontal plane to obtain the smoke area A = πmn, where m represents the major semi-axis of the smoke particle and n represents the minor semi-axis of the smoke particle;

[0018] Obtain the particle perimeter C = 2πn + 4(m - n) by calculating the number of pixels of the particle ridge line.

[0019] Preferably, the extraction of the texture features specifically includes the following steps:

[0020] Set the pixel displacement parameters (a, b), where a represents the angle, a ∈ (0°, 45°, 90°, 135°), and b represents the distance;

[0021] Generate pairs of gray values from a single pixel point and its offset point;

[0022] Traverse each pixel point to generate a gray-level co-occurrence matrix and calculate the eigenvalues of the gray-level co-occurrence matrix.

[0023] Preferably, the eigenvalues include energy, contrast, inverse difference, and entropy.

[0024] Preferably, the extraction of the shape change specifically includes the following steps:

[0025] Divide the gray image into multiple blocks;

[0026] Adopt the inter-frame difference method to calculate the absolute error of each block respectively, and regard the block with the absolute error greater than the threshold as the moving block;

[0027] Divide the entire plane into 8 directions, namely 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°, and label them as directions 1 to 8 in sequence;

[0028] Calculate the absolute error of the moving block in the 8 directions respectively;

[0029] Regard the direction corresponding to the minimum absolute error as the block movement direction;

[0030] The calculation formula for the absolute error is:

[0031]

[0032] Among them, S represents the absolute error, (i, j) represents the pixel coordinates, l represents the motion block, g represents the gray value, and t represents the current moment.

[0033] Preferably, the key features screened out include:

[0034] Establish an integrated image feature set X r×k ={X 1 , X 2 ,… X k}, where X represents the eigenvalue, r represents the number of samples, and k represents the feature dimension;

[0035] Calculate the mutual trust mean value between each image feature and the error category, and sort them, and select the top z image features as the features with the greatest correlation with the error category;

[0036]

[0037] Use the minimum redundancy criterion to eliminate the redundant features in the features with the greatest correlation;

[0038]

[0039] Among them, maxR 1 (Φ, E) represents the mean value of mutual information, Φ represents the feature subset with z features X f , E represents the error category, minR 2 (Φ), R 2 represents the minimum redundancy, x and y represent two continuous random variables, p(x) represents the probability density of variable x, p(y) represents the probability density of variable y, and p(x, y) represents the joint probability density of variables x and y.

[0040] Preferably, the Gaussian mixture model is:

[0041]

[0042] Among them, N(X; μ X , Σ) represents the Gaussian distribution function, μ X represents the mean value of sample X, represents the unbiased estimate of μ X , z represents the feature dimension of sample X, Σ represents the covariance matrix of sample X, and q represents the number of image samples.

[0043] Preferably, the training and learning specifically include the following steps:

[0044] Label w known image feature samples and mix them with u unknown image feature samples to obtain a mixed feature sample set X(w+u)×z ;

[0045] Introduce a semi-supervised Gaussian mixture model to fit the image feature samples, and calculate the probability and likelihood function that the mixed feature samples belong to a certain warning level.

[0046] Use the expectation-maximization algorithm for iteration until the change amount between the ε-th step and the (ε + 1)-th step in the likelihood function is less than the error value threshold.

[0047] The beneficial effects of the present invention are as follows:

[0048] 1. The steering knuckle installation detection and recognition method based on visual image recognition provided by the present invention obtains images at multiple unqualified occurrence moments, preprocesses them to obtain grayscale images, removes features with little relevance to unqualified warnings by screening key features of the grayscale images, and judges the warning level of the real-time detection area by establishing a joint model based on the Gaussian mixture model and the semi-supervised Gaussian mixture model, with fast calculation speed and high accuracy.

[0049] 2. The steering knuckle installation detection and recognition method based on visual image recognition provided by the present invention removes features with little relevance to the error category through the maximum correlation and minimum redundancy image feature selection criterion, greatly improving the calculation efficiency and clustering accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of the steering knuckle installation detection and recognition method based on visual image recognition provided by the present invention.

[0051] Figure 2 It is a flowchart of extracting the morphological features of the tightening area in an embodiment of the present invention.

[0052] Figure 3 It is a flowchart of extracting the texture features in an embodiment of the present invention.

[0053] Figure 4 It is a flowchart of extracting the shape change in an embodiment of the present invention.

[0054] Figure 5 It is a flowchart of screening out key features in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following specific embodiments illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0056] It should be noted that in the description of the present invention, the terms indicating the direction or positional relationship such as "middle", "upper", "lower", "horizontal", "inner", etc. are based on the direction or positional relationship shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, so it should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0057] In addition, it should also be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "set", "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0058] As Figure 1 shown, the steering knuckle installation detection and recognition method based on visual image recognition includes:

[0059] S100. Obtain images at multiple unqualified occurrence times and perform preprocessing to obtain grayscale images.

[0060] Traverse and calculate the grayscale value of each pixel point of the image;

[0061] Select a sliding window with a length of, where is a positive integer;

[0062] Traverse each pixel, calculate the grayscale values in the neighborhood of each pixel point and sort them, and find the median;

[0063] Replace the grayscale value of this pixel with the median.

[0064] S200. Extract multiple image features from the grayscale image and use the maximum correlation minimum redundancy algorithm to screen out key features.

[0065] Among them, the image features include smoke morphology features, smoke color features, smoke texture features, flame color features, flame texture features, shape changes, and area changes.

[0066] S300. Build a Gaussian mixture model, and use the key features and error categories for training and learning to establish a joint model of the Gaussian mixture model and the semi-supervised Gaussian mixture model.

[0067] The S400 obtains the monitoring images of the monitoring area in real time, and inputs the key features of the monitoring images into the joint model to determine the warning level of the monitoring area.

[0068] By obtaining the images at multiple unqualified occurrence times and preprocessing them to obtain grayscale images, removing the features with little relevance to unqualified warnings by screening the key features of the grayscale images, and establishing a joint model based on the Gaussian mixture model and the semi-supervised Gaussian mixture model to determine the warning level of the real-time detection area, the calculation speed is fast and the accuracy is high.

[0069] Such as Figure 2 As shown, the extraction of the smoke morphology features specifically includes the following steps:

[0070] S211: By labeling the non-connected particle regions in the grayscale image, the number of particles can be obtained;

[0071] S212: Statistically calculate the pixels within the cross-sectional area of the smoke particles projected on the horizontal plane to obtain the smoke area A = πmn, where m represents the long semi-axis of the smoke particles and n represents the short semi-axis of the smoke particles;

[0072] S212: Obtain the particle perimeter C = 2πn + 4(m - n) by calculating the number of pixels of the particle ridge line.

[0073] Such as Figure 3 As shown, the extraction of the texture features specifically includes the following steps:

[0074] S221: Set the pixel displacement parameters (a, b), where a represents the angle, a ∈ (0°, 45°, 90°, 135°), and b represents the distance;

[0075] S222: Generate grayscale value pairs for a single pixel point and its offset point;

[0076] S223: Traverse each pixel point to generate a gray-level co-occurrence matrix and calculate the eigenvalues of the gray-level co-occurrence matrix.

[0077] We can usually use some scalars to represent the features of the gray-level co-occurrence matrix. The commonly used ones are as follows:

[0078] (1) Energy, also known as angular second moment. The formula is:

[0079]

[0080] Among them, G represents the gray-level co-occurrence matrix, and its size is κ×κ. As can be seen from the formula, energy is the sum of the squares of the element values of the gray-level co-occurrence matrix, which reflects the uniformity of the gray-level distribution of the image and the coarseness of the texture. If all the values of the gray-level co-occurrence matrix are equal, it means that the gray-level distribution of the image is relatively average and the texture is finer, so the ASM value is small; on the contrary, if the values are different, it indicates that the gray-level values of the image have changed drastically in a certain area, showing a thick texture on the image, and the ASM value is large.

[0081] (2) Contrast. The formula is:

[0082]

[0083] Contrast reflects the clarity of the image and the depth of the texture grooves. If the texture grooves are deep, it means that the texture is more obvious visually, and its contrast is large; conversely, the contrast is small, the gray-level distribution of the image is average, the grooves are shallow, and it is difficult to distinguish the texture visually.

[0084] (3) Inverse difference, also known as homogeneity. The formula is:

[0085]

[0086] Homogeneity is used to measure the amount of local change in the image texture. A large value indicates that there is little change between different regions of the image texture, and the gray-level distribution of the image is very average. On the contrary, it means that there are drastic changes in the image texture in different regions.

[0087] (4) Entropy. The formula is:

[0088]

[0089] Entropy is a measure of the amount of information in the image. Texture information also belongs to the information of the image. When all elements in the co-occurrence matrix have the greatest randomness, it indicates that obvious texture appears in the original image, and the entropy is larger at this time. On the contrary, when the image distribution is more uniform and there is no obvious texture, the entropy is smaller at this time.

[0090] As Figure 4 shown, the extraction of shape changes specifically includes the following steps:

[0091] S231. Divide the grayscale image into multiple blocks;

[0092] S232. Use the inter-frame difference method to calculate the absolute error of each block respectively, and regard the block with the absolute error greater than the threshold as the moving block;

[0093] S233. Divide the entire plane into 8 directions, namely 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°, and mark them as directions 1 to 8 in turn;

[0094] S234. Calculate the absolute error of the motion block in the eight directions respectively;

[0095] S235. Take the direction corresponding to the minimum absolute error as the motion direction of the block;

[0096] The calculation formula of the absolute error is:

[0097]

[0098] where S represents the absolute error, (i, j) represents the pixel coordinates, l represents the motion block, g represents the gray value, and t represents the current moment.

[0099] However, it is not enough to use only one of these features as the feature quantity for recognition, which may cause inaccurate clustering. Therefore, we try to find multiple unqualified image features for clustering. However, too many features will lead to too high a dimension of the clustering model, affecting the calculation time and efficiency. In particular, the introduction of some irrelevant or redundant features may affect the accuracy of the clustering model. Therefore, it is very necessary to screen the features and eliminate the irrelevant and redundant features.

[0100] The present invention screens the unqualified image features by using the unqualified warning level as the category information. Screen out the image features with relatively high correlation with the unqualified warning level, that is, the features that are useful for subsequent clustering and have a relatively high contribution rate. Assume X r×k ={X 1 , X 2 ,…X k} a comprehensive feature set containing the above image features, X represents the feature value, r represents the number of samples, and k represents the feature dimension; E r×k is the unqualified warning level diversity. Our task is to screen out z foam image features from these k features that are strongly correlated with the unqualified warning and weakly correlated with each other. This kind of correlation and redundancy are usually measured by mutual trust.

[0101] As Figure 5 shown, the specific steps for screening out the key features include the following:

[0102] S241. Establish the comprehensive image feature set X r×k ={X 1 , X 2 ,…X k}, X represents the feature value, r represents the number of samples, and k represents the feature dimension;

[0103] S242. Calculate the mutual trust mean value of each image feature and the error category, and sort them, and select the top z image features as the features with the largest correlation with the error category;

[0104] Mutual trust can be used as a measure of the correlation between two random variables, and its calculation formula is:

[0105]

[0106] Next, find a feature subset Φ with z features X × k = {X 1 , X 2 , … X k} from the comprehensive image feature Xr. These features should have the maximum correlation with the unqualified warning level category set E f . The maximum correlation criterion can be used for calculation, that is, calculate the mean of the mutual trust between each image feature X r×k and the unqualified warning level category set E: f Calculate the mutual trust between each feature and the unqualified warning level from the image feature set X

[0107]

[0108] and the category set E r×k , and sort the mutual trust in descending order. Theoretically, the first z image features among these k are the features with the maximum correlation with the unqualified warning level. r×k

[0109] S243. Use the minimum redundancy criterion to eliminate redundant features among the features with the maximum correlation;

[0110] The correlations between the above z image features may also be very large, with strong redundancy. When one of two mutually redundant image features is removed, the clustering result does not change very much. Therefore, the minimum redundancy criterion can be considered to eliminate redundant image features:

[0111]

[0112] Among them, maxR 1 (Φ, E) represents the mean of mutual information, Φ represents the feature subset with z features X f , E represents the error category, minR 2 (Φ), R 2 represents the minimum redundancy, x, y represent two continuous random variables, p(x) represents the probability density of variable x, p(y) represents the probability density of variable y, and p(x, y) represents the joint probability density of variables x and y.

[0113] Furthermore, the specific steps to establish a joint model of the Gaussian mixture model and the semi-supervised Gaussian mixture model are as follows:

[0114] First, a small number of known image feature samples are labeled and mixed with a large number of unlabeled image feature samples. After clustering the mixed image feature samples through a clustering model, the information of the labeled image feature samples within each cluster is mapped to the unlabeled foam image feature samples to achieve the classification and recognition of the mixed samples.

[0115] For a certain feature X of unqualified image samples at a certain warning level f =(x 1 ,x 2 …x r ), the fitting formula of its one-dimensional Gaussian distribution is as follows:

[0116]

[0117] Among them, μ x represents the mean of X f , and σ represents the standard deviation of X f , and are the unbiased estimates of μ x and σ respectively, and r is the number of foam image samples.

[0118] For k image features X r×k ={X 1 ,X 2 ,…X k} at a certain warning level, to describe its distribution, a multi-dimensional Gaussian distribution can be used for fitting, and its formula is as follows:

[0119]

[0120] Among them, N(X; μ X , Σ) represents the Gaussian distribution function, μ X represents the mean of the sample X, represents the unbiased estimate of μ X , z represents the feature dimension of the sample X, Σ represents the covariance matrix of the sample X, and q represents the number of image samples.

[0121] w known image feature samples are labeled and mixed with u unknown image feature samples to obtain a mixed feature sample set X (w+u)×z . To describe the distribution of all its data samples, a semi-supervised Gaussian mixture model can be introduced to fit the unqualified image feature samples:

[0122]

[0123] Among them, p(X i ) is the unqualified image X i(i = 1, 2, … h+q), (h+q is the total number of unqualified image samples) the probability belonging to each warning level, π λ is the coefficient of the λ-th Gaussian distribution, N(X i ; μ λ, Σ λ ) is the λ-th Gaussian distribution, μ λ is the mean of the foam image features of the λ-th Gaussian distribution sample, Σ λ is the covariance matrix of the unqualified image feature samples of the λ-th Gaussian distribution. In order to estimate the Gaussian mixture model of the above mixed image features, the maximum likelihood estimation method can be used for fitting, and its likelihood function can be expressed as:

[0124]

[0125] For L(X i ) it is difficult to solve using the direct derivative algorithm, so the expectation maximization algorithm is used for iteration until the change amount between the ε-th step and the ε+1-th step in the likelihood function is less than the error value threshold, the iteration ends, and the training and learning are completed.

[0126] The above content is only an embodiment of the present invention. Specific structures and characteristics and other common knowledge well-known in the solution are not described in detail here. It is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific implementation manners. Without departing from the present invention, several deformations and improvements can be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. A steering knuckle installation detection and recognition method based on visual image recognition, characterized in that: include: Acquire multiple steering knuckle installation and tightening images, and perform preprocessing to obtain grayscale images; Extracting multiple image features from the grayscale image and screening out key features using a maximum correlation minimum redundancy algorithm; the image features include morphological features of the tightening area, texture features of the tightening disk, shape changes, and area changes; The extraction of the morphological features of the tightening area specifically includes the following steps: By labeling the unconnected particle regions in the grayscale image, the number of particles can be obtained; The pixels within the cross-sectional area of ​​the smoke particle projection on the horizontal plane are statistically calculated to obtain the smoke area A = πmn, where m represents the major semi-axis of the smoke particle and n represents the minor semi-axis of the smoke particle; The particle perimeter C = 2πn + 4(mn) is obtained by calculating the number of pixels along the particle ridge line; Constructing a Gaussian mixture model, and using the key features and error categories for training and learning, to establish a joint model of the Gaussian mixture model and the semi-supervised Gaussian mixture model; A monitoring image of the monitoring area is acquired in real time, and key features of the monitoring image are input into the joint model to judge the eligibility of the steering knuckle installation.

2. The steering knuckle installation detection and identification method based on visual image recognition according to claim 1, characterized in that: The pre-processing comprises: Traversing and calculating the grayscale value of each pixel of the image; Select a sliding window of length L, where L = 2N + 1, and N is a positive integer; Traverse each pixel, calculate the grayscale value in the L×L neighborhood of each pixel and sort them to find the median; The grayscale value of the pixel is replaced by the median value.

3. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 2, characterized in that: The extraction of the texture features specifically includes the following steps: Set the pixel displacement parameters (a, b), where a represents the angle, a∈(0°, 45°, 90°, 135°), and b represents the distance; A single pixel and its offset point generate a gray value pair; A gray level co-occurrence matrix is ​​generated by traversing each pixel point, and the eigenvalues ​​of the gray level co-occurrence matrix are calculated.

4. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 3, characterized in that: The eigenvalues ​​include energy, contrast, inverse distance and entropy.

5. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 4, characterized in that: The extraction of the shape change specifically includes the following steps: Dividing the grayscale image into a plurality of blocks; Using an inter-frame difference method to calculate the absolute errors of the blocks respectively, and taking the blocks with absolute errors greater than a threshold as motion blocks; Divide the entire plane into 8 directions, namely 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°, 5, and mark them as directions 1 to 8 in sequence; Calculating the absolute errors of the motion block in the eight directions respectively; The direction corresponding to the minimum absolute error is used as the block movement direction; The calculation formula of the absolute error is: Among them, S represents the absolute error, (i, j) represents the pixel coordinates, l represents the motion block, g represents the grayscale value, and t represents the current time.

6. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 5, characterized in that: The key features screened out include: Establish the image feature comprehensive set X r×k ={X1,X2,…X k }, X represents the eigenvalue, r represents the number of samples, and k represents the feature dimension; Calculate the mean mutual trust between each of the image features and the error category, sort them, and select the first z image features as the features with the greatest correlation with the error category; Using a minimum redundancy criterion to eliminate redundant features among the features with the greatest correlation; Among them, maxR1(Φ,E) represents the mean of mutual information, Φ represents the feature X with z f feature subset, E represents the error category, minR2(Φ), R2 represents the minimum redundancy, x, y represent two continuous random variables, p(x) represents the probability density of variable x, p(y) represents the probability density of variable y, and p(x, y) represents the joint probability density of variables x and y.

7. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 6, characterized in that: The Gaussian mixture model is: Among them, N(X;μ X ,Σ) represents the Gaussian distribution function, μ X represents the mean of sample X, Represents μ X is an unbiased estimate of , z represents the feature dimension of sample X, Σ represents the covariance matrix of sample X, and q represents the number of image samples.

8. The steering knuckle installation detection and identification method based on visual image recognition as claimed in claim 7, characterized in that: The training and learning specifically includes the following steps: Mark w known image feature samples and mix them with u unknown image feature samples to obtain a mixed feature sample set X (w+u)×z ; A semi-supervised Gaussian mixture model is introduced to fit the unqualified image feature samples, and the probability and likelihood function of the mixed feature samples belonging to a certain warning level are calculated; The expectation maximization algorithm is used to iterate until the change between the ε-th step and the ε+1-th step in the likelihood function is less than the error value threshold.

Images