A material recognition method based on sound signal and frequency domain image features

Abstract

The present invention is a material recognition method based on sound signals and frequency domain image features, which overcomes the problems of the existing material recognition methods in the art that they can damage objects, usually rely on expensive equipment, and cannot be integrated into portable devices. The present invention can extract the same feature to identify both solid materials and liquid materials, and can simultaneously identify solid and liquid materials without sacrificing recognition performance. The present invention includes the following steps: step one, data collection; step two, (1) drawing time-frequency domain feature images; (2) extracting image features; step three, (1) selecting a suitable time-frequency domain feature image combination; (2) using ECOC to obtain a classification model; step four, using error correction output coding combined with a majority voting method to perform material recognition.

Description

Technical field:

[0001] The present invention belongs to the technical field of material recognition, and relates to a material recognition method based on sound signals and frequency domain image features. Background technology:

[0002] Material identification plays a vital role in factories, security inspections, and our daily lives. Current material identification methods rely on contact with objects to identify their materials. For example, Andrew Hind et al. used an instrument to atomize an object and analyzed the resulting spectrum to determine the content of a specific material in an ore. Agilent Technologies inserted an instrument probe into a liquid to be tested and identified different liquid materials by measuring the dielectric constant. However, these methods require specialized equipment and specialized personnel, and they can damage the object. Consequently, research has been underway in recent years on non-contact material identification methods based on wireless signals that do not damage the object. For example, TagScan identifies similar liquids by measuring the phase of the RF signal as it passes through the liquid and the change in received signal strength. WiMate identifies materials based on the effect of an object on the amplitude and phase of the Wi-Fi signal. However, these methods typically rely on expensive equipment and cannot be integrated into portable devices.

[0003] The contact material identification method utilizes the differences in the physical and chemical properties of different materials, uses instruments and equipment to conduct qualitative or quantitative analysis, and then identifies the target material. There are two methods: (1) using atomic absorption spectroscopy to identify materials; (2) using dielectric constant to identify materials. However, using this method for material identification means that the object used for identification will be completely destroyed, and the cost is huge, making it unsuitable for long-term use. In addition, when performing material identification, the instrument needs to insert the probe into the liquid, which means that the probe needs to be frequently inspected and maintained, and the instrument cannot complete the material identification of corrosive and conductive liquids.

[0004] Currently, the more popular non-contact material recognition methods include the following: (1) material recognition based on radio frequency signals; (2) material recognition based on ultra-wideband signals; (3) material recognition based on radar signals; and (4) material recognition based on WiFi signals. However, radio frequency signals are not widely used in daily life, and material recognition requires the pre-marking of each target, which wastes resources. Ultra-wideband systems occupy a large bandwidth and can interfere with other wireless communication systems during operation. Radar signals and WiFi signals are easily affected by environmental interference. Summary of the invention:

[0005] The present invention aims to provide a material recognition method based on sound signals and frequency domain image features. This method overcomes the problems of existing material recognition methods, which are destructive, often rely on expensive equipment, and cannot be integrated into portable devices. The present invention extracts the same features to identify both solid and liquid materials, and simultaneously recognizes both solid and liquid materials without sacrificing recognition performance.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A material recognition method based on sound signals and frequency domain image features, characterized by comprising the following steps:

[0008] Step 1: Data collection;

[0009] Step 2: (1) draw the time-frequency domain feature image; (2) extract the image features;

[0010] Step 3: (1) Select appropriate time-frequency domain feature image combination; (2) Use ECOC to obtain the classification model;

[0011] Step 4: Use error correction output coding combined with majority voting method to perform material identification.

[0012] In step 1, the built-in microphone and speaker of the mobile phone are used as the receiver and transmitter respectively; the speaker sends the sound signal, and the microphone receives the sound signal transmitted back through the target; the sound signal is collected through two channels, and the signal is displayed as two columns of data; let X(t), Y(t), and H(t) be the transmitted signal, received signal, and the CIR of the sound signal at time t, respectively, and the following formula is obtained:

[0013]

[0014] In step 2 (1), in the feature extraction stage, only the relatively stable part of the middle part of the acoustic signal is used; the original acoustic signal and various time-frequency domain features are extracted and plotted into an image.

[0015] In step 2 (2), HOG features are extracted: the three-channel 24-bit color image is converted into a single-channel grayscale image, and then the Otsu method is used to find a suitable threshold to convert the grayscale image into a binary image and then perform linear gradient calculation; the pixel p in the x row and y column is x,y The horizontal gradient g x and vertical gradient g y Expressed as:

[0016] g x =p x+1,y -p x-1,y

[0017] gy =p x,y+1 -p x,y-1

[0018] The gradient amplitude g is calculated according to the following formula x,y and direction θ:

[0019]

[0020]

[0021] Since there are 9 directions θ, the data of each cell is accumulated into 9 containers and normalized to obtain the HOG feature, which is recorded as: HOG = [a1, a2, ..., a n ];

[0022] In step 2 (2), GLCM features are extracted: first, the color image is converted into a gray image. GLCM describes the probability value of the change between the gray levels x and y at a specific angle and displacement; for a given distance d, the GLCM at θ = 0°, 45°, 90° and 135° is calculated, and recorded as: g(d, 0), g(d, 45), g(d, 90), g(d, 135); in order to prevent the occurrence of negative entropy values, the four GLCMs are normalized, and their contrast, energy, correlation and homogeneity are calculated, and they are combined into a 4×4 matrix. The calculation formula is:

[0023]

[0024]

[0025]

[0026]

[0027] Where p(x,y) represents the frequency of each element, α i and α j represents the mean value, β i and β j Represents the mean square error; Next, the mean and standard deviation of each column of the matrix are calculated and combined into the final GLCM feature, which is expressed as: GLCM=[b1,b2,...,b m ]; Finally, the HOG and GLCM features are combined as the feature of material recognition, expressed as: Feature = [a1, a2, ..., a n ,b1,b2,...,b m ].

[0028] In step three (1), the time-frequency domain features extracted above are plotted into an image, and the image features are extracted and the average accuracy of material recognition is obtained using ECOC; the first three frequency domain feature maps are selected as the combined extracted image features, and the remaining feature image combinations are used in combination with the majority voting method.

[0029] In step three (2), the ECOC learning method is used for material recognition. A separate classifier is constructed for each image in the selected image combination. Three classifiers are constructed using the ECOC learning method. Each ECOC classifier uses a "one-to-one" encoding matrix to encode the image features. Then, a binary classifier is trained for every two labels in the i labels, which is called a base classifier. Then, the image features are input into all classifiers to train a classification model.

[0030] In step 4, the CIR of the test data is calculated and plotted into a frequency domain image, and then the image features are extracted. Then, the extracted features are input into the classification model obtained using ECOC to predict the code of the test data, and finally it is decoded to obtain the predicted label.

[0031] Compared with the prior art, the present invention has the following advantages and effects:

[0032] 1. The non-contact material recognition method based on sound signals of the present invention can be used to recognize solid materials and liquid materials. The method has high accuracy and strong robustness in various environments. At the same time, the present invention can recognize solid materials and liquid materials at the same time, and when recognizing solid and liquid materials at the same time, it still has good recognition performance. It can also identify whether there is liquid residue in the cup. The present invention can be used for subway security inspection and garbage sorting; it can help the daily life of visually impaired people. Moreover, the method only uses the built-in microphone and speaker of the mobile phone as the receiver and transmitter, which can be met by current smart phones. There is no need to carry additional equipment, it is very portable, and is not restricted by light, which is convenient for users to use;

[0033] 2. The difference between the present invention and previous material identification methods is that previous methods damage the object and are affected by the surrounding environment. The present invention's method does not touch the object when identifying the material, so it does not damage the object to be tested. This method is also unaffected by light, temperature, etc., and its performance remains stable in environments with different multipath intensities. A smartphone is used to collect the sound signal of the object to be tested, and the received sound signal is used to identify the material of the object to be tested;

[0034] 3. The accuracy of the present invention for solid and liquid material recognition is approximately 90.4% and 96.7%, respectively. The average precision and F-value for solid material recognition are 91.1% and 90.3%, respectively, while the average precision and F-value for liquid material recognition are 96.9% and 96.7%, respectively. Furthermore, the accuracy, precision, and F-value for empty cup recognition are all above 99.2%. Description of the drawings:

[0035] Figure 1 It is the CIR frequency domain diagram under different conditions of the present invention;

[0036] Figure 2 The device deployment diagram of the present invention includes: (a) deployment at different distances between the transceiver and the receiver; (b) deployment at different angles between the transceiver and the receiver; and (c) deployment in different multipath environments.

[0037] Figure 3 This is a histogram of material recognition accuracy under different frequency domain feature images of the present invention;

[0038] Figure 4 This is the overall performance diagram of the present invention, (a) solid material recognition performance (b) liquid material recognition performance (c) empty cup recognition performance. Specific implementation method:

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] This paper presents a material recognition method based on sound signals and frequency domain image features. An experiment was designed to study the relationship between CIR and different materials. The following conclusions were observed through the experiment:

[0041] 1. Different materials produce different CIR images

[0042] This paper studies the frequency domain images of solids and liquids of different materials based on the sound signal CIR. The results are as follows: Figure 1 The study found significant differences in frequency and STFT images from different solid and liquid materials, demonstrating that this information can be used to distinguish materials.

[0043] 2. Different mobile phone models will not affect CIR

[0044] In order to verify whether different mobile phone models affect the identification of materials, the author designed a comparative experiment on three different mobile phones: Redmi K30 5G, Mi6x, and vivo Y97, collected the sound signals of the same material, and drew the CIR frequency domain feature map. The experiment found that there are differences in the frequency domain feature maps of different materials under the same mobile phone, such as Figure 1 This shows that the method is universal across different mobile phones.

[0045] 3. The surrounding environment will not affect CIR

[0046] In the process of collecting sound signals, the signal collection environment will always change. The multipath environment is sometimes weak and sometimes strong, which leads to different CIR measurement results. In order to explore this problem, the present invention artificially increases the number of objects around the experiment to change the multipath environment. Figure 2 Then observe the CIR frequency domain graph of the same material in different multipath environments, as shown in (c). Figure 1 The experiment found that there is no significant difference between the images under different multipath environments, indicating that this method is not affected by the multipath environment.

[0047] Example:

[0048] The material recognition method based on sound signals and frequency domain image features of the present invention comprises the following steps:

[0049] Step 1: Data Collection

[0050] The present invention uses the built-in microphone and speaker of the mobile phone as the receiving end and the transmitting end respectively. The speaker sends the sound signal, and the microphone receives the sound signal transmitted back through the target. The present invention studied three types of solid and liquid materials to prove the feasibility of the design of the present invention, and collected 45 groups of sound signals of different materials. When collecting sound signals, each group of signals was recorded for about 8 seconds. The sound signals were saved in WAV (waveform audio) files for later feature extraction. Since the sound signals are collected through two channels, the signals are displayed as two columns of data. Let X(t), Y(t), H(t) be the transmitted signal, the received signal, and the CIR of the sound signal at time t, respectively, and the following formula can be obtained:

[0051]

[0052] Step 2: Feature Extraction

[0053] (1) Draw time-frequency domain feature images

[0054] During the feature extraction stage, the present invention only uses the relatively stable middle portion of the acoustic signal. When time-frequency domain features such as frequency, short-term energy, and MFCC are plotted as images, the differences between different materials are very obvious. Therefore, after plotting the time-frequency domain features as images, the present invention extracts image features as features for material identification. The present invention extracts the original acoustic signal and various time-frequency domain features and plots them as images, as shown in Table 1 below.

[0055] Table 1 Time-frequency domain characteristics

[0056]

[0057] (2) Extracting image features

[0058] The present invention extracts HOG features and GLCM features. First, the HOG features are extracted. The present invention converts the three-channel 24-bit color image into a single-channel grayscale image. Then, the Otsu method is used to find a suitable threshold to convert the grayscale image into a binary image and then perform linear gradient calculation. The HOG feature describes the geometric distribution of edges within a window containing 256×256 pixels. The window is divided into non-overlapping 4×4 pixel blocks, which are called cells. In addition, a block consists of 2×2 cells and overlaps with the adjacent block by 1 cell, so there are a total of 63×63 blocks in the window. x row y column pixel p x,y The horizontal gradient g x and vertical gradient g y Expressed as:

[0059] g x =p x+1,y -p x-1,y

[0060] g y =p x,y+1 -p x,y-1

[0061] Next, the present invention calculates the gradient amplitude g according to the following formula x,y and direction θ:

[0062]

[0063]

[0064] Since there are 9 directions θ, the data of each cell is accumulated into 9 containers and normalized to obtain the HOG feature, which is recorded as: HOG = [a1, a2, ..., a n ].

[0065] The present invention then extracts GLCM features. The present invention also converts the color image into a gray image first. GLCM describes the probability value of the change between the grayscale levels x and y at a specific angle and displacement. For a given distance d, the GLCMs at θ = 0°, 45°, 90°, and 135° are calculated and recorded as: g(d,0), g(d,45), g(d,90), g(d,135). Next, in order to prevent the occurrence of negative entropy values, the present invention normalizes the four GLCMs and calculates their contrast, energy, correlation, and homogeneity, and combines them into a 4×4 matrix. The calculation formula is:

[0066]

[0067]

[0068]

[0069]

[0070] Where p(x,y) represents the frequency of each element, α i and α j represents the mean value, β i and β j Next, the present invention calculates the mean and standard deviation of each column of the matrix and combines them into the final GLCM feature, which is expressed as: GLCM = [b1, b2, ..., b m Finally, the present invention combines HOG and GLCM features as the feature of material recognition, which is expressed as: Feature = [a1, a2, ..., a n ,b1,b2,...,b m ].

[0071] Step 3: Use ECOC to get the classification model

[0072] (1) Select appropriate time-frequency domain feature image combination

[0073] In order to determine which time-frequency domain feature image combination performs better in the majority voting method, the present invention plots the above-extracted time-frequency domain features into an image, extracts the image features and uses ECOC to obtain the average accuracy of material recognition, such as Figure 3 As shown in Table 2 (only the features with solid and liquid recognition accuracy above 70% are shown, and the feature explanations are shown in Table 2). Experiments show that the image feature recognition accuracy based on frequency domain feature maps is better than that based on time domain feature maps. Finally, the present invention selects the first three frequency domain feature maps as a combination to extract image features. Of course, the remaining feature image combinations can also be used in combination with the majority voting method to improve the recognition accuracy to a certain extent.

[0074] Table 2 Interpretation of time-frequency domain feature images

[0075]

[0076] (2) Using ECOC to obtain the classification model

[0077] The present invention adopts ECOC learning method for material recognition, which is a multiple binary classification algorithm of support vector machine, and can convert multi-classification problems into multiple binary classification problems for solution. The steps of solving the problem are encoding, training and decoding.

[0078] To maximize the advantages of the selected frequency-domain feature image combination, the present invention constructs a separate classifier for each image in the selected image combination. Specifically, the present invention uses the ECOC learning method to build three classifiers. In this method, each ECOC classifier encodes the image features using a one-to-one encoding matrix. Next, a binary classifier, called a base classifier, is trained for every two labels in the i-th set. The image features are then input into all classifiers to train and obtain a classification model.

[0079] Step 4: Use ECOC and majority voting method to identify materials

[0080] The CIR of the test data is calculated and plotted as a frequency domain image, followed by image feature extraction. The extracted features are then input into a classification model derived using ECOC to predict the code for the test data, which is then decoded to obtain the predicted label. Because the present invention constructs three classifiers, it obtains three sets of predicted labels. Finally, the present invention calculates the majority class label of all predicted labels using a majority voting method to obtain the final predicted result, completing the material identification of solids and liquids.

[0081] Experimental example:

[0082] Experimental setup:

[0083] 1. Device deployment: There are two types of device deployment in this invention, such as Figure 2 (a) In the first case, the transceiver is parallel to the horizontal direction and maintains a distance of 1 cm (dt) from the object to be measured, and maintains a height of 3 cm (dh) from the target placement plane. In the second case, the transceiver is perpendicular to the horizontal direction. In this case, the transceiver is perpendicular to the object to be measured and maintains a distance of 1 cm. During the data acquisition phase, the present invention places the object to be measured on a plane, and the position of the object, the position of the transceiver, and the surrounding environment remain fixed.

[0084] 2. Experimental setup: This paper uses three different solid materials (ceramic, glass, and iron) and liquid materials (water, milk tea, and green tea) to evaluate the method of this invention. In order to verify the feasibility of the system, this paper designed three types of experiments, including solid and liquid material recognition and empty cup recognition. In addition, this paper designed three angles of 45°, 90°, and 135° in the experiment to evaluate the effect of the angle of the transmitting and receiving end on material recognition, such as Figure 2 (b) At the same time, the present invention also sets up an experiment to verify the robustness of the present invention by changing the distance dt, and the distances include: 1cm, 2cm, 3cm, 4cm and 5cm. In addition, in order to study the impact of different multipath environments on the present method, the present invention collects sound signals of three multipath intensity environments for material recognition, including: weak multipath environment, medium multipath environment, and strong multipath environment, which are numbered as environment 1, environment 2 and environment 3 respectively. Figure 2 (c) shown.

[0085] 3. Parameter setting: The sampling rate of the sound signal sent by the transmitter is set to 48 kHz. Since the sampling rate of the built-in microphone and speaker in current smart phones is usually 48 kHz, the sampling rate set in the present invention is suitable for most mobile phones.

[0086] 4. Evaluation Method: This paper compares the performance of methods with and without majority voting, and with and without image feature extraction. It also compares the classification method of this paper with commonly used classification algorithms. Furthermore, to describe the performance of this method, this paper considers several evaluation metrics, such as accuracy, precision, and F-score. Accuracy refers to the proportion of true positive classes. The F-score combines the results of precision and recall.

[0087] Experimental results

[0088] The present invention averages the results of material recognition and empty cup recognition under different mobile phone models to obtain the final overall performance. The results are as follows:

[0089] 1) Identification performance of solid and liquid materials such as Figure 4 As shown in the figure, the accuracy of solid and liquid material recognition is approximately 90.4% and 96.7% respectively. Among them, the average precision and F value of solid material recognition are 91.1% and 90.3% respectively, while the average precision and F value of liquid material recognition are 96.9% and 96.7% respectively. In addition, the performance of empty cup recognition is also as shown in the figure. Figure 4 As shown, the accuracy, precision and F-value are all higher than 99.2%.

[0090] 2) Compared to other material recognition methods, this method is highly robust and unaffected by changes in lighting and ambient conditions. Furthermore, it only requires a microphone and speaker capable of transmitting and receiving audio signals during both material and empty cup recognition, requirements currently met by smartphones. This makes the method highly portable.

[0091] The present invention designs several sets of experiments, uses two-fold cross validation to evaluate the stability of the method, and calculates the average value of multiple runs as the final recognition result. In addition, in order to ensure that the training data and test data do not overlap, the data set is randomly divided into two equal parts.

[0092] Experimental results show that the recognition accuracy of this method for solid and liquid materials exceeds 90.45% and 96.72% respectively, and the recognition accuracy for empty cups is as high as 99.21%. Figure 4 As can be seen from the figure, this method achieves higher recognition accuracy for liquid materials than for solid materials. This may be due to two reasons. First, sound signals reflect, refract, and attenuate differently in solids and liquids, leading to different recognition results. Furthermore, the extracted features are more suitable for liquid materials, which may result in lower recognition results for solid materials.

[0093] The present invention also designed experiments to verify the effectiveness of this method. The experiments were conducted with and without majority voting and image features to evaluate their impact on performance. The recognition model used in this invention was then compared with commonly used recognition models, using the vivo Y97 mobile phone as an example. The experimental results showed that the majority voting method and image feature extraction resulted in higher recognition performance for material recognition. Furthermore, compared with other learning methods, ECOC is more suitable for material recognition.

[0094] The present invention also designed an experiment to study whether different mobile phone models, surrounding environments, and different distances and angles between the transmitter and receiver have an impact on this method. The experimental results show that there is no significant impact. The following are the conclusions observed through the experiment:

[0095] a) Different models of mobile phones have good recognition performance

[0096] To study the impact of mobile phones on the performance of this method, we designed three different mobile phone models to collect sound signals and evaluate material recognition performance under the same environment and device configuration. The results showed that this method achieved high recognition performance on all mobile phones.

[0097] b) The surrounding environment will not affect the material recognition performance

[0098] To study the impact of the surrounding environment on the performance of this method, the present invention collected sound signals under three different multipath conditions for material recognition. The experiment found that the recognition performance of this method remained stable under different multipath conditions.

[0099] c) The deployment of the transmitter and receiver at different angles will not affect the material recognition performance

[0100] In this experiment, we trained and tested models for different transceiver deployment angles. The results show that recognition accuracy at the LOS angle is better than at the other two angles, with little degradation in material recognition accuracy at the other two angles.

[0101] d) Deploying the transmitter and receiver at different distances will not affect the material recognition performance

[0102] The experiment used five distances, ranging from 1 cm to 5 cm, to evaluate the effect of the distance (dt) between the transmitter and receiver and the object on material recognition. The experiment found no significant change in material recognition performance at different distances.

[0103] Definitions of abbreviations and key terms used in this invention:

[0104] Error Correcting Output Codes (ECOC)

[0105] Channel Impulse Response CIR (Channel Impulse Response)

[0106] Line of sight (LOS) transmission of sound signals

[0107] Histogram of Oriented Gradient (HOG)

[0108] Gray-level Co-occurrence Matrix (GLCM)

[0109] Short-time Fourier transform STFT (Short-time Fourier transform)

[0110] Mel Frequency Cepstrum Coefficient MFCC

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any equivalent structural changes made using the contents of the description and drawings of the present invention should be included in the patent protection scope of the invention.

Claims

1. A material recognition method based on sound signals and frequency domain image features, characterized by: The following steps are involved: Step 1: Data collection; Step 2: (1) draw the time-frequency domain feature image; (2) Extract image features; Step 3: (1) Select appropriate time-frequency domain feature image combination; (2) Use ECOC to obtain the classification model; Step 4: Use error-correcting output coding combined with majority voting method to perform material identification; In step 3 (1), the time-frequency domain features extracted above are plotted into an image, and the image features are extracted and the average accuracy of material recognition is obtained using ECOC; The first three frequency domain feature maps are selected as a combination to extract image features, and the remaining feature image combinations are used in conjunction with the majority voting method; In step 3 (2), the ECOC learning method is used for material recognition. A separate classifier is constructed for each image in the selected image combination. Three classifiers are constructed using the ECOC learning method. Each ECOC classifier uses a "one-to-one" encoding matrix to encode the image features. Then, a binary classifier is trained for every two labels in the i labels, which is called a base classifier. Then, the image features are input into all classifiers to train a classification model. In step 4, the CIR of the test data is calculated and plotted into a frequency domain image, and then the image features are extracted. Then, the extracted features are input into the classification model obtained using ECOC to predict the code of the test data, and finally it is decoded to obtain the predicted label.

2. The material recognition method based on sound signals and frequency domain image features according to claim 1, characterized in that: In step 1, the built-in microphone and speaker of the mobile phone are used as the receiver and transmitter respectively; the speaker sends the sound signal, and the microphone receives the sound signal transmitted back through the target; the sound signal is collected through two channels, and the signal is displayed as two columns of data; let X(t), Y(t), and H(t) be the transmitted signal, received signal, and the CIR of the sound signal at time t, respectively, and the following formula is obtained:

3. The material recognition method based on sound signals and frequency domain image features according to claim 1, characterized in that: In step 2 (1), in the feature extraction stage, only the relatively stable part of the middle part of the acoustic signal is used; the original acoustic signal and various time-frequency domain features are extracted and plotted into an image.

4. The material recognition method based on sound signals and frequency domain image features according to claim 1, characterized in that: In step 2 (2), HOG features are extracted: the three-channel 24-bit color image is converted into a single-channel grayscale image, and then the Otsu method is used to find a suitable threshold to convert the grayscale image into a binary image and then perform linear gradient calculation; the pixel p in the x row and y column is x,y The horizontal gradient g x and vertical gradient g y Expressed as: g x =p x+1,y -p x-1,y g y =p x,y+1 -p x,y-1 The gradient amplitude g is calculated according to the following formula x,y and direction θ: Since there are 9 directions θ, the data of each cell is accumulated into 9 containers and normalized to obtain the HOG feature, which is recorded as: HOG = [a1, a2, ..., a n ]; In step 2 (2), GLCM features are extracted: first, the color image is converted into a gray image. GLCM describes the probability value of the change between the gray levels x and y at a specific angle and displacement; for a given distance d, the GLCM at θ = 0°, 45°, 90° and 135° is calculated, and recorded as: g(d, 0), g(d, 45), g(d, 90), g(d, 135); in order to prevent the occurrence of negative entropy values, the four GLCMs are normalized, and their contrast, energy, correlation and homogeneity are calculated, and they are combined into a 4×4 matrix. The calculation formula is: Where p(x,y) represents the frequency of each element, α i and α j represents the mean value, β i and β j Represents the mean square error; Next, the mean and standard deviation of each column of the matrix are calculated and merged into the final GLCM feature, which is expressed as: GLCM=[b1,b2,...,b m ]; Finally, the HOG and GLCM features are combined as the feature of material recognition, expressed as: Feature = [a1, a2, ..., a n ,b1,b2,...,b m ].

Images