Material identification method and related device

Abstract

The invention provides a material identification method and a related device, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a detection signal obtained through the signal detection of a to-be-identified object; carrying out material recognition on the detection signal through a material recognition model, and obtaining a predicted material category with the maximum prediction probability of the to-be-recognized object and the corresponding credibility; if the credibility is greater than a credibility threshold, determining the predicted material category as the identified material category of the to-be-identified object, the determination mode of the credibility threshold comprising: performing material identification on verification samples in a verification set through a material identification model to obtain the verification sample material category with the maximum prediction probability and the corresponding credibility, verifying the credibility of the sample material category, wherein the credibility comprises a first credibility of correct classification and a second credibility of wrong classification; and determining a credibility threshold based on the first credibility and the second credibility. According to the scheme of the embodiment, the accuracy of the finally determined material category of the to-be-identified object is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a material identification method and related device, and the related device includes a material identification device, a computer device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] In the field of interactive tablets, in order to meet more efficient interactive experience, there is a demand for identifying the material of an object, such as identifying the interactive material of an interactive tablet. Through automatic identification of the material of an object, more efficient interaction can be achieved, and manual operation can be reduced. When automatically identifying the material of an object, a sensor is used to collect a characteristic signal that can reflect the characteristics of the object, and then material identification is performed based on the collected characteristic signal. However, in actual application scenarios, there may be situations where the sensor is triggered by mistake. For example, taking an elastic wave sensor as an example, when the external noise is strong, the elastic wave sensor is easily excited by the noise to generate an elastic wave signal, thereby causing misidentification of the material. Therefore, there is a demand for further improving the accuracy of the material identification result. SUMMARY

[0003] Therefore, it is necessary to provide a material identification method and related device capable of improving the accuracy of the material identification result, and the related device includes a material identification device, a computer device, a computer readable storage medium and a computer program product.

[0004] In a first aspect, the present application provides a material identification method, wherein the method comprises:

[0005] obtaining a detection signal obtained by signal detection on a to-be-identified object;

[0006] performing material identification on the detection signal through a material identification model to obtain a predicted material category with the maximum prediction probability of the to-be-identified object and a confidence level corresponding to the predicted material category;

[0007] in a case where the confidence level is greater than a confidence level threshold, determining the predicted material category as a material category of the to-be-identified object identified;

[0008] wherein the determination manner of the confidence level threshold comprises:

[0009] performing material identification on a verification sample in a verification set through the material identification model to obtain a verification sample material category with the maximum prediction probability and a corresponding confidence level, the confidence level of the verification sample material category including a first confidence level of correct classification of the verification sample material category and a second confidence level of incorrect classification of the verification sample material category;

[0010] determine the confidence threshold based on the first confidence and the second confidence.

[0011] Based on the material identification method of the embodiments of the present application as described above, in the process of material identification, after obtaining the detection signal corresponding to the to-be-identified object and determining the predicted material category with the maximum prediction probability based on the material identification of the detection signal, the confidence of the predicted material category is obtained. In the case where the confidence is greater than the confidence threshold, the predicted material category with the maximum prediction probability is taken as the material category of the to-be-identified object. Therefore, the predicted material category with the maximum prediction probability of the detection signal is further determined in combination with the confidence, which effectively improves the accuracy of the finally determined material category of the to-be-identified object. Moreover, the confidence threshold used for comparing the confidence is determined by the material identification model based on the first confidence corresponding to the verification sample with correct material category classification and the second confidence corresponding to the verification sample with incorrect material category classification after the material identification of the verification sample in the verification set. Therefore, the confidence threshold is determined based on the verification result of the verification sample in the verification set, which reflects the verification situation of the material classification model in the verification set, improves the accuracy of the obtained confidence threshold, and can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0012] In some embodiments, the determination of the confidence threshold based on the first confidence and the second confidence comprises:

[0013] determining the average value and mean square deviation of the first confidence of the verification sample in the verification set, and the average value and mean square deviation of the second confidence of the verification sample in the verification set;

[0014] calculating the confidence threshold based on the average value and mean square deviation of the first confidence, and the average value and mean square deviation of the second confidence.

[0015] Based on the embodiment, based on the confidence of the verification sample in the verification set, the average value and mean square deviation of the confidence of the verification sample with correct material category classification and the verification sample with incorrect material category classification are calculated respectively, and the confidence threshold is calculated and determined accordingly. When the confidence threshold is determined based on the verification result of the verification sample in the verification set, the average value and mean square deviation are calculated and evaluated comprehensively from two dimensions, which can further improve the accuracy of the determined confidence threshold and can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0016] Since the average value represents the central tendency of a group of data, and the mean square deviation represents the dispersion degree of a group of data, the combination of the average value and the mean square deviation to comprehensively evaluate the reliability can better represent the comprehensive characteristics of the reliability of the verification sample with correct material category classification and the comprehensive characteristics of the reliability of the verification sample with incorrect material category classification, and can further improve the accuracy of the determined reliability threshold, and can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0017] In some embodiments, the reliability threshold is a weighted sum of a first difference value and a first sum value, the first difference value is a difference between the average value and the mean square deviation of the first reliability, and the first sum value is a sum of the average value and the mean square deviation of the second reliability.

[0018] Based on this embodiment, when determining the reliability threshold, the weighted sum of the difference between the average value and the mean square deviation of the first reliability and the sum of the average value and the mean square deviation of the second reliability is determined, so that the difference between the average value and the mean square deviation of the reliability corresponding to the verification sample with correct material category classification is considered, and the comprehensive influence of the average value and the mean square deviation of the reliability corresponding to the verification sample with incorrect material category classification is considered, and the accuracy of the obtained reliability threshold is further improved. By calculating the difference between the average value and the mean square deviation of the first reliability, since the mean square deviation represents the dispersion degree of a group of data, the influence of the dispersed samples in the verification sample with correct material category classification is reduced, and for the verification sample with incorrect material category classification, by calculating the sum of the average value and the mean square deviation of the second reliability, the central tendency of the verification sample with incorrect classification is considered, and the influence of the dispersion is also considered, and the accuracy of the obtained reliability threshold is further improved, which can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0019] In some embodiments, the reliability threshold is the average value of the first difference value and the first sum value, that is, the average value of the difference between the average value and the mean square deviation of the first reliability and the sum of the average value and the mean square deviation of the second reliability.

[0020] In some embodiments, the method further comprises:

[0021] obtaining the material identification model to perform material identification on the instance signal, and obtaining the instance material category and the corresponding reliability;

[0022] filtering the target reliability from the reliability of the instance material category;

[0023] update the confidence threshold based on the average and mean square deviation of the target confidence.

[0024] Based on this embodiment, after the material recognition model deployment application is applied, the target confidence is filtered out from the confidence of the instance material category based on the result of the material recognition of the instance signal by the material recognition model within the preset time period, and the confidence threshold is updated in combination with the average and mean square deviation of the target confidence, so that the confidence threshold can be updated in combination with the actual application result after the material recognition model deployment application, which can further improve the accuracy of the determined confidence threshold, and on this basis, can help to further improve the accuracy of the finally determined material category of the to-be-recognized object.

[0025] In some embodiments, the material recognition model comprises a feature mapping module, a material classification module and a confidence recognition module connected in sequence.

[0026] Based on this embodiment, material recognition can be performed by the material recognition model comprising the feature mapping module, the material classification module and the confidence recognition module.

[0027] In some embodiments, the confidence recognition module comprises a Relu layer and a Tanh layer connected in sequence.

[0028] Based on this embodiment, the range of the confidence is limited by the Tanh layer based on the calculation of the confidence by the Relu layer, which helps to set the range of the confidence threshold accordingly and is conducive to deployment application.

[0029] In some embodiments, the detection signal obtained by signal detection on the to-be-recognized object comprises:

[0030] obtaining an elastic wave signal output by a multi-channel elastic wave sensor;

[0031] converting the effective signal in the multi-channel elastic wave signal into a frequency domain signal;

[0032] fusing the multi-channel frequency domain signals to obtain the detection signal.

[0033] Based on this embodiment, the detection signal obtained by signal detection on the to-be-recognized object is a signal obtained by fusing the multi-channel frequency domain signals after converting the effective signal in the multi-channel elastic wave signal into a frequency domain signal, so that the final obtained detection signal takes into account both the effective signal in the multi-channel elastic wave signal and the fusion performance of the multi-channel, and using the detection signal obtained on this basis for material recognition can improve the accuracy of the material recognition result.

[0034] In some embodiments, the valid signals in the multi-channel elastic wave signals include elastic wave signals with signal amplitudes greater than a minimum valid signal threshold and less than a maximum valid signal threshold.

[0035] Therefore, whether an elastic wave signal is valid can be determined based on the signal amplitude of the elastic wave signal. If the signal amplitude is less than the minimum valid signal threshold, it indicates that the energy of the elastic wave signal is too low. If the signal amplitude is greater than the maximum valid signal threshold, it indicates that the elastic wave signal has amplitude clipping distortion. Therefore, by determining elastic wave signals with signal amplitudes greater than the minimum valid signal threshold and less than the maximum valid signal threshold as valid signals, valid signals in the multi-channel elastic wave signals can be conveniently identified and analyzed.

[0036] In some embodiments, the fusing of the multi-channel frequency domain signals to obtain the detection signal includes performing weighted average processing on the multi-channel frequency domain signals to obtain the detection signal.

[0037] Based on this embodiment, the detection signal is obtained by performing weighted average processing on the multi-channel frequency domain signals, which reduces the influence of signal differences between elastic wave sensors on material classification and improves the stability of the detection signal.

[0038] In some embodiments, the fusing of the multi-channel frequency domain signals to obtain the detection signal includes obtaining a system function of vibration propagation of the sensors and fusing the multi-channel frequency domain signals based on the system function of each sensor to obtain the detection signal.

[0039] Based on this embodiment, when fusing the multi-channel frequency domain signals, the multi-channel frequency domain signals are fused based on the system function of vibration propagation of each sensor in the multi-channel elastic wave sensors to obtain the detection signal, which effectively considers the vibration propagation performance of each sensor and improves the accuracy of the obtained detection signal.

[0040] In some embodiments, the first layer of the material identification model includes a convolution layer, and the fusing of the multi-channel frequency domain signals to obtain the detection signal includes fusing the multi-channel frequency domain signals through the convolution layer of the first layer of the material identification model to obtain the detection signal, where one channel of the convolution layer corresponds to one frequency domain signal.

[0041] Based on this embodiment, a convolution layer is arranged at the first layer of the material identification model, and one channel of the convolution layer corresponds to one frequency domain signal, so that the fusing of the frequency domain signals can be realized through the material identification model, improving the convenience of fusing the frequency domain signals.

[0042] In some embodiments, after obtaining the detection signal, the detection signal is preprocessed before material recognition, and the preprocessing includes normalization processing.

[0043] Based on this embodiment, after obtaining the detection signal, the detection signal is preprocessed, such as normalization, to reduce the influence of noise, which helps to further improve the accuracy of the final determined material category of the object to be recognized.

[0044] In some embodiments, the fusion of the multi-channel frequency domain signals to obtain the detection signal includes:

[0045] Obtain the touch information corresponding to the elastic wave signal;

[0046] According to the touch information corresponding to the elastic wave signal, determine the touch weight information of each elastic wave signal;

[0047] Based on the touch weight information of each elastic wave signal, the frequency domain signals corresponding to each elastic wave signal are weighted and fused to obtain the detection signal.

[0048] Based on this embodiment, when fusing the multi-channel elastic wave signals, the touch weight information of each elastic wave signal is determined by using the touch information corresponding to the elastic wave signal, and the frequency domain signals are weighted and fused based on the touch weight information, so that the relevant touch information of the touch obtained from the elastic wave information can be well utilized, thereby improving the accuracy of the obtained detection signal. On this basis, the accuracy of material recognition can be further improved.

[0049] In some embodiments, the material recognition model includes a touch information processing module, a feature mapping module, a material classification module, and a credibility recognition module connected in sequence;

[0050] The touch information processing module is configured to determine the touch weight information of each elastic wave signal based on the touch information corresponding to the multi-channel elastic wave signal;

[0051] The feature mapping module is configured to fuse the multi-channel elastic wave signals based on the touch weight information of each elastic wave signal to obtain the detection signal;

[0052] The material classification module is configured to perform material recognition on the detection signal to obtain a predicted material category with the maximum prediction probability of the object to be recognized;

[0053] The credibility recognition module is configured to determine the credibility corresponding to the predicted material category.

[0054] Based on the embodiment, by deploying a touch information processing module in the material identification model, the determination of the touch weight information can be realized by the touch information processing module, and on this basis, the elastic wave signals are weighted and fused, thereby improving the convenience of determining the touch weight information and the convenience of fusing the frequency domain signals.

[0055] In some embodiments, the touch information includes at least one of a touch position and a touch area, the touch weight information includes at least one of a position weight and an area weight, and the touch information processing module includes at least one of a position processing module and an area processing module.

[0056] The position processing module is configured to determine the position weight of each elastic wave signal based on the touch position corresponding to the multi-channel elastic wave signal.

[0057] The area processing module is configured to determine the area weight of each elastic wave signal based on the touch area corresponding to the multi-channel elastic wave signal.

[0058] Based on the embodiment, when determining the touch weight information of the elastic wave signal based on the touch information, both the position and the area of the touch are considered, the touch weight information of the elastic wave signal is determined from two dimensions of position and area, the accuracy of the determined touch weight information can be improved, thereby further improving the accuracy of the obtained detection information and the accuracy of material identification.

[0059] In a second aspect, the present application also provides a material identification device, which comprises:

[0060] A signal acquisition module is configured to acquire a detection signal obtained by performing signal detection on a to-be-identified object.

[0061] A material identification module is configured to perform material identification on the detection signal by using a material identification model, to obtain a predicted material category with the highest prediction probability of the to-be-identified object and a corresponding confidence level of the predicted material category.

[0062] A material determination module is configured to determine the predicted material category as a material category of the to-be-identified object if the confidence level is greater than a confidence level threshold.

[0063] A threshold determination module is configured to perform material identification on a verification sample in a verification set by using the material identification model, to obtain a verification sample material category with the highest prediction probability and a corresponding confidence level, wherein the confidence level of the verification sample material category includes a first confidence level of correct classification of the verification sample material category and a second confidence level of incorrect classification of the verification sample material category; and determine the confidence level threshold based on the first confidence level and the second confidence level.

[0064] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method in any of the above embodiments when executing the computer program.

[0065] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program implements the steps of the method in any of the above embodiments when executed by a processor.

[0066] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program, and the computer program implements the steps of the method in any of the above embodiments when executed by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 An application scenario diagram of the material identification method in an embodiment;

[0068] Figure 2 An application scenario diagram of the material identification method in another embodiment;

[0069] Figure 3 A flow diagram of the material identification method in another embodiment;

[0070] Figure 4 A flow diagram of determining the credibility threshold in an embodiment;

[0071] Figure 5 A flow diagram of updating the credibility threshold in an embodiment;

[0072] Figure 6 A model structure diagram of the material identification model in an embodiment;

[0073] Figure 7 A flow diagram of obtaining the detection signal in an embodiment;

[0074] Figure 8 A diagram of the elastic wave modalities collected by the elastic wave sensors at different positions in an embodiment;

[0075] Figure 9 A flow diagram of fusing the signals of multiple channels in an embodiment;

[0076] Figure 10 A principle diagram of identifying the material by the material identification model in combination with the touch position and the touch area in an example;

[0077] Figure 11A structural schematic diagram of a material identification model in one embodiment;

[0078] Figure 12 A structural schematic diagram of a material identification model in another embodiment;

[0079] Figure 13 A structural schematic diagram of a material identification model in another embodiment;

[0080] Figure 14 A structural schematic diagram of a material identification model in another embodiment;

[0081] Figure 15 A structural block diagram of a material identification device in one embodiment;

[0082] Figure 16 An internal structural diagram of a computer device in one embodiment;

[0083] Figure 17 An internal structural diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0084] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the protection scope of the present application.

[0085] The embodiments of the technical solutions of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0086] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein are only for the purpose of describing specific embodiments of the present application and are not intended to limit the present application; the terms “include” and “have” and any variations thereof in the specification and claims of the present application and the above description of drawings are intended to cover non-exclusive inclusion.

[0087] In the description of the embodiments of the present application, the technical terms “first”, “second”, etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of “a plurality of” is two or more, unless otherwise explicitly specified.

[0088] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all referring to a common embodiment, or an embodiment that is independent of other embodiments. One of ordinary skill in the art will recognize that the embodiments described herein can be combined with other embodiments in various ways.

[0089] In the description of embodiments of the present application, the term“a plurality of” refers to two or more (including two), and similarly, the term“a plurality of groups” refers to two or more groups (including two groups), and the term“a plurality of pieces” refers to two or more pieces (including two pieces).

[0090] At present, in the process of material identification, some possible ways are based on machine learning or deep learning. After analyzing the detection signal in the time domain and / or frequency domain, the signal features are extracted and classified. In the classification process, one of the preset materials is output as the classified result. However, in the technical scenario of material identification, there may be a situation that the sensor is triggered by mistake, that is, there are relatively many non-pre-set types of materials that cause the sensor to be triggered, thereby causing the misidentification of the material. For example, taking the elastic wave sensor as an example, when the external noise is strong, the elastic wave sensor is easily excited by the noise to generate an elastic wave signal, thereby causing the misidentification of the material. For example, in the application scenario of writing on an interactive tablet, if a non-pre-set material contacts the interactive tablet, it will cause the elastic wave sensor to be triggered by mistake, thereby generating an elastic wave signal, and on this basis, the material identification is easy to produce misidentification, and thus easy to cause misoperation of the tablet.

[0091] To solve this problem, it is found through research that the credibility of the identified material can be further evaluated after the material is identified. Only in the case that the identified material is credible, the identified material is taken as the final output. In the case that the material is identified by a material identification model, a related layer for calculating the credibility can be added after the layer responsible for material probability calculation of the material identification model, so as to determine whether the material with the highest probability is credible, thereby improving the accuracy of material identification.

[0092] Based on this, the embodiments of the present application provide a material identification method to improve the accuracy of material identification.

[0093] The material identification method provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the signal detection device 101 is used for detecting and outputting a sensing signal, the processor 102 is in communication connection with the signal detection device 101, so as to obtain a detection signal based on the sensing signal output by the signal detection device 101, and identify the material type of the object to be identified based on the detection signal. Among them, the signal detection device 101 and the processor 102 can be located in the same device, or can be located in different devices, and the signal detection device 101 can include one sensor or multiple sensors, wherein the sensor can be a single-channel sensor or a multi-channel sensor. Among them, the processor 102 can be a processor in any device that needs to identify the material of the object, such as an interactive tablet.

[0094] Taking the identification of the writing material of the interactive tablet as an example, the material identification method provided by the embodiment of the application can be applied to the application environment shown. Figure 2 The interactive tablet 1000 includes a display screen 100, an elastic wave detection device 200, and a processing device 300. Among them, the elastic wave detection device 200 and the processing device 300 are generally invisible on the appearance of the interactive tablet 1000. The interactive tablet 1000 can also be provided with a touch frame (not shown in the figure), which can be an infrared touch frame. Taking the infrared touch frame as an example, the infrared touch frame 300 is arranged around the interactive tablet 1000 to form a touch detection area of the interactive tablet 1000, and the four corners of the interactive tablet 1000 refer to the four edges of the interactive tablet 1000. In specific applications, the position of the infrared touch frame can be set according to the actual area to be detected, which can be arranged only around the display screen 100 or around the entire interactive tablet 1000. The touch frame and the elastic wave detection device 200 are in communication connection with the processing device 300.

[0095] The user can use a finger or a writing pen as a touch object to realize touch operation on the display screen of the interactive tablet by clicking or moving, and the touch frame 200 detects the touch operation of the touch object in the touch detection area, generates corresponding touch data and sends it to the processing device 300. After receiving the touch data, the processing device 300 responds according to the touch position represented by the touch data to realize the touch function.

[0096] As an implementation, the elastic wave detection device 200 can be arranged on the display screen 100 to detect the elastic wave signal generated when the touch object touches the display screen of the interactive panel. For example, the elastic wave detection device includes an elastic wave collection plate and at least one elastic wave sensor. The elastic wave sensor is arranged at the frame of the display screen or the inner side of the cover plate. When the touch object touches the display screen, an elastic wave signal with a characteristic is generated, which propagates from the contact point to the periphery of the display screen. The piezoelectric elastic wave sensor arranged at the frame of the display screen or the inner side of the cover plate can convert the elastic wave signal into a voltage signal and send the voltage signal to the elastic wave collection plate. The voltage signal is transmitted to the IC chip with a temperature compensation circuit through the elastic wave collection plate for amplification processing, and is converted into a digital elastic wave signal through an analog-to-digital conversion circuit. It should be noted that the position of the elastic wave detection device is not limited in the present scheme, as long as it can detect the elastic wave signal generated when the touch object touches the display screen of the interactive panel.

[0097] The elastic wave detection device 200 can be a multi-channel elastic wave sensor, i.e., an elastic wave sensor capable of detecting multiple elastic wave signals, or a collection of multiple elastic wave sensors, wherein the multiple different sensors can be arranged on the display screen 100 in a row-column arrangement or any other desired manner. After receiving the elastic wave signal output by the elastic wave detection device 300, the processing device 300 can identify the material category based on the elastic wave signal, i.e., identify the writing material category of the panel.

[0098] Reference Figure 3 The embodiments of the present application provide a material identification method, which is taken as an example of the processor 102 or the processing device 300. The method includes:

[0099] Step S301: obtaining a detection signal obtained by signal detection on a to-be-identified object.

[0100] The to-be-identified object refers to an object whose material needs to be identified. The detection signal refers to a signal corresponding to the to-be-identified object. The material of the to-be-identified object is associated with the characteristics of the detection signal. The material of the to-be-identified object that triggers the detection signal can be determined by analyzing the detection signal. For example, the detection signal can be an elastic wave signal detected and output by an elastic wave detection device, such as a multi-channel elastic wave signal. The to-be-identified object can be a touch object that touches the display screen and causes the elastic wave detection device to detect the elastic wave signal. The touch object can be a touch object of a preset material, or a touch object of a non-preset material that causes the elastic wave sensor to be triggered incorrectly.

[0101] Step S302: performing material identification on the detection signal by the material identification model to obtain a predicted material category with the maximum prediction probability of the to-be-identified object and a confidence degree corresponding to the predicted material category.

[0102] When performing material identification on the detection signal, the material identification can be performed based on the signal features of the obtained detection signal. Some possible ways of performing material identification on the detection signal can be performed by a material identification model trained by machine learning or deep learning. The material identification model trained by machine learning or deep learning extracts signal features from the detection signal and performs material identification based on the signal features.

[0103] Taking the material identification model performing material identification on the detection signal as an example, the material identification model can analyze the prediction probability of the material category of the detection signal as each preset material category. Generally, the material category with the maximum prediction probability can be taken as the material category of the to-be-identified object identified by the material identification model. Taking an example in which the preset material categories include two categories A and B, the material identification model can obtain the probability a1 of the material category of the to-be-identified object being category A and the probability b1 of the material category being category B. If a1 is greater than b1, category A will be taken as the predicted material category of the to-be-identified object. In the case where the preset material categories include more categories, the preset material category with the maximum probability will be taken as the predicted material category of the to-be-identified object based on the same principle. In some cases, the material identification model will take the preset material category with the maximum probability as the predicted material category of the to-be-identified object only when the maximum probability is greater than a probability threshold.

[0104] It should be understood that the preset material category can also include only one preset material category. In this case, the material identification model can obtain the probability of the material category of the to-be-identified object being the preset material category and take the preset material category as the predicted material category of the to-be-identified object when the probability is greater than a probability threshold.

[0105] In the embodiments of the present application, taking the material identification model performing material identification on the detection signal as an example, on the basis of obtaining the predicted material category with the maximum prediction probability of the to-be-identified object by the material identification model, the confidence degree corresponding to the predicted material category is also obtained. The confidence degree reflects the degree of confidence of the predicted material category.

[0106] Step S303: determining the predicted material category as the material category of the identified to-be-identified object when the confidence degree is greater than a confidence threshold.

[0107] When the confidence degree is greater than the confidence threshold, it indicates that the predicted material category is reliable, and thus the predicted material category can be determined as the material category of the identified to-be-identified object, i.e., the predicted material category is taken as the final material category of the identified to-be-identified object.

[0108] In some embodiments, if the credibility is less than or equal to the credibility threshold, it is considered that the current recognition result is unreliable, and the current recognition result can not be output or a prompt information can be output. The prompt information can be that the recognition result is wrong, the recognition result is unreliable, the recognition result is not a preset material category, a misoperation, etc., but is not limited thereto.

[0109] Based on the material recognition method of the embodiments of the present application, in the process of material recognition, after obtaining the detection signal corresponding to the object to be recognized and determining the predicted material category with the maximum prediction probability based on the material recognition of the detection signal, the credibility corresponding to the predicted material category is obtained. If the credibility is greater than the credibility threshold, the predicted material category with the maximum prediction probability is taken as the material category of the object to be recognized. Therefore, the predicted material category with the maximum prediction probability of the detection signal is further determined by combining the credibility, which effectively improves the accuracy of the final determined material category of the object to be recognized.

[0110] In some embodiments, the credibility threshold is determined in the following manner:

[0111] The material recognition model is used to analyze the detection signal to identify the material category of the object to be recognized that triggers the detection signal. Through the material recognition model, the predicted material category with the maximum prediction probability of the object to be recognized can be analyzed, and the credibility corresponding to the predicted material category is output. The credibility can be used to determine whether the predicted material category with the maximum prediction probability is reliable.

[0112] Based on the first credibility and the second credibility, the credibility threshold is determined.

[0113] The material recognition model is used to analyze the detection signal to identify the material category of the object to be recognized that triggers the detection signal. Through the material recognition model, the predicted material category with the maximum prediction probability of the object to be recognized can be analyzed, and the credibility corresponding to the predicted material category is output. The credibility can be used to determine whether the predicted material category with the maximum prediction probability is reliable.

[0114] The credibility threshold is used as a comparison standard to determine whether the predicted material category with the maximum prediction probability is reliable, and the material recognition result of the verification sample in the verification set can be determined by the material recognition model.

[0115] The training of the material identification model can be based on samples, each of which includes a detection signal of an object and a corresponding material classification label. After obtaining the samples, the samples can be divided into a training set, a validation set, and a test set. The material identification model to be trained is trained on the training set, and the hyperparameters of the material identification model are optimized on the validation set. For example, when the material identification of the validation samples in the validation set is performed, the predicted material category obtained by the material identification of the validation samples is compared with the material classification label of the validation samples, and thus the hyperparameters of the material identification model are determined, for example, the confidence threshold in the present application is determined. Generally, after the material identification model is optimized on the validation set, the optimized material identification model can be further trained in combination with the training set to obtain the trained material identification model.

[0116] Subsequently, the material identification model trained based on the training set and the validation set can be tested by the test set to determine whether the model performance meets the requirements, for example, whether the accuracy rate meets the preset accuracy threshold. If the accuracy rate of the material identification for the test set meets the preset accuracy threshold, the training is ended, and the final trained material identification model is obtained. The accuracy rate is the proportion of the number of test samples whose material identification result is correct to the number of samples in the test set.

[0117] Thus, in the embodiments of the present application, the confidence threshold can be determined comprehensively based on the verification of the validation samples in the validation set by the material identification model, in combination with the confidence of the correct classification of the material category of the validation samples (referred to as the first confidence in the embodiments of the present application) and the confidence of the incorrect classification of the material category of the validation samples (referred to as the second confidence in the embodiments of the present application).

[0118] Based on this embodiment, the confidence threshold used for comparing the confidence is determined based on the first confidence corresponding to the validation samples whose material category classification is correct and the second confidence corresponding to the validation samples whose material category classification is incorrect after the material identification of the validation samples in the validation set by the material identification model, so that the confidence threshold is determined based on the verification result of the validation samples in the validation set, which reflects the verification situation of the material classification model in the validation set, thereby improving the accuracy of the obtained confidence threshold and helping to further improve the accuracy of the finally determined material category of the object to be identified.

[0119] The manner of determining the confidence threshold based on the first confidence and the second confidence is not limited, and some examples are illustrated below.

[0120] In some embodiments, the reference Figure 4As shown, based on the first credibility and the second credibility, a credibility threshold is determined, comprising:

[0121] Step S401: determining the average value and the mean square deviation of the first credibility of the verification sample in the verification set, and the average value and the mean square deviation of the second credibility of the verification sample in the verification set.

[0122] Wherein, as described above, the first credibility refers to the credibility corresponding to the verification sample whose material category classification is correct when the material identification model performs material identification on the verification sample, therefore, the average value of the first credibility is the average value of the first credibility of the plurality of verification samples whose material category classification is correct, and the mean square deviation of the first credibility is the mean square deviation of the first credibility of the plurality of verification samples whose material category classification is correct.

[0123] Since the average value represents the central tendency of a group of data, and the mean square deviation reflects the dispersion degree of a group of data, therefore, the average value and the mean square deviation are combined to comprehensively evaluate the credibility, which can better represent the comprehensive characteristics of the credibility of the verification sample whose material category classification is correct, and the comprehensive characteristics of the credibility of the verification sample whose material category classification is incorrect, can further improve the accuracy of the determined credibility threshold, and can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0124] Step S402: based on the average value and the mean square deviation of the first credibility, and the average value and the mean square deviation of the second credibility, the credibility threshold is calculated and determined.

[0125] Wherein, as described above, the second credibility refers to the credibility corresponding to the verification sample whose material category classification is incorrect when the material identification model performs material identification on the verification sample, therefore, the average value of the second credibility is the average value of the second credibility of the plurality of verification samples whose material category classification is incorrect, and the mean square deviation of the second credibility is the mean square deviation of the second credibility of the plurality of verification samples whose material category classification is incorrect.

[0126] Based on this embodiment, for the credibility of the verification sample in the verification set, for the verification sample whose material category classification is correct, and for the verification sample whose material category classification is incorrect, the average value and the mean square deviation of their credibility are calculated respectively, and the credibility threshold is calculated and determined accordingly, so that when the credibility threshold is determined based on the verification result of the verification sample in the verification set, the average value and the mean square deviation are calculated respectively from two dimensions and then comprehensively evaluated, which can further improve the accuracy of the determined credibility threshold, and can help to further improve the accuracy of the finally determined material category of the to-be-identified object.

[0127] In the step S402, after obtaining the mean value and the mean square deviation of the first confidence and the mean value and the mean square deviation of the second confidence, the manner of calculating the confidence threshold is not limited, and in some embodiments, the confidence threshold includes a weighted sum value of a first difference value and a first sum value, the first difference value is a difference value of the mean value and the mean square deviation of the first confidence, and the first sum value is a sum value of the mean value and the mean square deviation of the second confidence. That is, the weighted sum value of the difference between the mean value and the mean square deviation of the first confidence and the sum of the mean value and the mean square deviation of the second confidence is taken as the confidence threshold.

[0128] Taking the mean value as a and the mean square deviation as b as an example, the mean value and the mean square deviation of the first confidence can be denoted as a1 and b1 respectively, the mean value and the mean square deviation of the second confidence can be denoted as a2 and b2 respectively, and the confidence threshold is denoted as f. The manner of determining the confidence threshold can be expressed by a formula as follows:

[0129] f=s1*(a1-b1)+s2*(a2+b2) (1)

[0130] Wherein, s1 and s2 are the first weight of the first confidence and the second weight of the second confidence respectively, and s1+s2=1.

[0131] Based on this embodiment, when determining the confidence threshold, the weighted sum value of the difference between the mean value and the mean square deviation of the first confidence and the sum of the mean value and the mean square deviation of the second confidence is determined, so that the difference between the mean value and the mean square deviation of the confidence corresponding to the verification sample classified correctly in terms of the material category of the verification sample is considered, and the comprehensive influence of the mean value and the mean square deviation of the confidence corresponding to the verification sample classified incorrectly in terms of the material category of the verification sample is considered, thereby further improving the accuracy of the obtained confidence threshold. Moreover, by calculating the difference between the mean value and the mean square deviation of the first confidence, since the mean square deviation reflects the dispersion degree of a group of data, the influence of the dispersed samples in the verification sample classified correctly in terms of the material category of the verification sample is reduced, and for the verification sample classified incorrectly in terms of the material category of the verification sample, by calculating the sum of the mean value and the mean square deviation of the second confidence, the concentration trend of the verification sample classified incorrectly is considered, and the influence of the dispersion is also considered, thereby further improving the accuracy of the obtained confidence threshold, which can help to further improve the accuracy of the finally determined material category of the object to be recognized.

[0132] Wherein, in the weighted sum, the specific value of the first weight and the second weight is not limited, and can be determined based on the degree of emphasis on the verification sample with correct material category classification and the verification sample with incorrect material category classification. In some embodiments, the verification sample with correct material category classification and the verification sample with incorrect material category classification can be considered to the same extent, and at this time, the first weight includes 50%, and the second weight includes 50%. In this case, the formula (1) of the above-mentioned method of determining the confidence threshold can be expressed as:

[0133] y = (a1-b1) / 2 + (a2+b2) / 2 (2)

[0134] In some specific examples, the above formula (2) can be further expressed as:

[0135] (3)

[0136] Wherein, represents the confidence threshold, represents the average value of the first confidence, i.e. the average value of the confidence of the verification sample with correct material category classification, represents the mean square error of the first confidence, i.e. the mean square error of the confidence of the verification sample with correct material category classification, represents the average value of the second confidence, i.e. the average value of the confidence of the verification sample with incorrect material category classification, represents the mean square error of the second confidence, i.e. the mean square error of the confidence of the verification sample with incorrect material category classification.

[0137] Wherein, the above-mentioned confidence threshold determined can remain unchanged after the material recognition model is deployed, i.e. after the material recognition model is deployed, the same confidence threshold is used to determine whether the predicted material category with the highest probability is feasible. The confidence threshold can also be updated, for example, the confidence threshold is updated every interval, the confidence threshold is updated after a certain number of instance signals are identified, or the confidence threshold is updated at other required times.

[0138] In some embodiments, referring to Figure 5 The above method further comprises:

[0139] Step S501: obtaining the material recognition of the instance signal by the material recognition model, and obtaining the instance material category and the corresponding confidence.

[0140] The instance signal refers to a signal obtained by analyzing an object corresponding to the instance signal by the material recognition model after the material recognition model is trained and deployed, and the instance material category refers to a material category determined by the material recognition model by analyzing the instance signal. For example, when the material recognition model is deployed on an interactive tablet, the elastic wave signal detected by the elastic wave detection device when the object touches the display screen is the instance signal, and the material category of the object output by the material recognition model by analyzing the elastic wave signal is the instance material category.

[0141] Step S502: filtering a target credibility from the credibilities of the instance material categories.

[0142] The way of filtering the target credibility from the credibilities of the instance material categories is not limited, and in some embodiments, the credibilities of all the instance material categories can be regarded as the target credibility. In the embodiments of the present application, the credibility greater than a preset threshold value in the credibility of the instance material category can be filtered as the target credibility. The preset threshold value is not limited and can be set according to actual needs.

[0143] Step S503: updating the credibility threshold value based on the average value and mean square deviation of the first credibility, the average value and mean square deviation of the second credibility, and the average value and mean square deviation of the target credibility.

[0144] When updating the credibility threshold value, a new credibility threshold value can be calculated based on the average value and mean square deviation of the first credibility, the average value and mean square deviation of the second credibility, and the average value and mean square deviation of the target credibility, and the new credibility threshold value is used to replace the credibility threshold value in the future to update the credibility threshold value.

[0145] Based on formula (1), the way of calculating the new credibility threshold value based on the average value and mean square deviation of the first credibility, the average value and mean square deviation of the second credibility, and the average value and mean square deviation of the target credibility can be represented by formula as follows:

[0146] y=k1* (a1-b1) +k2* (a2+b2) +k3* (a3+b3) (4)

[0147] Wherein, a3, b3 are the average value and mean square deviation of the target credibility respectively, k1, k2, k3 are coefficients, and k1+k2+k3=1.

[0148] For example, k1 and k3 are 0.25, and k2 is 0.5, and the above formula (2) can be further represented as:

[0149] (5)

[0150] wherein, represents an updated credibility threshold value, represents an average value of the target credibility, represents a mean square error of the target credibility.

[0151] Based on this embodiment, after the material recognition model is deployed and applied, the target credibility is filtered out from the credibility of the instance material category based on the result of the instance signal material recognition by the material recognition model within the preset time period, and the credibility threshold value is updated in combination with the average value and the mean square error of the target credibility, so that the credibility threshold value can be updated in combination with the actual application result after the material recognition model is deployed and applied, the accuracy of the determined credibility threshold value can be further improved, and on this basis, the accuracy of the finally determined material category of the object to be recognized can be further improved.

[0152] wherein, the specific structure of the material recognition model is not limited, and in some embodiments, the material recognition model comprises a feature mapping module, a material classification module and a credibility recognition module connected in sequence, and the credibility recognition module comprises a Relu layer and a Tanh layer connected.

[0153] The feature mapping module is used to extract the signal features of the detection signal, and usually contains one or more fully connected layers, Relu layers and BN (Batch Normalization) layers. Among them, the fully connected layer can map the distributed features to the sample label space. For multi-channel elastic wave signals, the multi-channel elastic wave signals can be mapped to the same space through the fully connected layer. The combination of the Relu layer and the BN layer can better learn the nonlinear features, and can accelerate the convergence without losing the accuracy of the model. It should be understood that the feature mapping module can also be implemented in other ways as long as it can extract the signal features of the detection signal.

[0154] wherein, the number of the fully connected layers, the Relu layers and the BN layers contained in the feature mapping module is not limited, and in some embodiments, the feature mapping module can contain two layers of the structure of the fully connected layers, the Relu layers and the BN layers, that is, the feature mapping module can contain the fully connected layers, the Relu layers, the BN layers, the fully connected layers, the Relu layers and the BN layers connected in sequence.

[0155] The material classification module is used to realize the classification processing, and in some embodiments, the material classification module can include a fully connected layer to realize the classification processing.

[0156] The credibility recognition module is used to realize analysis and processing of the credibility, and includes a Relu layer and a Tanh layer connected in sequence. Through the combination of the Relu layer and the Tanh layer, the analysis of the non-negative credibility can be realized, and the output range of the credibility can be limited to the range of [0, 1].

[0157] Accordingly, taking the feature mapping module including two full connection layers, a Relu layer and a BN layer, the material classification module including one full connection layer, and the credibility recognition module including a Relu layer and a Tanh layer connected in sequence as an example, the model structure of the material recognition model can be as shown in Figure 6 .

[0158] Based on the model shown in Figure 6 , the feature mapping module includes two stacked full connection layers, a Relu layer and a BN layer, the material classification module includes a full connection layer, and the credibility recognition module includes a Relu layer and a Tanh layer. Assuming that the number of preset material categories is t, after the stacking of the two full connection layers, the Relu layer and the BN layer, the original input signal is nonlinearly mapped into a feature signal, and then the probability of each preset material category is obtained through a full connection layer , and finally the credibility of the probability of each preset material category is obtained through the Relu layer and the Tanh layer .

[0159] Based on this embodiment, the credibility recognition module connected to the material classification module in the material recognition model, on the basis of determining the credibility through the Relu layer, also limits the range of the credibility through the Tanh layer, thereby helping to set the range of the credibility threshold value and being conducive to deployment and application.

[0160] Among the material recognition models in each of the above embodiments, the credibility of the predicted material category corresponding to the predicted probability is determined at the same time as the predicted material category with the maximum predicted probability is determined. Therefore, in some embodiments, in the process of training the material recognition model, the model training loss of the material recognition model is calculated, including:

[0161] Based on the predicted category of the training sample and the credibility of each preset material category obtained by the material recognition model in the material recognition of the training sample, the training loss of the material recognition model is calculated.

[0162] Based on the embodiment, in the process of training the material identification model, the prediction category of the training sample obtained by the material identification of the training sample and the confidence of the training sample in each preset material category can be combined to comprehensively calculate the training loss, so that the training loss calculated in the training process considers both the prediction category of the training sample identified by the material identification model and the influence of the confidence in each preset material category, which can improve the accuracy of the obtained training loss, which is helpful to accelerate the convergence of the material identification model training process and improve the material identification accuracy of the trained material identification model.

[0163] The way of calculating the training loss of the material identification model based on the prediction category of the training sample obtained by the material identification of the material identification model and the confidence of each preset material category is not limited, and in some embodiments, the calculated training loss of the material identification model can include a first loss and a second loss, wherein the first loss is determined based on the category label of the training sample and the prediction category of the training sample, and the second loss is determined based on the confidence of each preset material category of the training sample. It can be understood that the final obtained training loss can be the sum of the first loss and the second loss.

[0164] Therefore, when the training loss of the material identification model is calculated comprehensively, two aspects can be combined, one aspect is to combine the difference between the prediction category of the training sample predicted by the material identification model and the real category label of the training sample to evaluate the influence of the loss of the prediction probability in the process of identifying the material category, and the other aspect is to combine the confidence of the training sample in each preset material category to evaluate the loss of the confidence of the training sample in each preset material category, which can be used to optimize the determination of the confidence, such as optimizing the layer outputting the confidence in the material identification model. The comprehensive calculation of the loss from different aspects can improve the coverage and accuracy of the calculated training loss, improve the accuracy of the obtained training loss, which is helpful to accelerate the convergence of the material identification model training process and improve the material identification accuracy of the trained material identification model.

[0165] In determining the first loss, different ways can be used, and in the embodiment of the present application, in the first loss, the confidence can be combined based on the consideration of the prediction category of the training sample and the classification label.

[0166] In some embodiments, when the second loss is determined based on the confidence of the training sample in each preset material category, the confidence in each preset material category can be considered comprehensively with the confidence in multiple (for example, all) preset material categories, and in some specific examples, the second loss is determined according to the standard deviation of the confidence of the training sample in each preset material category.

[0167] Based on this embodiment, in determining the first loss, the credibility of the training samples in the preset material categories is considered, which allows for a comprehensive calculation of the first loss and improves its accuracy. When determining the second loss, it is based on the standard deviation of the credibility of the training samples in each preset material category, taking into account the dispersion of credibility and further improving the accuracy of the second loss.

[0168] Considering that the model output mostly follows a Dirichlet distribution during model training, in some specific examples, the first loss can be the sum of squares of the differences between the class labels of the training samples and the first ratio of the training samples. Here, the first ratio can be the ratio of the confidence level of a training sample in a preset material category to the sum of the confidence levels of the training samples in all preset material categories. The second loss can be the ratio of the first product and the second product. The first product is the product of the difference between the sum of the confidence levels of the training samples in all preset material categories and the confidence levels of the training samples in all preset material categories, plus the confidence levels of the training samples in all preset material categories. The second product is the product of the square of the sum of the confidence levels of the training samples in all preset material categories plus 1.

[0169] Let the class label of training sample i be denoted as The confidence level of training sample i in the preset material category j is The sum of the confidence levels of the training samples in each preset material category is denoted as . Then the first ratio of the training samples can be denoted as In this embodiment of the application, the first ratio can be The predicted class of the training sample, the first product can be denoted as: The second product can be written as The training loss for training sample i is Then the loss function for calculating the training loss can be shown in the following equation (6):

[0170] (6)

[0171] in, .

[0172] In the above formula, i represents the i-th training sample, and j represents the j-th preset material category. Indicates shared ownership There are several preset material categories. The category labels are denoted as follows: When j is the true classification of the training sample, then =1, otherwise =0, The first loss, This is the second loss.

[0173] It can be understood that the loss function for calculating the training loss described above is only an example, and in other embodiments, the manner of calculating the first loss can also use other loss calculation manners, such as local loss calculation, and other calculation formulas can be used to calculate the loss function when the output of the model meets other data distributions, and the embodiments of the present application do not make specific limitations.

[0174] In some embodiments, as shown in FIG. 3, the detection signal obtained by the step S301 of acquiring the signal detection of the object to be identified includes: Figure 7

[0175] Step S701: Obtain the elastic wave signal output by the multi-channel elastic wave sensor.

[0176] The elastic wave sensor is a sensor based on elastic wave technology. Elastic wave is a kind of stress wave, which is the form of stress and strain transmission in elastic medium caused by disturbance or external force. When the particles of a certain material deviate from the equilibrium position, i.e. strain, the particles vibrate under the action of elastic force, and at the same time, the strain and vibration of the surrounding particles are caused, and the propagation process of the vibration in the elastic medium is called elastic wave.

[0177] The multi-channel elastic wave sensor refers to a sensor that can obtain multiple elastic wave signals. For example, it can be applied to an interactive panel, and elastic wave sensors can be arranged at different positions of the interactive panel. The multiple elastic wave sensors together form a multi-channel elastic wave sensor. Similarly, for example, it can be applied to an interactive panel, and piezoelectric sensors for detecting elastic wave signals can be arranged at different positions of the interactive panel, and the multiple piezoelectric sensors are connected to the same controller. The multiple piezoelectric sensors and the controller together form a multi-channel elastic wave sensor. It can be understood that the multi-channel elastic wave sensor can also be in other forms, as long as it can detect multiple elastic wave signals.

[0178] Step S702: Convert the effective signal in the multi-channel elastic wave signal to a frequency domain signal.

[0179] Through the multi-channel elastic wave sensor, multiple elastic wave signals can be obtained, i.e. multiple elastic wave signals.

[0180] ​In some processing manners, the multiple elastic wave signals can be converted into frequency domain signals and then enter subsequent processing processes. In the embodiments of the present application, the multiple elastic wave signals are screened to screen out effective signals in the multiple-channel elastic wave signals, and the screened effective signals are converted into frequency domain signals to enter subsequent processing processes, so as to avoid the influence of processing invalid signals on accuracy and efficiency.

[0181] The manner of screening effective signals from the multiple-channel elastic wave signals is not limited, and in some embodiments, the effective signals in the multiple-channel elastic wave signals include elastic wave signals with signal amplitudes greater than a minimum effective signal threshold and less than a maximum effective signal threshold.

[0182] Therefore, whether the elastic wave signal is effective can be judged based on the signal amplitude of the elastic wave signal. If the signal amplitude is less than the minimum effective signal threshold, it means that the energy of the elastic wave signal is too low, and if the signal amplitude is greater than the maximum effective signal threshold, it means that the elastic wave signal has amplitude distortion. Therefore, by determining the elastic wave signal with a signal amplitude greater than the minimum effective signal threshold and less than the maximum effective signal threshold as an effective signal, the effective signal in the multiple-channel elastic wave signal can be conveniently identified and analyzed.

[0183] Step S703: Fusing the multiple-channel frequency domain signals to obtain a detection signal.

[0184] Based on the embodiment, the detection signal obtained by signal detection on the object to be identified is a signal obtained by fusing the multiple-channel frequency domain signals after converting the effective signals in the multiple-channel elastic wave signals into frequency domain signals. The final detection signal considers both the effective signals in the multiple-channel elastic wave signals and the fusion performance of the multiple channels. Using the detection signal obtained on this basis for material identification can improve the accuracy of the material identification result.

[0185] In some embodiments, after obtaining the detection signal, before performing material identification on the detection signal, the method further includes: pre-processing the detection signal, and the pre-processing includes normalization processing.

[0186] Based on the embodiment, after obtaining the detection signal, the detection signal is also pre-processed such as normalization, which reduces the influence of noise and helps to further improve the accuracy of the finally determined material category of the object to be identified.

[0187] When performing multi-channel fusion, it is considered that during signal acquisition, a single elastic wave sensor can only reflect part of the information of the vibration characteristics of the touched object, so there is a certain difference between the signals collected by each elastic wave sensor. In combination with the fact that the vibration characteristics of the touched object are reflected in the signals collected by each elastic wave sensor, the signals collected by each elastic wave sensor are fused to obtain a detection signal. Figure 8As shown, mode 1 represents the mode of the collected elastic wave at position 1, mode 2 represents the mode of the collected elastic wave at position 2, and so on. As can be seen, when position 1 is touched, the modes of the elastic wave collected by the sensors at different positions are different, and there is a certain difference in the amplitudes of the elastic wave signals of different channels. Therefore, in order to reduce the influence of the signal difference between the elastic wave sensors on the material classification and improve the stability of the signal, fusion processing can be performed on the signals of the multiple elastic wave sensors.

[0188] The manner of fusing the multi-channel frequency domain signals is not limited, and the following examples are provided in combination with several fusion manners.

[0189] In some embodiments, the multi-channel frequency domain signals are fused to obtain a detection signal, including: obtaining the detection signal by weighted average processing of the multi-channel frequency domain signals.

[0190] Based on this embodiment, the detection signal is obtained by weighted average processing of the multi-channel frequency domain signals, which reduces the influence of the signal difference between the elastic wave sensors on the material classification and improves the stability of the detection signal.

[0191] In some embodiments, the multi-channel frequency domain signals are fused to obtain a detection signal, including: obtaining the detection signal by weighted average processing of the multi-channel frequency domain signals.

[0192] The system function of vibration propagation of the sensor refers to the function of vibration to the sensor. Due to the difference in the positions of the sensors of the elastic wave signals of each channel, the system functions of the sensors also differ. Therefore, the multi-channel frequency domain signals can be fused by combining the system functions of vibration propagation of the sensors to obtain the detection signal.

[0193] Based on this embodiment, when the multi-channel frequency domain signals are fused, the multi-channel frequency domain signals are fused based on the system functions of vibration propagation of the sensors in the multi-channel elastic wave sensors to obtain the detection signal, which effectively considers the vibration propagation performance of each sensor and improves the accuracy of the obtained detection signal.

[0194] In some embodiments, the multi-channel frequency domain signals can also be fused by a material recognition model. At this time, the first layer of the material recognition model includes a convolution layer, the multi-channel frequency domain signals are fused to obtain a detection signal, including: the multi-channel frequency domain signals are fused by the convolution layer of the first layer of the material recognition model to obtain the detection signal, wherein one input channel of the convolution layer corresponds to one frequency domain signal.

[0195] The one input channel of the convolution layer corresponds to one frequency domain signal, that is, one channel of the convolution layer corresponds to one channel of the elastic wave signal. If the elastic wave signal is valid, the corresponding frequency domain signal is input. If the elastic wave signal is invalid, there is no corresponding frequency domain signal input, that is, the input of the channel of the convolution layer is 0.

[0196] Based on the embodiment, by setting a convolution layer at the first layer of the material identification model and corresponding one input channel of the convolution layer to one frequency domain signal, the fusion of the frequency domain signal can be realized through the material identification model, and the convenience of fusing the frequency domain signal is improved.

[0197] In other embodiments, considering that the elastic wave signals generated will also have certain differences when the touch information (such as touch position, touch area, etc.) is different, the multi-channel frequency domain signals are fused to obtain a detection signal, including: Figure 9

[0198] Step S901: Obtain touch information corresponding to the elastic wave signal.

[0199] The touch information corresponding to the elastic wave signal refers to the information corresponding to the touch action that generates the elastic wave signal. The specific type of touch information is not limited. For example, the touch information can include the touch position, and can also include the touch area, and can also include the touch position and the touch area at the same time.

[0200] Step S902: Determine the touch weight information of each elastic wave signal according to the touch information corresponding to the elastic wave signal.

[0201] Taking the identification of the writing material of the interactive panel as an example, based on the structure of the interactive panel, when the same material touch object (such as a finger, a joint, a pen head of a stylus, or a pen tail of a stylus, etc.) touches the interactive panel, the touch information (such as touch position and touch area, etc.) will also be different. Based on the difference of the touch information, the elastic wave signals generated will also have certain changes, so that when the elastic wave signal is obtained, the touch information corresponding to the elastic wave signal can be obtained, and the touch weight information of the elastic wave signal can be determined according to the elastic wave signal.

[0202] Taking an example that the touch information includes at least one of the touch position and the touch area, the determined touch weight information can include at least one of the position weight and the area weight, wherein the position weight is a weight determined based on the touch position, and the area weight is a weight determined based on the touch area.

[0203] In combination with the touch information to determine the touch weight information, the touch weight information can be determined in combination with the normalization processing.

[0204] ​Taking touch information, including touch location, and recognizing the writing material of the interactive flat panel as an example, let the width and height of the interactive flat panel be w and h respectively, and the touch locations in the x and y directions be respectively... and The normalized touch locations are as follows: and The touch position can then be normalized to between 0 and 1 using the following formula to adapt to interactive tablets of different sizes:

[0205]

[0206] Step S903: Based on the touch weight information of each elastic wave signal, the frequency domain signals corresponding to each elastic wave signal are weighted and fused to obtain the detection signal.

[0207] Based on the determined touch weight information of each elastic wave signal, the frequency domain signals corresponding to the determined elastic wave signals can be weighted and fused to obtain the detection signal.

[0208] Based on this embodiment, when fusing multi-channel elastic wave signals, the touch weight information of each elastic wave signal is determined by utilizing the touch information corresponding to the elastic wave signal, and the frequency domain signal is weighted and fused based on the touch weight information. This allows for better utilization of the touch information related to the obtained elastic wave information, thereby improving the accuracy of the obtained detection signal and further enhancing the accuracy of material recognition.

[0209] The process of determining the touch weight information of each elastic wave signal and, based on the touch weight information of each elastic wave signal, performing weighted fusion of the frequency domain signals corresponding to each elastic wave signal to obtain the detection signal can be carried out based on the material recognition model.

[0210] Accordingly, in some embodiments, when the touch weight information of each elastic wave signal is determined through a material recognition model, the principle of material recognition through the material recognition model can be as follows: Figure 10 As shown.

[0211] refer to Figure 11 As shown, when the touch weight information of each elastic wave signal is determined through a material recognition model, the material recognition model may include: a touch information processing module 1101, a feature mapping module 1102, a material classification module 1103, and a confidence recognition module 1104 connected in sequence; wherein:

[0212] The touch information processing module 1101 is used to determine the touch weight information of each elastic wave signal based on the touch information corresponding to the multi-channel elastic wave signals.

[0213] The feature mapping module 1102 is configured to perform fusion processing on the multi-channel elastic wave signals based on the touch weight information of each elastic wave signal to obtain a detection signal.

[0214] The material classification module 1103 is configured to perform material identification on the detection signal to obtain a predicted material category with the highest prediction probability of the object to be identified.

[0215] The credibility identification module 1104 is configured to determine the credibility corresponding to the predicted material category.

[0216] Based on this embodiment, by deploying a touch information processing module in the material identification model, the determination of the touch weight information can be realized by the touch information processing module, and the weighted fusion of each elastic wave signal is performed on this basis, thereby improving the convenience of determining the touch weight information and the convenience of fusing the frequency domain signals.

[0217] In some embodiments, the touch information includes at least one of a touch position and a touch area, the touch weight information includes at least one of a position weight and an area weight, and the touch information processing module 905 includes at least one of a position processing module and an area processing module, wherein:

[0218] The position processing module is configured to determine the position weight of each elastic wave signal based on the touch position corresponding to the multi-channel elastic wave signal.

[0219] The area processing module is configured to determine the area weight of each elastic wave signal based on the touch area corresponding to the multi-channel elastic wave signal.

[0220] Based on this embodiment, when determining the touch weight information of the elastic wave signal based on the touch information, both the position of the touch and the area of the touch are considered, and the touch weight information of the elastic wave signal can be determined from two dimensions of position and area, thereby improving the accuracy of the determined touch weight information, further improving the accuracy of the obtained detection information, and further improving the accuracy of the material identification.

[0221] Taking an example in which the touch information includes both the touch position and the touch area, the model structure of the material identification model in one of the examples can be as shown in Figure 12 .

[0222] Figure 12 As shown in the above table, the position processing module includes a fully connected layer and a Tanh layer, the area processing module includes a fully connected layer and a Tanh layer, the feature mapping module includes a convolution layer, a fully connected layer, a product layer, a BN layer and a Relu layer connected in sequence, the material classification module includes a fully connected layer, and the credibility identification module includes a Relu layer and a Tanh layer.

[0223] In the material identification, for the spectrum of the multi-channel elastic wave signal, first, the multi-channel data is fused into single-channel data through the convolution layer (the convolution kernel size can be an empirical parameter, and the output channel number is 1) of the feature mapping module, and then a fully connected layer (the number of neurons is an empirical parameter) is passed to obtain a feature vector.

[0224] For the touch position and the touch area of the multi-channel elastic wave signal, two different fully connected layers (the number of neurons is the same as the number of fully connected layers) are passed respectively, and then a Tanh layer (in other embodiments, it can also be a sigmoid layer) is passed to obtain the position weight and the area weight respectively, which are in the range of 0-1.

[0225] The product layer of the feature mapping module sequentially multiplies the feature vector with the position weight and the area weight point by point, and then passes through a BN layer, a Relu layer and a fully connected layer (the number of neurons is the number of preset categories) to obtain the probability of each preset category , and finally passes through a Relu layer and a Tanh layer of the credibility processing module to obtain the credibility of each category .

[0226] Taking the touch information including the touch position and the touch area as an example, the weights of the touch position and the touch area can also be fused sequentially, and the model structure of the material identification model in other examples can also be as shown in Figure 13 and Figure 14 .

[0227] For example, as shown in Figure 13 , the position processing module includes a fully connected layer and a sigmoid layer, the area processing module includes a fully connected layer and a sigmoid layer, the feature mapping module includes a convolution layer, a fully connected layer, a product layer, a BN layer and a Relu layer connected in sequence, the material classification module includes a fully connected layer, and the credibility recognition module includes a Relu layer and a Tanh layer.

[0228] For the touch position of the multi-channel elastic wave signal, a fully connected layer and a sigmoid layer are passed to obtain the position weight, and the obtained position weight is in the range of 0-1.

[0229] For the touch area of the multi-channel elastic wave signal, a fully connected layer and a sigmoid layer are passed to obtain the area weight, and the obtained area weight is in the range of 0-1.

[0230] For the spectrum of the multi-channel elastic wave signal, first, the multi-channel data is combined with the position weight to fuse into single-channel data through the weighted sum convolution layer (the convolution kernel size is an empirical parameter, and the output channel number is 1) of the feature mapping module, and then a fully connected layer (the number of neurons is an empirical parameter) is passed to obtain a feature vector.

[0231] The product layer of the feature mapping module pointwise multiplies the feature vector with the area weight, and then passes through a BN layer and a Relu layer and a full connection layer (the number of neurons is the number of preset categories) to obtain the probability of each preset category , and finally passes through a Relu layer and a Tanh layer of the credibility processing module to obtain the credibility of each category .

[0232] Based on the above-mentioned embodiments, the following will be illustrated in combination with some specific application examples. In the following examples, the recognition of writing materials applied to interactive tablets is taken as an example, and the material recognition is performed by a material recognition model.

[0233] The material recognition model of the embodiments of the present application can be a material recognition model based on a deep neural network, which can be the model shown in Figure 6 、 Figure 12 、 Figure 13 or Figure 14 , but is not limited thereto.

[0234] When training the material recognition model, a sufficient number of samples can be collected, each sample including an elastic wave signal and a corresponding material category label, and the material category label can be determined by manual annotation. The samples are divided into a training set, a validation set and a test set. For the ith sample, the material category is q, and a one-hot material category vector of the sample can be established , where i represents the ith sample, and j represents the jth material category.

[0235] In the process of training the material recognition model, the elastic wave signal in the training sample in the training set is input into the material classification model, the material classification model performs material classification and recognition on the elastic wave signal, outputs the material classification probability of the elastic wave signal to each preset material category and the corresponding credibility, and combines each material classification probability, credibility and corresponding material category label to calculate the training loss . The loss function for calculating the training loss can be shown in the following formula (6)

[0236] (6)

[0237] wherein, .

[0238] wherein, represents the credibility of the ith sample in the jth preset material category.

[0239] Based on the calculated training loss, the material recognition model can be optimized, for example, the Adam optimization algorithm can be used to optimize the model. Among them, the number of rounds of training of the material recognition model and the amount of each batch of samples can be set as needed.

[0240] After a certain number of rounds of training or after the end of each round of training, the material recognition model obtained by training is used to recognize the materials of the verification samples in the verification set, and on this basis, the initial value of the confidence threshold can be set. By recognizing the materials of the verification samples in the verification set through the material recognition model, the first confidence of the correctly classified verification samples and the second confidence of the incorrectly classified verification samples are counted, and the mean of the confidence of all correctly classified verification samples (i.e. the average of the first confidence) and the standard deviation (i.e. the mean square deviation of the first confidence) , and the mean of the confidence of all incorrectly classified verification samples (i.e. the average of the second confidence) and the standard deviation (i.e. the mean square deviation of the second confidence) , then the confidence threshold can be calculated based on formula (3) .

[0241] If the recognition accuracy of the test samples in the test set reaches the preset requirement, for example, reaches 0.9975 or more, it is considered that the material recognition model meets the requirements, and the last updated model parameters are used as the final parameters of the material recognition model.

[0242] After training the material recognition model, the trained material recognition model can be deployed to a specific device that needs to perform material recognition, such as an interactive tablet, to perform material recognition when needed.

[0243] Taking writing as an example, when a touch object performs touch operations such as clicking or moving on the display screen, the multi-channel elastic wave sensor collects multi-channel elastic wave signals, and performs signal quality assessment on the multi-channel elastic wave signals to determine whether the elastic wave signals of each channel are valid signals. The filtered valid signals are subjected to Fourier transform to obtain frequency domain signals, and then the obtained multiple frequency signals are subjected to multi-channel fusion to obtain fused single-channel signals.

[0244] Among them, when performing multi-channel fusion, considering that during signal acquisition, a single elastic wave sensor can only reflect part of the information of the vibration characteristics of the touch object, therefore, there is a certain difference between the signals collected by each elastic wave sensor. For example Figure 8As shown, mode 1 represents the mode of the collected elastic wave at position 1, mode 2 represents the mode of the collected elastic wave at position 2, and the others are similar. It can be seen that when position 1 is touched, the modes of the elastic wave collected by the sensors at different positions are different, and there is a certain difference in the amplitudes of the elastic wave signals of different channels. Therefore, in order to reduce the influence of the signal difference between the elastic wave sensors on the material classification and improve the stability of the signal, fusion processing can be performed on the multi-elastic wave sensor signals.

[0245] In some embodiments, the fusion of the multi-elastic wave sensor signals can be realized by a weighted average strategy. Let the effective elastic wave signals be where n is the signal length, k is the number of channels, is the elastic wave signal of each channel, and the fusion signal is obtained by weighted average as shown in formula (7)

[0246] (7)

[0247] where is a weight parameter, which can be set to 1 in experiments At this time, the fusion weights of each channel are consistent.

[0248] In other embodiments, the fusion of each elastic wave signal can be performed in combination with the touch position and touch area corresponding to each elastic wave signal to obtain a single-channel signal after fusion.

[0249] After the single-channel signal is obtained by fusion, the single-channel signal can be preprocessed to reduce noise. For example, amplitude normalization can be used for preprocessing to scale the signal to 0-1. Let and represent the maximum and minimum values in respectively, denote the preprocessed data, and formula (8) gives the calculation process of amplitude normalization

[0250] (8)

[0251] After the single-channel signal is preprocessed, it can be input into a material recognition model. The material recognition model identifies the probability of classifying the single-channel signal into each preset material category, takes the preset material category with the maximum probability as the predicted material category, and compares the confidence of the predicted material category with the confidence threshold. If the confidence is lower than the confidence threshold, it is considered that the current recognition result is unreliable, and a prompt information is output, for example, -1 is output. If the confidence is greater than or equal to the confidence threshold, it is considered that the current recognition result is reliable, and the predicted material category is output, that is, the preset material category with the maximum probability is output.

[0252] Wherein, after the material identification model is deployed, the credibility threshold can also be dynamically adjusted. For example, for a certain number (the specific number can be set as needed) of instance signals identified recently, the credibility of the instance signals with credibility greater than the preset threshold is filtered as the target credibility, and the credibility threshold is updated in combination with the manner in formula (5).

[0253] It should be understood that in the above description of the specific examples, the single-channel signal obtained after preprocessing is input into the material identification model, and the above description of the embodiments can also be that the multi-channel elastic wave signal is directly input into the material identification model, the single-channel signal is obtained by fusing the multi-channel elastic wave signal through the material identification model, and material identification is performed on this basis. It can also be that the multi-channel elastic wave signal and the touch position and touch area are simultaneously input into the material identification model, the single-channel signal is obtained by fusing the multi-channel elastic wave signal through the material identification model in combination with the touch position and touch area, and material identification is performed on this basis.

[0254] It should be understood that although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise stated herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0255] Based on the same inventive concept, the embodiments of the present application also provide a material identification device for implementing the above-mentioned material identification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more material identification device embodiments provided below can refer to the limitations of the material identification method described above, and will not be repeated here.

[0256] In one embodiment, as shown in Figure 15 A material identification device is provided, comprising: a signal acquisition module 1501, a material identification module 1502, and a material determination module 1503, wherein:

[0257] The signal acquisition module 1501 is configured to acquire a detection signal obtained by signal detection on a to-be-identified object;

[0258] The material identification module 1502 is configured to perform material identification on the detection signal by using the material identification model to obtain a predicted material category with the highest prediction probability of the object to be identified and a confidence level corresponding to the predicted material category.

[0259] The material determination module 1503 is configured to determine the predicted material category as the material category of the object to be identified when the confidence level is greater than the confidence level threshold.

[0260] The threshold determination module 1504 is configured to perform material identification on the verification sample in the verification set by using the material identification model to obtain a verification sample material category with the highest prediction probability and a corresponding confidence level, wherein the confidence level of the verification sample material category includes a first confidence level of the verification sample material category classified correctly and a second confidence level of the verification sample material category classified incorrectly; and determine the confidence level threshold based on the first confidence level and the second confidence level.

[0261] In some embodiments, the threshold determination module 1504 is configured to determine the average value and the mean square deviation of the first confidence level of the verification sample in the verification set, and the average value and the mean square deviation of the second confidence level of the verification sample in the verification set; and calculate and determine the confidence level threshold based on the average value and the mean square deviation of the first confidence level and the average value and the mean square deviation of the second confidence level.

[0262] In some embodiments, the confidence level threshold includes a weighted sum of a first difference value and a first sum value, the first difference value is a difference value of the average value and the mean square deviation of the first confidence level, and the first sum value is a sum value of the average value and the mean square deviation of the second confidence level.

[0263] In some embodiments, the threshold determination module 1504 is further configured to obtain an instance material category and a corresponding confidence level obtained by performing material identification on an instance signal by using the material identification model; select a target confidence level from the confidence levels of the instance material categories; and update the confidence level threshold based on the average value and the mean square deviation of the first confidence level, the average value and the mean square deviation of the second confidence level, and the average value and the mean square deviation of the target confidence level.

[0264] In some embodiments, the material identification model includes a feature mapping module, a material classification module, and a confidence level identification module connected in sequence, and the confidence level identification module includes a Relu layer and a Tanh layer connected in sequence.

[0265] In some embodiments, the device further includes a model training module configured to train the material identification model, and during the training of the material identification model, calculate a training loss of the material identification model based on a predicted category of a training sample obtained by performing material identification on the training sample by using the material identification model and a confidence level of each preset material category.

[0266] In some embodiments, the training loss comprises a first loss and a second loss, the first loss is determined based on the class label of the training sample and the predicted class of the training sample, and the second loss is determined based on the confidence of the training sample in each preset material class.

[0267] In some embodiments, the second loss is determined according to a standard deviation of the confidence of the training sample in each preset material class.

[0268] In some embodiments, the signal acquisition module 1501 is configured to acquire elastic wave signals output by a multi-channel elastic wave sensor; convert effective signals in the multi-channel elastic wave signals into frequency domain signals; and fuse the multi-channel frequency domain signals to obtain a detection signal.

[0269] In some embodiments, the signal acquisition module 1501 acquires touch information corresponding to the elastic wave signals; determines touch weight information of each elastic wave signal according to the touch information corresponding to the elastic wave signals; and performs weighted fusion on the frequency domain signals corresponding to each elastic wave signal based on the touch weight information of each elastic wave signal to obtain the detection signal.

[0270] In some embodiments, the material recognition model comprises, in sequence, a touch information processing module, a feature mapping module, a material classification module, and a confidence recognition module.

[0271] The touch information processing module is configured to determine touch weight information of each elastic wave signal based on touch information corresponding to the multi-channel elastic wave signals.

[0272] The feature mapping module is configured to perform fusion processing on the multi-channel elastic wave signals based on the touch weight information of each elastic wave signal to obtain a detection signal.

[0273] The material classification module is configured to perform material recognition on the detection signal to obtain a predicted material class with the maximum predicted probability of the object to be recognized.

[0274] The confidence recognition module is configured to determine a confidence corresponding to the predicted material class.

[0275] In some embodiments, the touch information comprises at least one of a touch position and a touch area, the touch weight information comprises at least one of a position weight and an area weight, and the touch information processing module comprises at least one of a position processing module and an area processing module.

[0276] The position processing module is configured to determine the position weight of each elastic wave signal based on the touch position corresponding to the multi-channel elastic wave signals.

[0277] The area processing module is configured to determine the area weight of each elastic wave signal based on the touch area corresponding to the multi-channel elastic wave signals.

[0278] The modules in the material identification device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the modules.

[0279] In an embodiment, a computer device, which can be a server, has an internal structure as shown in FIG. 6A. The computer device includes a processor, a memory, and a network interface connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data, such as a material identification model, a credibility threshold, and the like. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a material identification method. Figure 16

[0280] In an embodiment, a computer device, which can be a terminal, has an internal structure as shown in FIG. 6B. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented by WIFI, a mobile cellular network, NFC (near field communication), or other technologies. The computer program is executed by the processor to implement a material identification method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball, or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad, a mouse, or the like. Figure 17

[0281] It should be understood by those skilled in the art that, Figure 16 , Figure 17 ​​The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0282] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the material identification method in any of the above embodiments when executing the computer program.

[0283] In one embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implementing the steps of the material identification method in any of the above embodiments when executed by a processor.

[0284] In one embodiment, a computer program product is provided, comprising a computer program, and the computer program implementing the steps of the material identification method in any of the above embodiments when executed by a processor.

[0285] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0286] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0287] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0288] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A material identification method, characterized in that, The method includes: Acquire the detection signal obtained by performing signal detection on the object to be identified; The detection signal is identified by a material recognition model to obtain the predicted material category with the highest predicted probability of the object to be identified, and the confidence level of the predicted material category. If the confidence level is greater than the confidence level threshold, the predicted material category is determined as the material category of the identified object. The determination method for the credibility threshold includes: The material identification model is used to identify the material of the verification samples in the verification set to obtain the material category of the verification sample with the highest prediction probability and the corresponding confidence level. The confidence level of the material category of the verification sample includes a first confidence level for the verification sample material category being correctly classified and a second confidence level for the verification sample material category being incorrectly classified. The credibility threshold is determined based on the first credibility and the second credibility.

2. The method according to claim 1, characterized in that, The step of determining the credibility threshold based on the first credibility and the second credibility includes: Determine the mean and standard deviation of the first confidence level of the validation samples in the validation set, and the mean and standard deviation of the second confidence level of the validation samples in the validation set; The confidence threshold is calculated and determined based on the average value and mean square deviation of the first confidence level, and the average value and mean square deviation of the second confidence level.

3. The method according to claim 2, characterized in that, The confidence threshold is a weighted sum of a first difference and a first sum, where the first difference is the difference between the average value and the root mean square deviation of the first confidence, and the first sum is the sum of the average value and the root mean square deviation of the second confidence.

4. The method according to claim 2, characterized in that, The method further includes: Obtain the instance material category and corresponding confidence level obtained by the material recognition model performing material recognition on the instance signal; Filter the target credibility from the credibility of the example material categories; The confidence threshold is updated based on the average value and mean squared error of the first confidence level, the average value and mean squared error of the second confidence level, and the average value and mean squared error of the target confidence level.

5. The method according to claim 1, characterized in that, The method further includes: During the training process of obtaining the material recognition model, the training loss of the material recognition model is calculated based on the predicted category of the training sample obtained by material recognition of the training sample and the confidence level of each preset material category.

6. The method according to claim 5, characterized in that, The training loss includes a first loss and a second loss. The first loss is determined based on the category label of the training sample and the predicted category of the training sample, and the second loss is determined based on the confidence level of each preset material category of the training sample.

7. The method according to claim 6, characterized in that, The second loss is determined based on the standard deviation of the confidence level of the training samples in each preset material category.

8. The method according to any one of claims 1 to 7, characterized in that, The material recognition model includes a feature mapping module, a material classification module, and a credibility recognition module connected in sequence.

9. The method according to any one of claims 1 to 7, characterized in that, The acquisition of the detection signal obtained by signal detection of the object to be identified includes: Acquire elastic wave signals output from a multi-channel elastic wave sensor; Convert the effective signal in the multi-channel elastic wave signal into a frequency domain signal; The detection signal is obtained by fusing the frequency domain signals from multiple channels.

10. The method according to claim 9, characterized in that, The process of fusing the frequency domain signals from multiple channels to obtain the detection signal includes: Obtain the touch information corresponding to the elastic wave signal; Based on the touch information corresponding to the elastic wave signals, determine the touch weight information of each elastic wave signal; Based on the touch weight information of each elastic wave signal, the frequency domain signals corresponding to each elastic wave signal are weighted and fused to obtain the detection signal.

11. The method according to claim 10, characterized in that, The material recognition model includes a touch information processing module, a feature mapping module, a material classification module, and a credibility recognition module connected in sequence. The touch information processing module is used to determine the touch weight information of each elastic wave signal based on the touch information corresponding to the multi-channel elastic wave signals. The feature mapping module is used to fuse the multi-channel elastic wave signals based on the touch weight information of each elastic wave signal to obtain the detection signal. The material classification module is used to identify the material of the detection signal and obtain the predicted material category with the highest predicted probability of the object to be identified. The credibility recognition module is used to determine the credibility of the predicted material category.

12. The method according to claim 11, characterized in that, The touch information includes at least one of touch position and touch area; the touch weight information includes at least one of position weight and area weight; and the touch information processing module includes at least one of position processing module and area processing module. The position processing module is used to determine the position weight of each elastic wave signal based on the touch position corresponding to the multi-channel elastic wave signal; The area processing module is used to determine the area weight of each elastic wave signal based on the touch area corresponding to the multi-channel elastic wave signals.

13. A material identification device, characterized in that, The device includes: The signal acquisition module is used to acquire the detection signal obtained by the signal detection of the object to be identified; The material recognition module is used to perform material recognition on the detection signal through a material recognition model, obtain the predicted material category with the highest predicted probability of the object to be identified, and the confidence level corresponding to the predicted material category; The material determination module is used to determine the predicted material category as the material category of the identified object when the confidence level is greater than the confidence level threshold. The threshold determination module is used to identify the material of the verification samples in the verification set through the material recognition model, obtain the material category of the verification sample with the highest predicted probability and the corresponding confidence level. The confidence level of the material category of the verification sample includes a first confidence level for correctly classifying the material category of the verification sample and a second confidence level for incorrectly classifying the material category of the verification sample. Based on the first confidence level and the second confidence level, a confidence threshold is determined.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Images