Cognitive detection method and related device, electronic device and storage medium

Abstract

The present application discloses a cognitive detection method and related devices, electronic devices and storage media, wherein the cognitive detection method includes: recognizing the speech to be tested of the target object based on the target acoustic model to obtain the phoneme sequence of the speech to be tested; wherein the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the training of the sample target speech collected from the target object in advance; and decoding is performed based on the phoneme sequence to obtain the recognition text of the speech to be tested; then cognitive detection is performed based on the recognition text to obtain the cognitive detection result of the target object; wherein the cognitive detection result includes whether the target object has cognitive impairment. The above scheme can improve the accuracy of cognitive detection while reducing the detection cost.

Description

Technical Field

[0001] The present application relates to the field of smart medical technology, and in particular to a cognitive detection method and related devices, electronic equipment and storage media. Background Art

[0002] The purpose of cognitive testing for the subject to be tested is to better understand the physical health of the subject to be tested, and then determine whether the subject to be tested has cognitive impairment. For example, when conducting cognitive testing on young children, the results of cognitive testing can help further understand the brain development of young children; or when conducting cognitive testing on the elderly, the results of cognitive testing can help further understand the physical health of the elderly, and so on.

[0003] At present, there are two methods for testing cognitive status: one is not user-friendly and has a high cost, while the other is low cost but the accuracy is difficult to guarantee. In view of this, how to reduce the cost of testing while improving the accuracy of cognitive testing has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a cognitive detection method and related devices, electronic devices and storage media, which can reduce the detection cost while improving the accuracy of cognitive detection.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a cognitive detection method, comprising: recognizing the speech to be tested of the target object based on the target acoustic model to obtain a phoneme sequence of the speech to be tested; wherein, the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameters of the target object, and the weight parameters of the target object are obtained based on the training of the sample target speech collected in advance from the target object; and decoding is performed based on the phoneme sequence to obtain the recognition text of the speech to be tested; and then cognitive detection is performed based on the recognition text to obtain the cognitive detection result of the target object; and the cognitive detection result includes whether the target object has cognitive impairment.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a cognitive detection device, including a recognition module, a decoding module and a detection module. The recognition module is used to recognize the speech to be tested of the target object based on the target acoustic model to obtain a phoneme sequence of the speech to be tested; wherein the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameters of the target object, and the weight parameters of the target object are obtained based on the training of the sample target speech collected from the target object in advance; the decoding module is used to decode based on the phoneme sequence to obtain the recognition text of the speech to be tested; the detection module is used to perform cognitive detection based on the recognition text to obtain the cognitive detection result of the target object; and the cognitive detection result includes whether the target object has cognitive impairment.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory storing program instructions, and the processor being used to execute the program instructions to implement the cognitive detection method in the above first aspect.

[0008] In order to solve the above technical problem, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, wherein the program instructions are used to implement the cognitive detection method in the above first aspect.

[0009] The above scheme obtains the phoneme sequence of the speech to be tested by recognizing the speech to be tested of the target object based on the target acoustic model; the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the sample target speech collected in advance for the target object for training; and decoding is performed based on the phoneme sequence to obtain the recognition text of the speech to be tested; cognitive detection is then performed based on the recognition text to obtain the cognitive detection result of the target object; and the cognitive detection result includes whether the target object has cognitive impairment. On the one hand, since the weight parameter of the target object is obtained based on the sample target speech collected in advance for the target object for training, it helps to improve the recognition result of the speech to be tested and improve the accuracy of cognitive detection. On the other hand, since cognitive detection is performed based on the recognition text without relying on various biological and chemical examinations, it helps to reduce the detection cost as much as possible and improve the detection experience of the person to be tested. Therefore, the accuracy of cognitive detection can be improved while reducing the detection cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an embodiment of the cognitive detection method of the present application;

[0011] Figure 2 is a schematic diagram of a framework of an embodiment of a reference model;

[0012] Figure 3 yes Figure 1 A schematic diagram of a model of another embodiment of step S11;

[0013] Figure 4 yes Figure 1 Schematic diagram of feature fusion in step S13 of an embodiment;

[0014] Figure 5 yes Figure 1 A schematic diagram of feature fusion in another embodiment of step S13;

[0015] Figure 6 It is a schematic diagram of the framework of an embodiment of the cognitive detection device of the present application;

[0016] Figure 7 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0017] Figure 8 It is a schematic diagram of the framework of an embodiment of the computer-readable storage medium of the present application. Specific embodiments

[0018] The following will combine the accompanying drawings of the specification to elaborate in detail on the solutions of the embodiments of the present application.

[0019] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the present application.

[0020] In this article, the terms "system" and "network" are often used interchangeably. The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this article means two or more than two.

[0021] Please refer to Figure 1 , Figure 1 It is a schematic flowchart of an embodiment of the cognitive detection method of the present application.

[0022] Specifically, it may include the following steps:

[0023] Step S11: Recognize the speech to be measured of the target object based on the target acoustic model to obtain the phoneme sequence of the speech to be measured.

[0024] In an implementation scenario, the target object can be one of the subjects to be detected. Exemplarily, there can be N subjects to be detected, and the target object is the i-th one (i is greater than 0 and less than or equal to N). The specific position of the target object among the subjects to be detected can be selected according to the actual situation and will not be specifically limited here.

[0025] In an implementation scenario, a phoneme is the smallest speech unit divided according to the natural attributes of speech. If analyzed according to the pronunciation actions in a syllable, one action can form one phoneme. Exemplarily, the Chinese syllable "ā" has only one phoneme, "ài" has two phonemes, "dài" has three phonemes, etc. The phoneme sequence of the speech to be measured can be the phoneme sequence composed of different syllables in the speech to be measured, and the phoneme sequence of the speech to be measured can be recognized according to the actual situation and will not be specifically limited here.

[0026] In one implementation scenario, the phoneme sequence of the speech to be tested may be obtained by manual annotation or based on a target acoustic model. The method for obtaining the phoneme sequence of the speech to be tested may be selected according to actual conditions and is not specifically limited here.

[0027] In the disclosed embodiment, the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameters of the target object, and the weight parameters of the target object are obtained based on the training of sample target speech collected in advance for the target object, that is, in order to obtain the weight parameters of the target object, the sample target speech of the target object can be collected first, and then the weight parameters of the target object can be obtained based on the training of the sample target speech of the target object; the weight parameters of the target object can be weighted and trained according to the sample target speech of each target object, so that the target acoustic model can have a higher accuracy when performing speech recognition on target objects with different attributes.

[0028] In an implementation scenario, the target acoustic model is obtained by adding the weight parameters of the target object to the pre-trained reference acoustic model, and the reference acoustic model is trained based on the following sample voices in sequence: the sample first voice of the first sample object, the sample second voice of the second sample object; and: the first sample object and the second sample object both have the same object attributes as the target object. Exemplarily, the first sample object and the second sample object are both cognitively impaired people. At this time, the sample first voice can be cognitively normal voice data in the cognitively impaired people, and the sample second voice can be cognitively impaired voice data in the cognitively impaired people; or, the first sample object can have the same object attributes as the target object and does not have cognitive impairment, and the second sample object has the same object attributes as the target object. For example, the first sample object is a normal population, and the second sample object is a cognitively impaired population. At this time, the sample first voice is ordinary voice data, the sample second voice can be cognitively impaired voice data in the cognitively impaired population, or cognitively normal voice data in the cognitively impaired population, or both. The attributes of the sample objects can be selected according to actual conditions, and are not specifically limited here. Furthermore, the selection of sample objects will vary depending on the target object. If the target object is young people, the sample object should also be young people. If the target object is the elderly, the sample object should also be the elderly. The target object and sample object can be selected according to actual conditions and are not specifically limited here.

[0029] In another implementation scenario, the target acoustic model is obtained by adding the weight parameters of the target object to the pre-trained reference acoustic model, and the reference acoustic model is trained based on the following sample voices in sequence: the sample first voice of the first sample object, the sample second voice of the second sample object, and the sample third voice of the third sample object; and the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object. Exemplarily, the sample first voice can be voice data of the general population, the sample second voice can be cognitively normal voice data in the cognitive impairment population, and the sample third voice can be cognitive impairment voice data in the cognitive impairment population, and the sample voice data can be selected according to actual conditions. And the training method of the reference acoustic model can be selected according to actual conditions, and is not specifically limited here. In the above method, the target acoustic model is obtained by adding weight parameters to the reference acoustic model, which can further improve the generalization effect of the target acoustic model, and the reference acoustic model is obtained by training different data in sequence, which helps to improve the scope of application of the reference acoustic model.

[0030] In a specific implementation scenario, the sample target speech is annotated with a sample phoneme sequence; the weight parameter training step can first perform phoneme recognition on the sample target speech based on a pre-trained reference acoustic model to obtain a first prediction sequence; and the first sample feature output by the hidden layer of the pre-trained reference acoustic model is weighted based on the weight parameter, and the first prediction sequence is predicted based on the first sample feature; then based on the difference between the sample phoneme sequence annotated with the sample target speech and the first prediction sequence, the network parameters of the pre-trained reference acoustic model are maintained and the weight parameters are adjusted. Exemplarily, since the speaking style of people with cognitive impairment is more different than that of people with normal cognition, in order to further optimize the speech recognition effect of people with cognitive impairment, a target acoustic model suitable for the target individual can be pre-trained for the target object. Please refer to Figure 2 , Figure 2 Schematic diagram of the framework of the reference model 1 embodiment, such as Figure 2 As shown in the figure, the acoustic model is a reference acoustic model. Before using the acoustic model to process the sample target speech, the speech features (such as Figure 2The inverse spectrum features shown in the figure), the sample target speech can be obtained by collecting the target object through a reading task; in addition, exemplarily, the acoustic model can output three phonemes (the specific meaning of which can be found in the technical details about the three phonemes, which will not be repeated here), and the first prediction sequence (that is, the phoneme sequence of the sample target speech predicted by the acoustic model) can be obtained based on the three phonemes output by the acoustic model; wherein, the sample phoneme sequence annotated with the sample target speech can be obtained in the form of manual annotation, or by automatically transcribing the sample target speech and performing network recognition to obtain it. The method for obtaining the sample phoneme sequence annotated with the sample target speech can be selected according to the actual situation, and is not specifically limited here. Based on the pre-trained reference acoustic model, for the target object i, a weight vector of the target object can be added in the last hidden layer l. The output formula corresponding to the hidden layer is as follows:

[0031]

[0032] a(c)=2 / (1+e -c )

[0033] Among them, ° represents the multiplication of corresponding elements of the vector, It is a nonlinear transformation process, which can also be understood as an activation function. where i represents the i-th target object, l represents the number of layers, and a(·) is used for weighted processing. Transformed into a value between 0 and 1, that is, the output of the lth layer is determined based on the product of the output of the l-1th layer and the weight. It should be noted that in this process, W lT The network parameters in will not change, and The initial value of can be selected according to actual conditions and is not specifically limited here. Based on the difference between the sample phoneme sequence annotated by the sample target speech and the first prediction sequence, the weight parameters are adjusted on the premise of maintaining the network parameters of the pre-trained reference acoustic model. The above method, by adjusting the weight parameters based on the difference between the sample phoneme sequence annotated by the sample target speech and the first prediction sequence, on the basis of maintaining the network parameters of the pre-trained reference acoustic model, further improves the effect of speech recognition in the process of speech recognition of the target object, thereby improving the accuracy of cognitive detection.

[0034] In an implementation scenario, the training step of training a reference acoustic model may be to train a reference acoustic model based on a sample first voice; and then train a reference acoustic model converged by the sample first voice training based on a sample second voice. Exemplarily, the sample first voice may be sample voice data of a general population. By training the reference acoustic model with sample voice data of a general population, the universality of the reference acoustic model can be made better. The sample second voice may be sample voice data of a cognitively impaired population. The reference acoustic model converged by the sample first voice training based on the sample second voice training can further improve the recognition effect of the reference acoustic model. The sample voice can be selected according to the actual situation, and no specific limitation is made here. In the process of acquiring voice data, it can be acquired through a clip-on microphone or through a recording device. The specific method of acquiring voice data can be selected according to the actual situation, and no specific limitation is made here.

[0035] In another implementation scenario, in order to train a reference acoustic model, the reference acoustic model can be first trained based on the sample first voice, and the pathological voice data can be preprocessed and combined with DFCNN (deep fully convolutional neural network, full sequence convolutional neural network), LSTM (Long Short Term, which can learn long-term dependent information) and other network structures, and the reference acoustic model can be trained based on the sample first voice. At this time, the sample first voice can be the voice data of the general population, illustratively, it can be open source voice data, or it can be recording data in public places. The sample first voice can be selected according to the actual situation, and no specific limitation is made here. At this time, in order to make the reference acoustic model more suitable for people with cognitive impairment, the collected text data of the pathological voice can be divided according to the cognitive impairment label, and divided into two categories: normal cognition and cognitive impairment, wherein the cognitive impairment label can be distinguished according to the detection data of clinical biomarkers, for example, brain imaging, cerebrospinal fluid, etc. The cognitive impairment label can also be distinguished according to the scoring of the scale, and the method of obtaining the cognitive impairment label can be selected according to the actual situation, and no specific limitation is made here. Based on the reference acoustic model trained by the sample second voice and converged by the sample first voice training, that is, on the basis of the reference acoustic model trained by the sample first voice, the reference acoustic model is trained based on the sample second voice, so that the reference acoustic model can learn the voice data of the cognitively impaired population, thereby improving the final recognition effect. The reference acoustic model trained by the sample second voice training is then trained based on the sample set voice; and the sample set voice comes from the sample set, and the sample set is obtained by merging the sample second voice and the sample third voice. The sample set can include two types of sample voice data, namely, normal cognition and cognitive impairment, which are distinguished among the cognitively impaired population. The sample set can also include sample voice data of normal cognition, cognitive impairment and sample voice data of the general population. The sample set can be selected according to the actual situation, and no specific limitation is made here. The above method, by training the reference acoustic model based on different sample voices, helps to improve the scope of use of the reference acoustic model, further improve the recognition effect of the sample target voice, and thus improve the generalization effect of the reference acoustic model.

[0036] It should be noted that, in order to solve the problem of low accuracy of Chinese speech recognition in people with cognitive impairment, the acoustic model can be trained through the Chinese speech of people with cognitive impairment based on methods such as transfer learning, adversarial training and unsupervised adaptation, so that the recognition effect for patients with cognitive impairment can be significantly improved. In this process, transfer learning is to transfer the parameters of the trained model (pre-trained model) to the new model to help the new model training, which can make the generalization ability of the new model training results stronger. Adversarial training can further enhance the robustness of the neural network, that is, in the process of adversarial training, the samples will be mixed with some small perturbations (small changes, but likely to cause misclassification), so that the neural network can adapt to such changes and thus be robust to adversarial samples. Therefore, based on the collected and labeled sample speech data, considering the speech specificity of people with cognitive impairment compared with normal adults, and the differences in speech performance in people with cognitive impairment due to different cognitive states, the acoustic model for people with cognitive impairment can be optimized based on the two-step adaptive method of transfer learning and adversarial training acoustic model. The collection and annotation can be transcribed and annotated by a machine, or collected and annotated manually. Since the speech recognition effect of people with cognitive impairment is poor, the sample speech data of people with cognitive impairment can be processed manually to make the collection and annotation effect better. The collection and annotation method can be selected according to the actual situation and is not specifically limited here. After collecting the sample speech data, the collected sample speech data can be processed using preprocessing technologies such as speech enhancement and speech activity detection technology. The preprocessing can include data noise reduction, data enhancement, etc. The sample speech data can be enhanced to increase the data volume and improve the audio quality. The sentences in the sample speech data can also be disconnected to make the sentence expression clearer. The preprocessing method can be selected according to the actual situation and is not specifically limited here.

[0037] In a specific implementation scenario, the sample set speech is annotated with a sample phoneme sequence and a sample speech object to which the sample set speech belongs. The sample set speech annotation can be extracted by a feature extractor or manually annotated. The sample set speech annotation method can be selected according to the actual situation and is not specifically limited here. Based on the reference acoustic model that converges after the sample second speech training based on the sample set speech training, the sample set speech can be phoneme recognized based on the reference acoustic model that converges after the sample second speech training to obtain a second prediction sequence; the second prediction sequence is predicted based on the second sample feature, and the second sample feature is the feature representation output by the hidden layer of the reference acoustic model that converges after the sample second speech training; object prediction is performed based on the second sample feature to obtain a predicted speech object; based on the first difference between the sample phoneme sequence annotated by the sample set speech and the second prediction sequence, and the second difference between the sample speech object and the predicted speech object, a total loss is obtained; wherein the total loss is positively correlated with the first difference, and the total loss is negatively correlated with the second difference; finally, based on the total loss, the network parameters of the reference acoustic model are adjusted. It should be noted that in this process, the cognitive impairment speech data and cognitive normal speech data are combined, and the adversarial training method is used to make the reference acoustic model more applicable and the recognition result more accurate. In the above method, the total loss is obtained by the first difference and the second difference, and the network parameters of the reference acoustic model are adjusted based on the total loss, so that the recognition effect of the adjusted reference acoustic model is better.

[0038] For further information, see Figure 3 , Figure 3 yes Figure 1 A schematic diagram of a model of another embodiment of step S11 is shown in FIG. Figure 3 As shown, based on the reference acoustic model converged by the sample second speech training, the sample set speech is subjected to phoneme recognition to obtain a second prediction sequence, and the second sample feature is a feature representation output by the hidden layer of the reference acoustic model converged by the sample second speech training. In the figure, the sample set speech is subjected to the feature extractor to obtain the second sample feature. At this time, the object is predicted based on the second sample feature, that is, the second sample feature is extracted by the speaker discrimination model to obtain the predicted speaking object. In the feature extractor M f (x;θ f ), phoneme classification model M y (x;θ y ) and speaker identification model M s (x;θ s), the feature extractor is used to extract the features of the input speech frame, and on this basis, the phoneme classification model and the speaker discrimination model are used to predict the phoneme category and the speaker object corresponding to the input sample combined with the speech. Based on the first difference between the sample phoneme sequence marked by the sample set speech and the second predicted sequence, the first difference can be determined by calculating the loss value, that is, the loss function of the phoneme classification is expressed as:

[0039]

[0040] Where x represents the input, θ f ,θ y In this process, in order to predict the phoneme recognition corresponding to each pronunciation more accurately, the smaller the first difference value, the better the corresponding phoneme recognition effect.

[0041] The second difference between the sample speaker and the predicted speaker can be determined by calculating the loss value, that is, the loss function of speaker discrimination is expressed as:

[0042]

[0043] Where x represents the input, θ f ,θ s Represents the parameter, i represents the i-th target object. In this process, the purpose of training is to obtain the predicted speech object containing less feature information related to the target object. Therefore, the larger the second difference, the better the prediction effect. The total loss is obtained by the first difference and the second difference. The total loss is positively correlated with the first difference and negatively correlated with the second difference. The total loss can be expressed as:

[0044]

[0045] Among them, λ is a trade-off parameter between the phoneme classification loss and the speaker discrimination loss, and the negative sign in front of it indicates adversarial training for speaker discrimination to ensure that the output of the feature extractor does not contain speaker information. It should be noted that the ultimate goal of the reference acoustic model that converges based on the sample set speech training and the sample second speech training is to make the recognition result of the reference acoustic model contain as little feature information related to the target object as possible and contain as much information related to the phoneme sequence as possible, so that the discrimination result can more accurately identify the phoneme sequence.

[0046] Step S12: Decoding is performed based on the phoneme sequence to obtain a recognition text of the speech to be tested.

[0047] In one implementation scenario, before decoding based on the phoneme sequence to obtain the recognized text of the speech to be tested, a target decoding strategy selected by the user from among several decoding strategies may also be obtained; and the several decoding strategies include one-pass decoding and two-pass decoding, during which the user may freely select the one-pass decoding strategy or the two-pass decoding strategy as the target decoding strategy. The above method further improves the user experience by allowing the user to freely select the decoding strategy.

[0048] In one implementation scenario, decoding is performed based on a phoneme sequence. In order to obtain the recognition text of the speech to be tested, the phoneme sequence can be decoded based on a target decoding strategy to obtain the recognition text; and when the target decoding strategy is one-pass decoding, one-pass decoding can be performed based on statistical text probability to obtain the recognition text decoded once as the recognition text of the speech to be tested. When the target decoding strategy is two-pass decoding, the phoneme sequence and the recognition text decoded once are decoded twice based on the N-Gram language or neural network model to obtain the recognition text decoded twice as the recognition text of the speech to be tested. The process of processing text data can select a language model, or can directly perform manual statistics, etc. The method of processing text data can be selected according to the actual situation, and is not specifically limited here.

[0049] In another implementation scenario, decoding is performed based on a phoneme sequence. In order to obtain the recognition text of the speech to be tested, the phoneme sequence can be decoded based on a target decoding strategy to obtain the recognition text; and when the target decoding strategy is one-pass decoding, the phoneme sequence is decoded once based on an N-Gram language model to obtain the recognition text decoded once as the recognition text of the speech to be tested. When the target decoding strategy is two-pass decoding, the phoneme sequence and the recognition text decoded once are decoded twice based on a pre-trained language model to obtain the recognition text decoded twice as the recognition text of the speech to be tested, and the pre-trained language model is a neural network model. The above method uses different language models for decoding, thereby allowing users to further improve the accuracy of the decoding results while improving the user experience.

[0050] In one implementation scenario, the pre-trained language model is trained based on the following sample texts in sequence: the sample first text of the first sample object and the sample second text of the second sample object; and the first sample object has the same object attributes as the target object and does not have cognitive impairment, and the second sample object has the same object attributes as the target object and has cognitive impairment. At this time, the sample first text can be cognitively normal text data among people with cognitive impairment, and the sample second text can be cognitively impaired text data among people with cognitive impairment. The specific sample text data can be selected according to the actual situation and is not specifically limited here.

[0051] In another implementation scenario, the pre-trained language model is trained based on the following sample texts in sequence: the sample first text of the first sample object, the sample second text of the second sample object, and the sample third text of the third sample object; and the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object. At this time, the sample first text can be text data of the general population, the sample second text can be cognitively normal text data among people with cognitive impairment, the sample third text can be cognitively impaired text data among people with cognitive impairment, and the sample third text can also include a collection of cognitively normal text data and cognitively impaired text data among people with cognitive impairment, and both cognitively normal text data and cognitively impaired text belong to the cognitively impaired text set, and the difference between the two can be text data after being distinguished, for example, the text data of people with cognitive impairment can be distinguished according to cognitive impairment labels. Exemplarily, 15% of the words in the text corpus transcribed from the open source speech data can be masked, and a high-precision pre-trained language model can be obtained by joint training of the Masked Language Model (MLM) and the Next Sentence Prediction (NSP) task. A method similar to the two-step adaptive method of the language model based on transfer learning and adversarial training is adopted. First, the text data of normal cognition in the cognitively impaired population is combined to carry out transfer learning methods such as Fine-tune, and further obtain a pre-trained language model adapted to the pathological language of the elderly. Then, combined with all sample texts in the cognitively impaired population, training based on the adversarial training method is carried out to establish an adaptive pre-trained language model. The specific steps can refer to the training method of the aforementioned reference acoustic model, which will not be repeated here. The above method, by using different sample text data to train the pre-trained language model, makes the pre-trained language model more practical and has a better recognition effect on the text of the cognitively impaired population.

[0052] In one implementation scenario, the N-Gram language model includes a first N-Gram language model and a second N-Gram language model, the first N-Gram language model is established based on a sample first text of a first sample object, the second N-Gram language model is established based on at least one of a sample second text of a second sample object and a sample third text of a third sample object, and the recognition text decoded once is obtained by weighting the decoding result of the first N-Gram language model on the phoneme sequence and the decoding result of the second N-Gram language model on the phoneme sequence; and the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object. The sample first text can be text data of the general population, and the sample text of the cognitive impairment population is distinguished to obtain normal text data and cognitive impairment text data in the cognitive impairment population. At this time, the sample second text can be cognitive normal text data, and the third text can be cognitive impairment text data. Exemplarily, the first N-Gram language model is established based on a sample first text of a first sample object, where the sample first text may be sample text data of a general population; the second N-Gram language model is established based on a sample second text of a second sample object, where the sample second text may be text data of a cognitively impaired population; the probabilities of identical word strings in the first N-Gram language model and the second N-Gram language model are interpolated according to a certain ratio, which may be expressed by the following formula:

[0053] p apt (w i |w i-3；i-1 )=(1-ρ)p general (w i |w i-3；i-1 )+ρp elderly (w i |w i-3；i-1 )

[0054] Among them, ρ is the interpolation coefficient, w i-3；i-1 For word string w i-3 w i-2 w i-1 , p qeneral (.), p elderly (.) and p apt (.) are the words w appearing in the first N-Gram language model, the second N-Gram language model and the N-Gram language model respectively. i The probability of reducing the language confusion (Perplexity, PPL) in the training data of people with cognitive impairment can be expressed by the following formula:

[0055]

[0056] Among them, N is the total number of people, and then the interpolation coefficients of the language model of the cognitive impairment population are trained. The interpolation coefficients are the coefficients in the basic model. Since there are more text data of the general population in the model and less text data of the cognitive impairment population, the text data of the general population is finally trained better. Therefore, the interpolation coefficients of the language model of the cognitive impairment population are trained, which can be a better language training effect for the cognitive impairment population. The above method processes the recognition results of the language model to make the language model more suitable for cognitive impairment text data, thereby making the final decoding accuracy higher.

[0057] Step S13: Perform cognitive detection based on the recognized text to obtain cognitive detection results of the target object.

[0058] In one implementation scenario, cognitive detection is performed based on recognized text. In order to obtain cognitive detection results of the target object, an acoustic feature representation can be obtained based on the feature representation output by the hidden layer of the target acoustic model, and feature extraction can be performed based on the recognized text to obtain a language feature representation; cognitive detection is performed based on the acoustic feature representation and the language feature representation to obtain cognitive detection results. The cognitive detection results include whether the target object has cognitive impairment. The above method, by performing cognitive detection based on acoustic feature representation and language feature representation, helps to reduce the detection cost as much as possible, thereby making the cognitive detection results more accurate.

[0059] In a specific implementation scenario, cognitive detection is performed based on the acoustic feature representation and the language feature representation to obtain cognitive detection results. Since the introduction of external knowledge can effectively improve the accuracy of the model and the quality of the task, the acoustic feature representation and the language feature representation can be high-dimensionally spliced, and then the high-dimensional speech features obtained after the high-dimensional splicing are input into the fully connected layer, and finally the speech feature representation that integrates relevant external knowledge information is output. After obtaining the speech feature representation, a series of nonlinear transformations are performed on the speech feature representation to finally obtain the corresponding probability value, and the probability value is compared with the aforementioned label content to finally obtain the cognitive detection result. Exemplarily, since the aforementioned label content includes 0 and 1, when comparing the probability value with the aforementioned label content, a threshold value can be pre-set, for example, the threshold value is set to 0.8, that is, if the probability value is greater than or equal to 0.8, it means that the target object of the cognitive detection result has cognitive impairment, and if the probability value is less than 0.8, it means that the target object of the cognitive detection result does not have cognitive impairment. Since integrated learning, deep learning and other methods are used in the cognitive testing process to construct a cognitive state classification prediction model, and then a series of nonlinear transformations are performed on the speech feature representation through the cognitive state classification prediction model, the corresponding probability value is finally obtained, thereby making the results of cognitive testing more accurate.

[0060] In one implementation scenario, based on the feature representation output by the hidden layer of the target acoustic model, in order to obtain the acoustic feature representation, the feature representation output by the hidden layer of the target acoustic model can be used as the acoustic feature representation. The method for determining the acoustic feature representation can be determined according to actual conditions and is not specifically limited here.

[0061] In another implementation scenario, based on the feature representation output by the hidden layer of the target acoustic model, in order to obtain the acoustic feature representation, the feature representation output by the hidden layer of the target acoustic model can be used as the first acoustic feature, and based on the feature difference of the acoustic parameters between the test speech and the reference speech at several speech levels, the second acoustic feature is obtained; and the several speech levels include at least one of the following: sound frame level, syllable level, phrase level, sentence level, and the reference speech includes sample speech collected from a sample object with the same object attributes as the target object and without cognitive impairment, wherein the reference speech can be a sample second sample speech of the second sample object. Then, based on the first acoustic feature and the second acoustic feature, fusion is performed to obtain the acoustic feature representation. In the above method, by fusing based on the first acoustic feature and the second acoustic feature, the acoustic feature representation is determined, and in view of the pronunciation difference between patients with cognitive impairment and normal people, acoustic diversity is reflected, thereby improving the accuracy of cognitive detection.

[0062] For further information, see Figure 4 , Figure 4 yes Figure 1 The feature fusion diagram of step S13 in the embodiment of FIG. Figure 4As shown in FIG. 1 , the feature representation output by the hidden layer of the target acoustic model, that is, the feature representation output by intercepting the last layer of activated feature maps before the fully connected layer of the network, is used as the first acoustic feature. Based on the feature differences of the test speech and the reference speech on acoustic parameters at several speech levels, the second acoustic feature is obtained. The acoustic feature differences between patients with cognitive impairment and normal people are analyzed from the perspectives of voice physical parameters such as fundamental frequency, amplitude, formant, fundamental frequency perturbation, amplitude perturbation, etc. Local and global speech acoustic features are extracted at the speech frame, syllable, phrase and sentence levels, and feature statistics are calculated to construct the second acoustic feature. The second acoustic feature and the first acoustic feature are obtained through the pathological speech of the elderly. At this time, the second acoustic feature parameters may include frequency, energy, etc., wherein the features corresponding to the frequency parameters may include fundamental frequency (Pitch), formant frequency (Formant Freq), etc., the features corresponding to the energy parameters may include loudness, harmonic noise ratio (HNR), etc., the statistics corresponding to the fundamental frequency may include the mean value of Pitch at the frame / byte / word level, the maximum value of Pitch at the frame / byte / word level, the minimum value of Pitch at the frame / byte / word level, etc.; the formant frequency may include the mean value of Formant Freq at the frame / byte / word level, the maximum value of Formant Freq at the frame / byte / word level, the minimum value of Formant Freq at the frame / byte / word level, etc.; the loudness may include the mean value of Loudness at the frame / byte / word level, the maximum value of Loudness at the frame / byte / word level, the minimum value of Loudness at the frame / byte / word level, etc.; the harmonic noise ratio may include the mean value of HNR at the frame / byte / word level, the maximum value of HNR at the frame / byte / word level, the minimum value of HNR at the frame / byte / word level, etc. Then the second acoustic feature is concatenated with the first acoustic feature to obtain the acoustic feature representation.

[0063] In one implementation scenario, feature extraction is performed based on the recognized text. In order to obtain a language feature representation, text features can be extracted based on the recognized text to obtain a first language feature, wherein the text feature can be a text feature representation based on lexical and syntactic analysis. For example, it can be represented as vocabulary, syntax, etc. Vocabulary can include the number / proportion of nouns, the number / proportion of verbs, the number / proportion of pronouns, the number / proportion of different vocabulary, etc.; syntax can include the number / proportion of predicate syntax, the number / proportion of substantive syntax, the number / proportion of modifying syntax, etc. Based on the dictionary features and annotation features of each word segment in the recognized text, the second language feature is obtained; and the dictionary features are obtained based on the pathological dictionary knowledge base, and the annotation features are obtained based on the recursive neural network and the long short-term memory network of the attention constraint mechanism; and then the first language feature and the second language feature are fused to obtain a language feature representation. In the above method, by fusing the first language feature with the second language feature, the language feature representation for cognitive impairment is made more accurate, further improving the accuracy of cognitive detection.

[0064] In a specific implementation scenario, based on the dictionary features and annotation features of each word segment in the recognized text, in order to obtain the second language features, each word segment can be spliced ​​based on the dictionary features and annotation features of the word segment to obtain the spliced ​​features of the word segment; attention processing is performed based on the spliced ​​features of each word segment to obtain the second language features. The above method further obtains the second language features based on the speech recognition text representation information and external linguistic knowledge information, thereby improving the accuracy of cognitive detection.

[0065] For further information, see Figure 5 , Figure 5 yes Figure 1 A schematic diagram of feature fusion in another embodiment of step S13 is shown in FIG. Figure 5 As shown in the figure, first, we carry out the word segmentation work for pathological speech transcription text, and build a dictionary feature vector c(w i ). For words missing from the dictionary, in c(w i ) Each dimension of the feature vector is padded by adding 0 or interpolation prediction. Secondly, the above word segmentation results are used as input, and the pre-trained model is used to initialize the weights of the word embedding layer in the ACRNN model to learn and obtain the low-dimensional spatial representation of each word in the speech recognition text. This low-dimensional feature vector is used as the input of the long short-term memory layer to generate a high-dimensional annotation information representation of each word (h1, h2, ..., h T ), where h i is the hidden state of LSTM at time step i. Finally, the word annotation information feature representation and the dictionary feature vector c(w i ) is used as the input of the self-attention layer. By learning the connection function between the two, the attention distribution of the given text is constrained, thereby identifying the language feature representation that is more consistent with the vocabulary prior knowledge and is used for cognitive impairment risk prediction. The connection function can be expressed as follows:

[0066] f c (h i ,c(w i ))=tanh(W c [h i ||c(w i )]+b c )

[0067] Among them, || represents the splicing operation, W c and b c is a learning parameter. In this process, h i and w iThe features are fused, that is, the low-dimensional feature vector is concatenated with the high-dimensional feature representation, and then a nonlinear transformation is performed.

[0068] The above scheme obtains the phoneme sequence of the speech to be tested by recognizing the speech to be tested of the target object based on the target acoustic model; the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the sample target speech collected in advance for the target object for training; and decoding is performed based on the phoneme sequence to obtain the recognition text of the speech to be tested; cognitive detection is then performed based on the recognition text to obtain the cognitive detection result of the target object; and the cognitive detection result includes whether the target object has cognitive impairment. On the one hand, since the weight parameter of the target object is obtained based on the sample target speech collected in advance for the target object for training, it helps to improve the recognition result of the speech to be tested and improve the accuracy of cognitive detection. On the other hand, since cognitive detection is performed based on the recognition text without relying on various biological and chemical examinations, it helps to reduce the detection cost as much as possible and improve the detection experience of the person to be tested. Therefore, the accuracy of cognitive detection can be improved while reducing the detection cost.

[0069] See also Figure 6 , Figure 6 It is a schematic diagram of the framework of an embodiment of the cognitive detection device of the present application. The cognitive detection device 60 includes a recognition module 61, a decoding module 62 and a detection module 63. Among them, the recognition module 61 is used to recognize the speech to be tested of the target object based on the target acoustic model to obtain the phoneme sequence of the speech to be tested; the feature representation output by the hidden layer of the target acoustic model is weighted based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the sample target speech training collected from the target object in advance; the decoding module 62 is used to decode based on the phoneme sequence to obtain the recognition text of the speech to be tested; the detection module 63 is used to perform cognitive detection based on the recognition text to obtain the cognitive detection result of the target object; and the cognitive detection result includes whether the target object has cognitive impairment.

[0070] The above scheme, on the one hand, helps to improve the recognition results of the tested speech and the accuracy of cognitive testing because the weight parameters of the target object are obtained based on the sample target speech collected in advance. On the other hand, because cognitive testing is based on recognized text without relying on various biological and chemical tests, it helps to reduce the testing cost as much as possible and improve the testing experience of the person to be tested. Therefore, it is possible to improve the accuracy of cognitive testing while reducing the testing cost.

[0071] In some disclosed embodiments, a target acoustic model is obtained by adding weight parameters of a target object to a pre-trained reference acoustic model, and the reference acoustic model is trained in sequence based on the following sample speech: a sample first speech of a first sample object, a sample second speech of a second sample object, and a sample third speech of a third sample object; and the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

[0072] Therefore, by adding weight parameters to the reference acoustic model to obtain the target acoustic model, the generalization effect of the target acoustic model can be further improved, and the reference acoustic model is obtained by training with different data in turn, which helps to improve the scope of application of the reference acoustic model.

[0073] In some disclosed embodiments, the recognition module 61 includes a recognition submodule, which is used to perform phoneme recognition on the sample target speech based on a pre-trained reference acoustic model to obtain a first prediction sequence; and the first sample features output by the hidden layer of the pre-trained reference acoustic model are weighted based on weight parameters, and the first prediction sequence is predicted based on the first sample features; the recognition module 61 also includes an adjustment submodule, which is used to maintain the network parameters of the pre-trained reference acoustic model and adjust the weight parameters based on the difference between the sample phoneme sequence annotated by the sample target speech and the first prediction sequence.

[0074] Therefore, by adjusting the weight parameters based on the difference between the sample phoneme sequence annotated based on the sample target speech and the first prediction sequence while maintaining the network parameters of the pre-trained reference acoustic model, the effect of speech recognition is further improved in the process of speech recognition of the target object, thereby improving the accuracy of cognitive detection.

[0075] In some disclosed embodiments, the recognition module 61 includes a first training submodule, which is used to train a reference acoustic model based on a sample first speech; the recognition module 61 includes a second training submodule, which is used to train a reference acoustic model converged by the sample first speech training based on a sample second speech; the recognition module 61 also includes a third training submodule, which is used to train a reference acoustic model converged by the sample second speech training based on a sample set speech; and the sample set speech comes from a sample set, and the sample set is obtained by merging the sample second speech and the sample third speech.

[0076] Therefore, by training the reference acoustic model based on different sample speech, it helps to improve the scope of use of the reference acoustic model, further improve the recognition effect of the sample target speech, and then improve the generalization effect of the reference acoustic model.

[0077] In some disclosed embodiments, the sample set speech is annotated with a sample phoneme sequence and a sample speech object to which the sample set speech belongs; the third training submodule includes a recognition unit, which is used to perform phoneme recognition on the sample set speech based on a reference acoustic model that has converged through sample second speech training to obtain a second prediction sequence; and the second prediction sequence is predicted based on a second sample feature, and the second sample feature is a feature representation output by a hidden layer of the reference acoustic model that has converged through sample second speech training; the third training submodule includes a prediction unit, which is used to perform object prediction based on the second sample feature to obtain a predicted speech object; the third training submodule includes an acquisition unit, which is used to obtain a total loss based on a first difference between the sample phoneme sequence annotated by the sample set speech and the second prediction sequence, and a second difference between the sample speech object and the predicted speech object; and the total loss is positively correlated with the first difference, and negatively correlated with the second difference; the third training submodule also includes an adjustment unit, which is used to adjust the network parameters of the reference acoustic model based on the total loss.

[0078] Therefore, the total loss is obtained by using the first difference and the second difference, and the network parameters of the reference acoustic model are adjusted based on the total loss, so that the recognition effect of the adjusted reference acoustic model can be better.

[0079] In some disclosed embodiments, the cognitive detection device 60 includes an acquisition module, which is used to acquire a target decoding strategy selected by a user from several decoding strategies; and the several decoding strategies include one-pass decoding and two-pass decoding; the decoding module 62 includes a decoding submodule, which is used to decode the phoneme sequence based on the target decoding strategy to obtain a recognized text; and when the target decoding strategy is one-pass decoding, the phoneme sequence is decoded once based on the N-Gram language model to obtain the recognized text decoded once as the recognized text of the speech to be tested, and when the target decoding strategy is two-pass decoding, the phoneme sequence and the recognized text decoded once are decoded twice based on the pre-trained language model to obtain the recognized text decoded twice as the recognized text of the speech to be tested, and the pre-trained language model is a neural network model.

[0080] Therefore, by allowing users to freely choose decoding strategies, improving user experience and using different language models for decoding, the accuracy of decoding results can be further improved while users are improving their experience.

[0081] In some disclosed embodiments, the pre-trained language model is trained based on the following sample voices in sequence: a sample first voice of a first sample object, a sample second voice of a second sample object, and a sample third voice of a third sample object; and the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

[0082] Therefore, by using different sample text data to train the pre-trained language model, the pre-trained language model can be made more practical and have a better recognition effect on the text of people with cognitive impairment.

[0083] In some disclosed embodiments, the N-Gram language model includes a first N-Gram language model and a second N-Gram language model, the first N-Gram language model is established based on a sample first speech of a first sample object, the second N-Gram language model is established based on at least one of a sample second speech of a second sample object and a sample third speech of a third sample object, and the recognition text decoded once is obtained by weighting the decoding results of the phoneme sequence by the first N-Gram language model and the decoding results of the phoneme sequence by the second N-Gram language model; the second sample object has the same object attributes as the target object and does not have cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

[0084] Therefore, by processing the recognition results of the language model, the language model can be made more suitable for cognitive impairment text data, thereby making the final decoding accuracy higher.

[0085] In some disclosed embodiments, the detection module 63 includes an extraction submodule, which is used to obtain an acoustic feature representation based on the feature representation output by the hidden layer of the target acoustic model, and to perform feature extraction based on the recognized text to obtain a language feature representation; the detection module 63 also includes a detection submodule, which is used to perform cognitive detection based on the acoustic feature representation and the language feature representation to obtain a cognitive detection result.

[0086] Therefore, conducting cognitive testing based on acoustic feature representation and language feature representation can help reduce the testing cost as much as possible, thereby making the cognitive testing results more accurate.

[0087] In some disclosed embodiments, the extraction submodule includes an output unit, which is used to represent the feature output by the hidden layer of the target acoustic model as a first acoustic feature, and to obtain a second acoustic feature based on the feature difference between the test speech and the reference speech regarding acoustic parameters at several speech levels; and the several speech levels include at least one of the following: a sound frame level, a syllable level, a phrase level, and a sentence level, and the reference speech includes a sample speech collected from a sample object having the same object attributes as the target object and without cognitive impairment; the extraction submodule also includes a first fusion unit, which is used to fuse the first acoustic feature and the second acoustic feature to obtain an acoustic feature representation.

[0088] Therefore, by fusing the first acoustic feature and the second acoustic feature to determine the acoustic feature representation, the acoustic diversity is reflected in view of the pronunciation differences between patients with cognitive impairment and normal people, thereby improving the accuracy of cognitive detection.

[0089] In some disclosed embodiments, the extraction submodule includes an extraction unit, which is used to extract text features based on the recognized text to obtain a first language feature, and to obtain a second language feature based on the dictionary features and annotation features of each word segment in the recognized text; and the dictionary features are obtained based on a pathological dictionary knowledge base, and the annotation features are obtained based on a recursive neural network and a long short-term memory network with an attention constraint mechanism; the extraction submodule also includes a second fusion unit, which is used to fuse the first language feature and the second language feature to obtain a language feature representation.

[0090] Therefore, by fusing the first language features with the second language features, the language feature representation for cognitive impairment can be made more accurate, further improving the accuracy of cognitive detection.

[0091] In some disclosed embodiments, the extraction unit is further used to perform splicing on each word segment based on the dictionary features and annotation features of the word segment to obtain the splicing features of the word segment; and then perform attention processing based on the splicing features of each word segment to obtain the second language features.

[0092] Therefore, by further obtaining second language features based on speech recognition text representation information and external linguistic knowledge information, the accuracy of cognitive detection is improved.

[0093] See also Figure 7 , Figure 7: is a schematic diagram of a framework of an embodiment of an electronic device of the present application. The electronic device 70 includes a memory 71 and a processor 72 coupled to each other, the memory 71 stores program instructions, and the processor 72 is used to execute the program instructions to implement the steps in any of the above-mentioned cognitive detection method embodiments. Specifically, the electronic device 70 may include but is not limited to: a desktop computer, a laptop computer, a server, a mobile phone, a tablet computer, etc., which are not limited here.

[0094] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above-mentioned cognitive detection method embodiments. The processor 72 may also be referred to as a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip having the ability to process signals. The processor 72 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 72 may be implemented by an integrated circuit chip.

[0095] The above scheme, on the one hand, helps to improve the recognition results of the tested speech and the accuracy of cognitive testing because the weight parameters of the target object are obtained based on the sample target speech collected in advance. On the other hand, because cognitive testing is based on recognized text without relying on various biological and chemical tests, it helps to reduce the testing cost as much as possible and improve the testing experience of the person to be tested. Therefore, it is possible to improve the accuracy of cognitive testing while reducing the testing cost.

[0096] See also Figure 8 , Figure 8 The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor, and the program instructions 81 are used to implement the steps in any of the above-mentioned cognitive detection method embodiments.

[0097] The above scheme, on the one hand, helps to improve the recognition results of the tested speech and the accuracy of cognitive testing because the weight parameters of the target object are obtained based on the sample target speech collected in advance. On the other hand, because cognitive testing is based on recognized text without relying on various biological and chemical tests, it helps to reduce the testing cost as much as possible and improve the testing experience of the person to be tested. Therefore, it is possible to improve the accuracy of cognitive testing while reducing the testing cost.

[0098] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0099] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0100] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A cognitive detection method, characterized in that: include: The target object's speech to be tested is recognized based on the target acoustic model to obtain a phoneme sequence of the speech to be tested; wherein the feature representation output by the hidden layer of the target acoustic model is obtained by weighting based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the training of the sample target speech collected from the target object in advance; the target acoustic model is obtained by adding the weight parameter of the target object to the pre-trained reference acoustic model; Decoding is performed based on the phoneme sequence to obtain a recognition text of the speech to be tested; Perform cognitive testing based on the recognized text to obtain cognitive testing results of the target object; wherein the cognitive testing results include whether the target object has cognitive impairment; Among them, the training steps of the reference acoustic model include: training the reference acoustic model based on the sample first speech; training the reference acoustic model converged by the sample first speech training based on the sample second speech; obtaining the total loss based on the first difference between the sample phoneme sequence marked by the sample set speech and the second prediction sequence, and the second difference between the sample speech object to which the sample set speech belongs marked by the sample set speech and the predicted speech object; adjusting the network parameters of the reference acoustic model based on the total loss; wherein the second prediction sequence and the predicted speech object are both predicted based on the second sample feature, the second sample feature is the feature representation output by the hidden layer of the reference acoustic model converged by the sample second speech training, the total loss is positively correlated with the first difference, and negatively correlated with the second difference, the sample first speech, the sample second speech and the sample set speech are different sample speech.

2. The method according to claim 1, characterized in that The sample first speech is the speech of the first sample object, the sample second speech is the speech of the second sample object, and the sample set speech is obtained by combining the sample second speech and the sample third speech of the third sample object; The second sample object has the same object attributes as the target object and has no cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

3. The method according to claim 1, characterized in that The sample target speech is annotated with a sample phoneme sequence; the weight parameter training step includes: Performing phoneme recognition on the sample target speech based on a pre-trained reference acoustic model to obtain a first prediction sequence; wherein the first sample feature output by the hidden layer of the pre-trained reference acoustic model is weighted based on the weight parameter, and the first prediction sequence is predicted based on the first sample feature; Based on the difference between the sample phoneme sequence annotated by the sample target speech and the first predicted sequence, the network parameters of the pre-trained reference acoustic model are maintained, and the weight parameters are adjusted.

4. The method according to claim 1, characterized in that: Before decoding based on the phoneme sequence to obtain the recognition text of the speech to be tested, the method further includes: Obtaining a target decoding strategy selected by a user from among a plurality of decoding strategies; wherein the plurality of decoding strategies include one-pass decoding and two-pass decoding; The decoding based on the phoneme sequence to obtain the recognition text of the speech to be tested includes: The phoneme sequence is decoded based on the target decoding strategy to obtain the recognition text; wherein, when the target decoding strategy is the one-pass decoding, the phoneme sequence is decoded once based on the N-Gram language model to obtain the recognition text decoded once as the recognition text of the speech to be tested, and when the target decoding strategy is the two-pass decoding, the phoneme sequence and the one-pass decoded recognition text are decoded twice based on the pre-trained language model to obtain the two-pass decoded recognition text as the recognition text of the speech to be tested, and the pre-trained language model is a neural network model.

5. The method according to claim 4, characterized in that The pre-trained language model is trained based on the following sample texts in sequence: a sample first text of a first sample object, a sample second text of a second sample object, and a sample third text of a third sample object; The second sample object has the same object attributes as the target object and has no cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

6. The method according to claim 5, characterized in that The N-Gram language model includes a first N-Gram language model and a second N-Gram language model, wherein the first N-Gram language model is established based on a sample first text of a first sample object, and the second N-Gram language model is established based on at least one of a sample second text of a second sample object and a sample third text of a third sample object, and the recognition text decoded in one pass is obtained by weighting a decoding result of the phoneme sequence by the first N-Gram language model and a decoding result of the phoneme sequence by the second N-Gram language model; The second sample object has the same object attributes as the target object and has no cognitive impairment, the third sample object has the same object attributes as the target object and has cognitive impairment, and the first sample object is different from the second sample object and the third sample object.

7. The method according to claim 1, characterized in that The performing cognitive detection based on the recognized text to obtain the cognitive detection result of the target object includes: Based on the feature representation output by the hidden layer of the target acoustic model, an acoustic feature representation is obtained, and based on the recognition text, feature extraction is performed to obtain a language feature representation; A cognitive test is performed based on the acoustic feature representation and the language feature representation to obtain the cognitive test result.

8. The method according to claim 7, characterized in that The step of obtaining the acoustic feature representation based on the feature representation output by the hidden layer of the target acoustic model comprises: The feature representation output by the hidden layer of the target acoustic model is used as the first acoustic feature, and the second acoustic feature is obtained based on the feature difference of the acoustic parameters between the test speech and the reference speech at several speech levels; wherein the several speech levels include at least one of the following: a sound frame level, a syllable level, a phrase level, and a sentence level, and the reference speech includes a sample speech collected from a sample object having the same object attributes as the target object and having no cognitive impairment; The acoustic feature representation is obtained by fusing the first acoustic feature and the second acoustic feature.

9. The method according to claim 7, characterized in that: The extracting features based on the recognized text to obtain language feature representation includes: Extracting text features based on the recognized text to obtain a first language feature, and obtaining a second language feature based on the dictionary features and annotation features of each word segment in the recognized text; wherein the dictionary features are obtained based on a pathological dictionary knowledge base, and the annotation features are obtained based on a recursive neural network and a long short-term memory network with an attention constraint mechanism; The language feature representation is obtained by fusing the first language feature and the second language feature.

10. The method according to claim 9, characterized in that The step of obtaining the second language features based on the dictionary features and annotation features of each word segment in the recognized text includes: For each of the segmented words, splicing is performed based on the dictionary features and annotation features of the segmented words to obtain the splicing features of the segmented words; Attention processing is performed based on the concatenated features of the individual word segments to obtain the second language features.

11. A cognitive detection device, characterized in that: include: A recognition module, used for recognizing the speech to be tested of the target object based on the target acoustic model to obtain the phoneme sequence of the speech to be tested; wherein the feature representation output by the hidden layer of the target acoustic model is obtained by weighting based on the weight parameter of the target object, and the weight parameter of the target object is obtained based on the training of the sample target speech collected from the target object in advance; the target acoustic model is obtained by adding the weight parameter of the target object to the pre-trained reference acoustic model; A decoding module, used for decoding based on the phoneme sequence to obtain a recognition text of the speech to be tested; A detection module, configured to perform cognitive detection based on the recognized text to obtain a cognitive detection result of the target object; wherein the cognitive detection result includes whether the target object has cognitive impairment; Among them, the training steps of the reference acoustic model include: training the reference acoustic model based on the sample first speech; training the reference acoustic model converged by the sample first speech training based on the sample second speech; obtaining the total loss based on the first difference between the sample phoneme sequence marked by the sample set speech and the second prediction sequence, and the second difference between the sample speech object to which the sample set speech belongs marked by the sample set speech and the predicted speech object; adjusting the network parameters of the reference acoustic model based on the total loss; wherein the second prediction sequence and the predicted speech object are both predicted based on the second sample feature, the second sample feature is the feature representation output by the hidden layer of the reference acoustic model converged by the sample second speech training, the total loss is positively correlated with the first difference, and negatively correlated with the second difference, the sample first speech, the sample second speech and the sample set speech are different sample speech.

12. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the cognitive detection method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the cognitive detection method according to any one of claims 1 to 10.

Images