A somatization disorder identification method based on voice self-supervised learning

Abstract

The embodiment of the application provides a somatization disorder identification method based on voice self-supervised learning. After obtaining the to-be-verified voice information of a target object, the to-be-verified voice information is subjected to frame division and noise reduction processing to obtain a plurality of frames of first voice information from which noise is removed. Each frame of the first voice information is input into a target somatization disorder identification model to obtain a first feature fusion vector of the to-be-verified voice information. Based on the first feature fusion vector, a predicted identifier corresponding to the to-be-verified voice information is obtained, and the predicted identifier indicates whether the target object has somatization disorder. Since the target somatization disorder model is a model trained based on a self-supervised contrast coding model, the model can be used to learn and mine the supervised features of the voice sample under the condition that the voice sample label is scarce. In addition, the model can be used to automatically, quickly and accurately assign a health status identifier to the corresponding target object based on the voice information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, and in particular to a somatization disorder identification method based on speech self-supervised learning. BACKGROUND

[0002] Somatization disorder is a group of neurosis characterized by persistent worry or belief in various somatic symptoms. The speech information of patients with somatization disorder is different from that of normal people. Based on this, the health status of the target object can be determined by analyzing the speech information of the target object.

[0003] At present, the method for identifying whether the target object has somatization disorder is usually to ask the target object by artificial means using a pre-set questionnaire, and then the artificial combines the tone, speed, facial expression of the target object when answering the question and the content of the target object's answer, scores the target object's answer according to personal experience, and then determines whether the target object has somatization disorder according to the score.

[0004] However, the traditional somatization disorder identification method has the problem of low recognition efficiency. SUMMARY

[0005] Therefore, it is necessary to provide a somatization disorder identification method based on speech self-supervised learning, which can improve the somatization disorder identification efficiency.

[0006] In a first aspect, the embodiments of the present application provide a somatization disorder identification method based on speech self-supervised learning, comprising:

[0007] obtaining the to-be-verified speech information of the target object;

[0008] frame and noise reduction processing the to-be-verified speech information to obtain a plurality of frames of first speech information with noise removed;

[0009] inputting each frame of the first speech information into a target somatization disorder identification model to obtain a first feature fusion vector of the to-be-verified speech information; the target somatization disorder model is a model trained based on a self-supervised contrastive coding model; the first feature fusion vector is a feature vector that fuses the first speech information of each frame;

[0010] based on the first feature fusion vector, obtaining a predicted identifier corresponding to the to-be-verified speech information, the predicted identifier being used to indicate whether the target object has somatization disorder.

[0011] In a feasible implementation manner, based on the first feature fusion vector, obtaining a predicted identifier corresponding to the to-be-verified speech information, comprises:

[0012] Based on the first feature fusion vector, target feature data of the to-be-verified voice information is extracted;

[0013] Based on the target feature data and a preset parameter threshold, a predicted identifier corresponding to the to-be-verified voice information is determined.

[0014] In a feasible implementation manner, the training method of the target somatization disorder recognition model comprises:

[0015] A plurality of training sample data and a plurality of test sample data are obtained; each training sample data and each test sample data comprises second voice information of a sample object and a health state identifier corresponding to the second voice information, and the health state identifier is used to indicate whether the sample object corresponding to the second voice information has a somatization disorder;

[0016] The second voice information in each sample training data is subjected to frame division and noise reduction processing respectively, and a plurality of frames of third voice information corresponding to the second voice information in each sample training data is obtained;

[0017] Each frame of third voice information is input into an initial somatization disorder recognition model for feature fusion, a first feature fusion vector of the third voice information is determined, and the first feature fusion vector is a vector used to distinguish significant features in each third voice information;

[0018] Based on the first feature fusion vector, a predicted identifier corresponding to the second voice information of each test sample data is obtained; the predicted identifier is used to indicate predicted information of whether the sample object corresponding to the test sample data has a somatization disorder;

[0019] Based on each predicted identifier, the health state identifier in each test sample data, and the number of categories of the health state identifier, the accuracy of the initial somatization disorder recognition model is determined;

[0020] Based on the accuracy and a preset accuracy threshold, the parameters of the initial somatization disorder recognition model are adjusted until the accuracy is higher than the preset accuracy threshold, and a target somatization disorder recognition model is obtained.

[0021] In a feasible implementation manner, each frame of third voice information is input into an initial somatization disorder recognition model for feature fusion to determine a first feature fusion vector, which comprises:

[0022] Each frame of third voice information is input into an initial somatization disorder recognition model for analysis to obtain predicted voice information, the predicted voice information is voice information of a (t+k) th frame predicted based on third voice information of a t th frame, and t and k are both greater than or equal to 1 and are positive integers;

[0023] The parameters of the initial body integrity disorder recognition model are adjusted based on the third voice information of the t+kth frame and the predicted voice information until a preset convergence condition is reached, and the first feature fusion vector is output.

[0024] In a possible implementation, the third voice information of each frame is input into the initial body integrity disorder recognition model for analysis to obtain the predicted voice information, including:

[0025] The third voice information of each frame is input into the nonlinear encoder in the initial body integrity disorder recognition model to obtain a feature vector corresponding to the third voice information of each frame;

[0026] The feature vectors are input into the autoregressive sub-model in the initial body integrity disorder recognition model to obtain a second feature fusion vector; the second feature fusion vector is a vector obtained based on context semantic feature fusion of the third voice information corresponding to each frame before the t+1th frame;

[0027] The second feature fusion vector is multiplied by a preset linear matrix to obtain the predicted voice information.

[0028] In a possible implementation, the parameters of the initial body integrity disorder recognition model are adjusted based on the third voice information corresponding to the t+kth frame and the predicted voice information until a preset convergence condition is reached, and the first feature fusion vector is output, including:

[0029] The mutual information between the second feature fusion vector and the predicted voice information is determined.

[0030] When the mutual information does not reach a maximum value, the parameters in the autoregressive sub-model are adjusted, and the feature vectors are input into the adjusted autoregressive sub-model to determine a new second feature fusion vector.

[0031] Based on the new second feature fusion vector and a preset linear matrix, a new predicted voice information is determined.

[0032] According to the new predicted voice information and the new second feature fusion vector, a new mutual information is determined until the mutual information reaches a maximum value, and a similarity between the third voice information corresponding to the t+kth frame and the new predicted voice information is calculated.

[0033] Based on the similarity and a preset loss function, a loss value is determined, and when the loss value is less than or equal to a preset loss threshold, the first fusion feature vector is output.

[0034] In a possible implementation, based on the first feature fusion vector, a predicted identity corresponding to the second voice information of each test sample data is obtained, including:

[0035] Based on the first feature fusion vector, target feature data of second speech information in each test sample data is extracted;

[0036] Based on each target feature data and a preset parameter threshold, a prediction identifier corresponding to the second speech information of each test sample data is obtained.

[0037] In a feasible implementation manner, based on each prediction identifier, a health status identifier in each test sample data, and a number of categories of health status identifiers, an accuracy of an initial somatization disorder recognition model is determined, including:

[0038] The health status identifier in each test sample data and the prediction identifier corresponding to each test sample data are compared, and a recall rate of a somatization disorder prediction result of the test sample data is determined.

[0039] Based on the recall rate and the number of categories of health status identifiers, a non-weighted average recall rate is determined as the accuracy of the initial somatization disorder recognition model.

[0040] In a second aspect, an embodiment of the present application provides a somatization disorder recognition device based on self-supervised learning, including:

[0041] A first acquisition module is configured to acquire to-be-verified speech information of a target object.

[0042] A first preprocessing module is configured to perform frame division and noise reduction processing on the to-be-verified speech information, to obtain a plurality of frames of first speech information with noise removed.

[0043] A second acquisition module is configured to input each frame of the first speech information into a target somatization disorder recognition model, to acquire a first feature fusion vector of the to-be-verified speech information; the target somatization disorder model is a model trained based on a self-supervised contrastive coding model; and the first feature fusion vector is a feature vector fused from each frame of the first speech information.

[0044] A third acquisition module is configured to acquire, based on the first feature fusion vector, a prediction identifier corresponding to the to-be-verified speech information, the prediction identifier being used to indicate whether the target object has a somatization disorder.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, the memory storing a computer program, and the processor implementing steps of a method in any one of the embodiments of the first aspect and the second aspect when executing the computer program.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program implements steps of a method in any one of the embodiments of the first aspect and the second aspect when executed by a processor.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program which, when executed by a processor, implements the steps of the method in any embodiment of the first aspect and the second aspect.

[0048] The technical scheme provided by the embodiments of the present application can achieve at least the following beneficial effects:

[0049] The somatization disorder identification method based on voice self-supervised learning provided by the embodiments of the present application can achieve at least the following beneficial effects: BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 FIG. 1 is a schematic diagram of an electronic device according to an example embodiment of the present application;

[0051] Figure 2 FIG. 2 is a schematic diagram of another electronic device according to an example embodiment of the present application;

[0052] Figure 3 FIG. 3 is a flowchart of a somatization disorder identification method based on voice self-supervised learning according to an example embodiment of the present application;

[0053] Figure 4 FIG. 4 is a flowchart of a somatization disorder identification model training method according to an example embodiment of the present application;

[0054] Figure 5 FIG. 5 is a schematic diagram of predicted voice information according to an example embodiment of the present application;

[0055] Figure 6 FIG. 6 is a schematic diagram of a somatization disorder identification device based on voice self-supervised learning according to an example embodiment of the present application. DETAILED DESCRIPTION

[0056] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description herein relates to the drawings, in which the same numbers in different drawings represent the same or similar elements throughout. The following detailed description is not representative of all embodiments consistent with the present application. Rather, it is merely exemplary of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0057] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0058] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one piece of information from another. For example, a first information can also be termed a second information, and, similarly, a second information can also be termed a first information, without departing from the scope of the present application. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining" or "in response to ascertaining."

[0059] The disclosed embodiments can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known terminal devices, computer systems, environments, and / or configurations that can be suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computers, server computers, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputers, mainframe computers, and distributed cloud computing environments that include any of the above systems or the like.

[0060] Electronic devices such as terminal devices, computer systems, servers, and the like can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types. Computer systems / servers can be practiced in distributed cloud-computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud-computing environment, program modules can be located in local or remote computer system storage media including storage devices.

[0061] Somatization disorder is a group of neurosis characterized by persistent worry or belief in ideas of various somatic symptoms. Since the speech signal contains not only the physiological information of the patient, but also the psychological information of the patient, and the speech signal has the advantages of universality, non-invasiveness and easy collection, it has been effectively applied in the identification of mental disorders such as depression, so it is of great significance to use speech information for early identification of somatization disorder.

[0062] In the prior art, a pre-set questionnaire is usually used by artificial to ask the target object, and then the artificial combines the tone, speed, facial expression of the target object when answering the question and the content of the answer of the target object, scores the answer of the target object according to personal experience, and then judges whether the target object has somatization disorder according to the score.

[0063] However, the way of identifying whether the target object has somatization disorder according to experience has the problem of low efficiency. At the same time, if the database is generated, the process of labeling the sample is difficult, so there is currently no research on somatization speech recognition based on deep learning. Therefore, how to automatically identify somatization disorder has important research significance.

[0064] Currently, there is a lack of research on speech recognition of somatization disorder at home and abroad, mainly because there is a lack of large-scale regular collected speech database. The reason for the lack of regular collected speech database is that the labeling of samples in such a database often depends on professional doctors, and the entire database is very expensive. Therefore, how to use acoustic signal analysis means to process speech samples under the condition of scarcity of sample labels (i.e. unsupervised condition) to learn relevant supervised features is gradually becoming a research hotspot in the field of digital medicine to explore new digital phenotypes.

[0065] Therefore, an embodiment of the present application provides a somatization disorder identification method based on voice self-supervised learning. After obtaining voice information to be verified of a target object, the voice information to be verified is subjected to frame division and noise reduction processing to obtain a plurality of frames of first voice information in which noise is removed. Then, the first voice information of each frame is input into a target somatization disorder identification model to obtain a first feature fusion vector of the voice information to be verified. Further, based on the first feature fusion vector, a predicted identifier corresponding to the voice information to be verified is obtained, and the predicted identifier indicates whether the target object has somatization disorder. Since the target somatization disorder model is a model trained based on a self-supervised contrastive coding model, it does not need to collect and process information manually, can perform self-learning, and can learn and mine the supervised features of the voice sample under the condition that the voice sample label is scarce. Then, the voice information can be predicted according to the extracted first feature fusion vector to obtain a predicted identifier of the voice information, so as to analyze the health status of the sample object based on the voice information, that is, to automatically, quickly and accurately assign a health status identifier to the corresponding target object based on the voice information, and solve the problem of low recognition efficiency in the prior art.

[0066] The somatization disorder identification method based on voice self-supervised learning of the embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0067] The somatization disorder identification method based on voice self-supervised learning provided by the present application can be applied to an electronic device as shown in Figure 1 The electronic device can be a terminal, and its internal structure diagram can be as shown in Figure 1As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used for wired or wireless communication with an external terminal. Wireless communication can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a somatization disorder identification method based on voice self-supervised learning. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. It can also be an external keyboard, touchpad or mouse, etc. Among them, the terminal can be but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices.

[0068] In another application scenario, the electronic device can also be a server, and its internal structure diagram can be as shown in the figure. Figure 2 The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store sample voice data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a somatization disorder identification method based on voice self-supervised learning. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0069] Those skilled in the art can understand that Figure 1 and Figure 2 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0070] Figure 3 is a flowchart of a somatization disorder identification method based on voice self-supervised learning according to an example embodiment of the present application. Referring to Figure 3As shown, the method comprises:

[0071] S502, obtaining voice information to be verified of a target object.

[0072] Specifically, the voice information to be verified of the target object can be collected by a sound collection device and sent to the terminal. Alternatively, the voice information to be verified of the target object can be collected by the terminal, which is not limited herein.

[0073] The voice information to be verified is original audio data.

[0074] S504, performing frame division and noise reduction processing on the voice information to be verified to obtain a plurality of frames of first voice information with noise removed.

[0075] Specifically, after obtaining the voice information to be verified, short-time frame division processing is performed thereon, and a filter is used for filtering to obtain a plurality of frames of first voice information with noise removed.

[0076] The first voice information refers to a plurality of frames of voice information obtained by performing short-time frame division and noise reduction on original audio data.

[0077] S506, inputting the first voice information of each frame into a target somatization disorder recognition model to obtain a first feature fusion vector of the voice information to be verified. The target somatization disorder model is a model trained based on a self-supervised contrastive coding model. The first feature fusion vector is a feature vector fusing the first voice information of each frame.

[0078] The first feature fusion vector is a feature fusion vector output in the training process of the target somatization disorder model, so that the subsequent model uses the first feature fusion vector to extract corresponding feature information.

[0079] Specifically, the first voice information of each frame is input into the trained target somatization disorder recognition model, and the first feature fusion vector of the voice information to be verified is output by the target somatization disorder recognition model.

[0080] As an alternative embodiment, Figure 4 is a flowchart of a training process of a target somatization disorder recognition model according to an example embodiment of the present application. Referring to Figure 4 As shown, the training method of the target somatization disorder recognition model comprises:

[0081] S202, obtaining a plurality of training sample data and a plurality of test sample data; each training sample data and each test sample data comprises second voice information and a health state identifier corresponding to the second voice information, and the health state identifier is used to indicate whether the sample object corresponding to the second voice information has a somatization disorder.

[0082] It should be noted that the sample objects corresponding to each training sample data and each test sample data are different. The second voice information and the corresponding health state identifier are included in each training sample data and each test sample data. The health state identifier can be sick or normal.

[0083] The second voice information is the initial voice audio used during training.

[0084] Specifically, when the model training instruction is received, a plurality of training sample data and a plurality of test sample data can be obtained from the database. Alternatively, the plurality of training sample data and the plurality of test sample data sent by the upper computer can also be received, which is not limited herein.

[0085] S204, the second voice information in each sample training data is respectively subjected to frame processing to obtain a plurality of third voice information corresponding to the second voice information.

[0086] The third voice information is voice information obtained by short-time frame processing and noise reduction on the initial voice audio.

[0087] Specifically, the second voice information in each sample training data is subjected to short-time frame processing, i.e., the second voice information is divided into a plurality of shorter voice information. Optionally, the length of each frame can be 10 ms. In this case, the plurality of shorter voice information obtained by short-time frame processing on the second voice information is the third voice information.

[0088] Optionally, a filter can also be used to filter each frame of voice information obtained by short-time frame processing to obtain voice information with noise removed from the surrounding environment and the collection device. For example, a filter of 300-3000 Hz can be used for filtering. In this case, after the second voice information is subjected to short-time frame processing and filtering, the plurality of shorter voice information with noise removed obtained is the third voice information.

[0089] S206, each frame of third voice information is input into the initial somatization disorder recognition model for feature fusion to determine a first feature fusion vector of the third voice information. The first feature fusion vector of the third voice information is a vector for distinguishing the significant features in each third voice information.

[0090] The initial somatization disorder recognition model can include a self-supervised contrastive coding model. The first feature fusion vector of the third voice information is a feature that can directly distinguish whether the sample object is sick or normal.

[0091] Specifically, the third voice information of each frame is input into the initial somatization disorder identification model, the initial somatization disorder identification model performs feature fusion according to the third voice information, and outputs a first feature fusion vector of learning context information.

[0092] In an alternative embodiment, step S206 specifically comprises:

[0093] S2062, inputting the third voice information of each frame into the initial somatization disorder identification model for analysis, and obtaining predicted voice information, the predicted voice information being voice information of the (t+k) th frame predicted based on the third voice information of the t th frame, both t and k being greater than or equal to 1 and being positive integers.

[0094] Specifically, the third voice information of each frame is input into the initial somatization disorder identification model for analysis, and a feature vector of the third voice information of each frame is extracted, and then based on the feature vectors of the third voice information, an autoregressive model is used to predict voice information k frames after each third voice information.

[0095] Optionally, the specific implementation of step S2062 comprises:

[0096] S2062a, inputting the third voice information of each frame into a nonlinear encoder in the initial somatization disorder identification model to obtain a feature vector corresponding to the third voice information of each frame.

[0097] Specifically, the third voice information X of each frame is input into a nonlinear encoder g enc in the initial somatization disorder identification model, and the third voice signal x t of the t th frame is mapped to a feature vector z t , that is: z t =g enc (x t ), to obtain a feature vector corresponding to the third voice information of each frame.

[0098] The nonlinear encoder g enc may adopt the architecture of a convolutional neural network (CNN). Its parameters can be set as follows: the number of neurons is 512, the size of the convolution kernel is [10, 8, 4, 4, 2], the stride is [5, 4, 2, 2, 2], the activation function adopts a ReLU function, batch normalization is used to speed up the training process, and the batch size is set to 128.

[0099] S2062b, inputting the feature vectors into an autoregressive sub-model in the initial somatization disorder identification model to obtain a second feature fusion vector.

[0100] The second feature fusion vector is a vector obtained by performing context semantic feature fusion on the third speech information corresponding to each frame before the t+1th frame. It should be noted that the second feature fusion vector is a feature fusion vector extracted in the model training process.

[0101] Specifically, each feature vector z t is input into an autoregressive sub-model g ar of the initial somatization disorder recognition model to obtain a second feature fusion vector c t , i.e., c t = g ar (z ≤t ), where c t represents the second feature fusion vector of the third speech information of the tth frame after context feature fusion, and z ≤t represents all feature vectors before the third speech information of the tth frame (including the tth frame).

[0102] S2062c, multiplying the second feature fusion vector and the preset linear matrix to obtain predicted speech information.

[0103] Specifically, the predicted feature value, i.e., the predicted speech information, can be represented by multiplying the second feature fusion vector c k by a linear matrix W t , i.e., W k c t .

[0104] The steps S2062a-S2062c described above can be referred to as shown in FIG. 6. Figure 5 Based on the features of the existing speech information of each frame, the features of the future time can be predicted, and the training of the somatization disorder recognition model can be realized.

[0105] S2064, based on the third speech information of the t+kth frame and the predicted speech information, adjusting the parameters of the initial somatization disorder recognition model until a preset convergence condition is reached, and outputting the first feature fusion vector.

[0106] Specifically, after obtaining the third speech information of the t+kth frame and the predicted speech information, whether the initial somatization disorder recognition model converges can be determined by determining the similarity between the third speech information of the t+kth frame and the predicted speech information. If the convergence is not reached, the parameters of the initial somatization disorder recognition model are further adjusted until the initial somatization disorder recognition model converges, and in the case of convergence, the first feature fusion vector is output.

[0107] Optionally, the specific implementation of step S2064 includes:

[0108] S2064a, determining the mutual information between the second feature fusion vector and the predicted speech information.

[0109] wherein the formula for calculating mutual information is a conventional existing mutual information calculation formula, which is not described herein.

[0110] The third speech information x of the t+kth frame predicted has a reduced entropy value due to the introduction of the second feature fusion vector c of the current frame, and the purpose of training the model is to maximize the amount of entropy reduction, i.e., the amount of uncertainty reduction of x.

[0111] S2064b, when the mutual information does not reach the maximum value, adjusting the parameters in the autoregressive sub-model, and inputting each feature vector into the adjusted autoregressive sub-model to determine a new second feature fusion vector.

[0112] Specifically, the mutual information I can be maximized by iteratively optimizing the model, and when the mutual information does not reach the maximum value, the parameters in the autoregressive sub-model are adjusted, and each feature vector is input into the adjusted autoregressive sub-model to determine a new second feature fusion vector.

[0113] S2064c, determining a new predicted speech information based on the new second feature fusion vector and a preset linear matrix.

[0114] Specifically, when the new second feature fusion vector and the preset linear matrix are multiplied, the new predicted speech information is obtained.

[0115] S2064d, determining a new mutual information according to the new predicted speech information and the new second feature fusion vector, and calculating the similarity between the third speech information corresponding to the t+kth frame and the new predicted speech information when the mutual information reaches the maximum value.

[0116] Specifically, the new predicted speech information and the new second feature fusion vector are substituted into the determination formula of the mutual information to determine the new mutual information, and when the mutual information reaches the maximum value, the similarity between the third speech information corresponding to the t+kth frame and the new predicted speech information can be calculated by the density ratio formula: t+k wherein z k is the third speech information corresponding to the t+kth frame, W k is a linear matrix, f t+k (x t , c t ) is a density ratio function for representing the similarity between the prediction according to the context information c t+k and the future true value x

[0117] S2064e, determining a loss value based on the similarity and a preset loss function, and outputting the first fusion feature vector when the loss value is less than or equal to a preset loss threshold. ​

[0118] wherein the preset loss function can be a noise contrast estimation function, denoted as InfoNCE loss, and the formula is as follows: wherein x t+k is the predicted speech information, c t is the second feature fusion vector of the speech context feature fusion of the current t-th frame, f k (x t+k ,c t ) represents the predicted similarity of x t+k and c t , (x t+k ,c t ) represents a positive sample pair, and (x j ,c t ) represents a negative sample pair. The purpose of training optimization is to make the distance between the positive sample pairs small enough and the distance between the negative sample pairs large enough.

[0119] Specifically, the similarity is substituted into the preset loss function to determine the loss value, and when the loss value is less than or equal to a preset loss threshold, the first feature fusion vector is output.

[0120] In the training process of the model, a model training optimizer can be used to optimize the training process. The Adam optimizer can be selected, the learning rate is set to 0.002, the parameter beta1 is 0.9, and the parameter beta2 is 0.999. In order to ensure the continuity of the speech, the value of the step k is not easily set too large. For example, when the step k is set to 12, the total loss is

[0121] In the embodiments of the present application, the initial somatization disorder identification model is continuously iteratively optimized to obtain a trained model that meets the convergence condition, and the most significant first feature fusion vector is output to provide a basis for subsequent identification of the health status identifier.

[0122] In the embodiments of the present application, the third speech information of each frame is input into the initial somatization disorder identification model for analysis to obtain the predicted speech information. Then, based on the third speech information of the t+k-th frame and the predicted speech information, the parameters of the initial somatization disorder identification model are adjusted until the preset convergence condition is reached, and the first feature fusion vector is output. The initial somatization disorder identification model can be continuously iteratively optimized to obtain a trained model that meets the convergence condition, and the most significant first feature fusion vector is output to provide a basis for subsequent identification of the health status identifier.

[0123] S208, based on the first feature fusion vector, obtaining the predicted identifier corresponding to the second speech information of each test sample data, the predicted identifier being used to represent the prediction information of whether the sample object corresponding to the test sample data has somatization disorder.

[0124] Specifically, the first feature fusion vector is used to extract a test feature vector of the second voice information of each test sample data, and a corresponding prediction identifier is determined based on the test feature vector. That is, the prediction information of whether the sample object corresponding to each test sample data has somatization disorder is determined.

[0125] In an alternative embodiment, step S208 specifically comprises:

[0126] S2082, based on the first feature fusion vector, extracting target feature data of the second voice information in each test sample data.

[0127] S2084, based on each target feature data and a preset parameter threshold, obtaining a prediction identifier corresponding to the second voice information of each test sample data.

[0128] Specifically, according to the first feature fusion vector, the target feature data of the second voice information in each test sample data is extracted. Each target feature data can be a related parameter of the voiceprint information in each voice information. After obtaining each target feature data, each target feature data and a preset parameter threshold are compared to determine the prediction identifier corresponding to the second voice information of each test sample data. If each target feature data is less than the preset parameter threshold, the prediction identifier corresponding to the second voice information is determined to be healthy. For example, the preset parameter threshold is 3, the target feature data obtained is 2, and the prediction identifier is healthy. If the target feature data obtained is 4, the prediction identifier is ill.

[0129] S210, based on each prediction identifier, the health status identifier in each test sample data, and the number of categories of health status identifiers, determining the accuracy of the initial somatization disorder recognition model.

[0130] Specifically, after obtaining each prediction identifier, the health status identifier in each test sample data, and the number of categories of health status identifiers, the non-weighted average recall rate of the initial somatization disorder recognition model can be calculated as the accuracy of the model.

[0131] In an alternative embodiment, step S210 specifically comprises:

[0132] S2102, comparing the health status identifier in each test sample data with the prediction identifier corresponding to each test sample data to determine the recall rate of the somatization disorder prediction result of the test sample data.

[0133] S2104, based on the recall rate and the number of categories of health status identifiers, determining the non-weighted average recall rate as the accuracy of the initial somatization disorder recognition model.

[0134] The formula of the unweighted average recall (UAR) is as follows: N represents the number of types of health status identifiers (which can be 2 in this patent, i.e., healthy and ill), and Recall represents the recall rate of the identifier. class

[0135] Specifically, the health status identifiers in the test sample data can be compared with the prediction identifiers corresponding to the test sample data to determine the recall rate of the somatization disorder prediction result of the test sample data.

[0136] The recall rate and the number of types of health status identifiers are substituted into the UAR formula to obtain the accuracy of the initial somatization disorder identification model.

[0137] In S212, the parameters of the initial somatization disorder identification model are adjusted based on the accuracy and a preset accuracy threshold until the accuracy is higher than the preset accuracy threshold, and a target somatization disorder identification model is obtained.

[0138] The preset accuracy threshold can be as follows: The parameters of the initial somatization disorder identification model can include a learning rate, a model replacement seed, a step size, a size, etc., without limitation.

[0139] Specifically, when the accuracy is higher than the preset accuracy threshold, the target somatization disorder identification model is trained. If the accuracy is not higher than the preset accuracy threshold, the parameters of the initial somatization disorder identification model are adjusted until the target somatization disorder identification model is obtained.

[0140] ​In the embodiment of the present application, after obtaining a plurality of training sample data and a plurality of test sample data, the second voice information in each sample training data is subjected to frame processing respectively to obtain a plurality of frames of third voice information corresponding to the second voice information. Then each frame of third voice information is input into an initial somatization disorder recognition model for feature fusion to determine a first feature fusion vector. Further, based on the first feature fusion vector, a prediction label corresponding to the second voice information of each test sample data is obtained. Further, based on the prediction label, the health status label in each test sample data, and the number of categories of health status labels, the accuracy of the initial somatization disorder recognition model is determined, and based on the accuracy and the preset accuracy threshold, the parameters of the initial somatization disorder recognition model are adjusted until the accuracy is higher than the preset accuracy threshold, and a target somatization disorder recognition model is obtained. The useful context representation information can be extracted from the voice sample, and the extracted representation information and the representation information of the future time sample are compared and learned to obtain the key representation information that can best predict the future, that is, the vector of the most significant feature, that is, to learn and mine the supervised features of the voice sample under the condition of the scarcity of voice sample labels. Further, the first feature fusion vector extracted can be used to predict the voice information to obtain the prediction label of the voice information, so as to analyze the health status of the sample object based on the voice information, that is, to automatically, quickly and accurately assign the health status label to the corresponding target object based on the voice information, and to solve the problem of low recognition efficiency in the prior art.

[0141] S508, based on the first feature fusion vector, obtaining a prediction label corresponding to the voice information to be verified, the prediction label being used to indicate whether the target object has somatization disorder.

[0142] Specifically, according to the first feature fusion vector, the target feature data of the voice information to be verified is extracted. The target feature data is compared with the preset parameter threshold to determine the prediction label corresponding to the voice information to be verified.

[0143] In the embodiment of the present application, the obtained voice information to be verified of the target object is subjected to frame and noise reduction processing to obtain a plurality of frames of first voice information with noise removed. Then each frame of first voice information is input into the target somatization disorder recognition model to obtain a first feature fusion vector of the voice information to be verified. Further, according to the first feature fusion vector, a prediction label corresponding to the voice information to be verified is obtained, which indicates whether the target object has somatization disorder. The supervised features of the voice sample can be learned and mined under the condition of the scarcity of voice sample labels, and further, the health status label of the voice information can be automatically divided, thereby solving the problem of the scarcity of voice sample labels, and filling the research gap of somatization disorder voice recognition at home and abroad.

[0144] Figure 6is a schematic diagram of a somatization disorder recognition device based on self-supervised learning according to an example embodiment of the present application. Referring to Figure 6 The device is configured to implement all or part of the functions of the method embodiments described above. Specifically, the somatization disorder recognition device based on self-supervised learning comprises:

[0145] The first acquisition module 702 is configured to acquire the to-be-verified speech information of the target object.

[0146] The first preprocessing module 704 is configured to perform frame division and noise reduction processing on the to-be-verified speech information to obtain a plurality of frames of first speech information with noise removed.

[0147] The second acquisition module 706 is configured to input each frame of the first speech information into a target somatization disorder recognition model to acquire a first feature fusion vector of the to-be-verified speech information.

[0148] The third acquisition module 708 is configured to acquire a predicted identity corresponding to the to-be-verified speech information based on the first feature fusion vector, the predicted identity being used to indicate whether the target object has a somatization disorder.

[0149] The specific limitations of the somatization disorder recognition device based on self-supervised learning can be found in the limitations of the somatization disorder recognition based on self-supervised learning described above, and will not be repeated here.

[0150] In an embodiment, the third acquisition module 708 comprises:

[0151] The target feature extraction unit is configured to extract target feature data of the to-be-verified speech information based on the first feature fusion vector.

[0152] The first determination unit is configured to determine the predicted identity corresponding to the to-be-verified speech information based on the target feature data and a preset parameter threshold.

[0153] In an embodiment, the somatization disorder recognition device based on self-supervised learning further comprises a target somatization disorder training module, which comprises:

[0154] The first acquisition unit 602 is configured to acquire a plurality of training sample data and a plurality of test sample data. Each training sample data and each test sample data comprises second speech information and a health state identity corresponding to the second speech information, the health state identity being used to indicate whether a sample object corresponding to the second speech information has a somatization disorder.

[0155] The first preprocessing unit 604 is configured to perform frame division processing on the second speech information in each sample training data to acquire a plurality of frames of third speech information corresponding to the second speech information.

[0156] The fusion unit 606 is configured to input each frame of the third voice information into the initial somatization disorder identification model for feature fusion, and determine a first feature fusion vector of the third voice information, where the first feature fusion vector of the third voice information is a vector for distinguishing the significant features in each of the third voice information.

[0157] The second acquisition unit 608 is configured to acquire, based on the first feature fusion vector, a predicted identifier corresponding to the second voice information of each test sample data, where the predicted identifier is used to represent the prediction information of whether the sample object corresponding to the test sample data has the somatization disorder.

[0158] The second determination unit 610 is configured to determine, based on each predicted identifier, the health state identifier in each test sample data, and the number of categories of the health state identifier, the accuracy of the initial somatization disorder identification model.

[0159] The adjustment unit 612 is configured to adjust, based on the accuracy and a preset accuracy threshold, the parameters of the initial somatization disorder identification model until the accuracy is higher than the preset accuracy threshold, to obtain a target somatization disorder identification model.

[0160] In an embodiment, the fusion unit 606 includes:

[0161] The analysis subunit is configured to input each frame of the third voice information into the initial somatization disorder identification model for analysis, and acquire predicted voice information, where the predicted voice information is the voice information of the (t+k)th frame that is predicted based on the third voice information of the tth frame, and t and k are both greater than or equal to 1 and are positive integers.

[0162] The adjustment subunit is configured to adjust, based on the third voice information of the (t+k)th frame and the predicted voice information, the parameters of the initial somatization disorder identification model until a preset convergence condition is reached, and output the first feature fusion vector.

[0163] In an embodiment, the analysis subunit is specifically configured to input each frame of the third voice information into a nonlinear encoder in the initial somatization disorder identification model, to obtain a feature vector corresponding to each frame of the third voice information; input each feature vector into an autoregressive submodel in the initial somatization disorder identification model, to obtain a second feature fusion vector, where the second feature fusion vector is a vector obtained by performing context semantic feature fusion on the third voice information corresponding to each frame before the (t+1)th frame; and multiply the second feature fusion vector and a preset linear matrix, to obtain the predicted voice information.

[0164] In an embodiment, the analyzing subunit is specifically configured to determine mutual information between the second feature fusion vector and the predicted speech information; when the mutual information does not reach a maximum value, adjust parameters in the autoregressive submodel, input each feature vector into the adjusted autoregressive submodel, and determine a new second feature fusion vector; determine a new predicted speech information based on the new second feature fusion vector and a preset linear matrix; determine a new mutual information according to the new predicted speech information and the new second feature fusion vector, until the mutual information reaches the maximum value, calculate a similarity between the third speech information corresponding to the t+kth frame and the new predicted speech information; determine a loss value based on the similarity and a preset loss function, and when the loss value is less than or equal to a preset loss threshold, output the first fusion feature vector.

[0165] In an embodiment, the second obtaining unit 608 comprises:

[0166] The extracting subunit is configured to extract target feature data of the second speech information in each test sample data based on the first feature fusion vector;

[0167] The obtaining subunit is configured to obtain a predicted identifier corresponding to the second speech information of each test sample data based on each target feature data and a preset parameter threshold.

[0168] In an embodiment, the second determining unit 610 comprises:

[0169] The first determining subunit is configured to compare the health status identifier in each test sample data with the predicted identifier corresponding to each test sample data, and determine a recall rate of the somatization disorder prediction result of the test sample data.

[0170] The second determining subunit is configured to determine a non-weighted average recall rate as an accuracy of the initial somatization disorder identification model based on the recall rate and the number of categories of the health status identifier.

[0171] For specific limitations of the somatization disorder identification device based on self-supervised learning, refer to the limitations of the somatization disorder identification method based on self-supervised learning in the above, which will not be repeated here. Each module in the somatization disorder identification device based on self-supervised learning described above can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0172] In an embodiment, an electronic device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0173] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0174] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0175] It is understood by a person skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing relevant hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0176] It is easy to understand that, on the basis of the several embodiments provided by the present application, a person skilled in the art can combine, split, recombine and obtain other embodiments of the present application, and these embodiments do not exceed the protection scope of the present application.

[0177] The above specific embodiments have further detailed the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above is only a specific embodiment of the present application and does not limit the protection scope of the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A somatization disorder identification method based on voice self-supervised learning, characterized in that, The method comprises the following steps: obtaining voice information of a target object to be verified; frame dividing and noise reduction processing are performed on the voice information to be verified to obtain a plurality of frames of first voice information from which noise is removed; inputting the first voice information of each frame into a target somatization disorder recognition model to obtain a first feature fusion vector of the voice information to be verified; the target somatization disorder recognition model is a model trained based on a self-supervised contrastive coding model; the first feature fusion vector is a feature vector that fuses the first voice information of each frame; wherein the training method of the target somatization disorder recognition model comprises: obtaining a plurality of training sample data and a plurality of test sample data; each of the training sample data and each of the test sample data comprises second voice information and a health status identifier corresponding to the second voice information, the health status identifier being used to indicate whether the sample object corresponding to the second voice information has somatization disorder; frame dividing is performed on the second voice information in each of the training sample data to obtain a plurality of frames of third voice information corresponding to the second voice information; inputting the third voice information of each frame into an initial somatization disorder recognition model to perform feature fusion to determine a first feature fusion vector of the third voice information, the first feature fusion vector being a vector used to distinguish the significant features in each of the third voice information, the initial somatization disorder recognition model comprising a self-supervised contrastive coding model; based on the first feature fusion vector, a predicted identifier corresponding to the second voice information of each of the test sample data is obtained; the predicted identifier is used to indicate the prediction information of whether the sample object corresponding to the test sample data has somatization disorder; based on each of the predicted identifiers, the health status identifiers in each of the test sample data, and the number of categories of the health status identifiers, the accuracy of the initial somatization disorder recognition model is determined; based on the accuracy and a preset accuracy threshold, the parameters of the initial somatization disorder recognition model are adjusted until the accuracy is higher than the preset accuracy threshold, to obtain a target somatization disorder recognition model; based on the first feature fusion vector, target feature data of the voice information to be verified is extracted; based on the target feature data and a preset parameter threshold, a predicted identifier corresponding to the voice information to be verified is determined, the predicted identifier being used to indicate whether the target object has somatization disorder.

2. The method of claim 1, wherein, The method comprises the following steps: inputting the third voice information of each frame into the initial somatization disorder recognition model to perform analysis and obtain predicted voice information, the predicted voice information being voice information of the (t+k)th frame predicted based on the third voice information of the tth frame, t and k both being greater than or equal to 1 and being positive integers; The parameters of the initial body alienation disorder recognition model are adjusted based on the third voice information of the t+kth frame and the predicted voice information until a preset convergence condition is reached, and the first feature fusion vector of the third voice information is output.

3. The method of claim 2, wherein, The third voice information of each frame is input into an initial body alienation disorder recognition model for analysis to obtain predicted voice information, including: The third voice information of each frame is input into a nonlinear encoder in the initial body alienation disorder recognition model to obtain a feature vector corresponding to the third voice information of each frame; Each feature vector is input into an autoregressive sub-model in the initial body alienation disorder recognition model to obtain a second feature fusion vector; the second feature fusion vector is a vector obtained by context semantic feature fusion based on the third voice information of each frame before the t+1th frame; The second feature fusion vector and a preset linear matrix are multiplied to obtain the predicted voice information.

4. The method of claim 3, wherein, The parameters of the initial body alienation disorder recognition model are adjusted based on the third voice information of the t+kth frame and the predicted voice information until a preset convergence condition is reached, and the first feature fusion vector of the third voice information is output, including: The mutual information between the second feature fusion vector and the predicted voice information is determined; When the mutual information does not reach a maximum value, the parameters in the autoregressive sub-model are adjusted, and each feature vector is input into the adjusted autoregressive sub-model to determine a new second feature fusion vector; Based on the new second feature fusion vector and the preset linear matrix, a new predicted voice information is determined; According to the new predicted voice information and the new second feature fusion vector, a new mutual information is determined until the mutual information reaches a maximum value, and a similarity between the third voice information of the t+kth frame and the new predicted voice information is calculated; Based on the similarity and a preset loss function, a loss value is determined, and when the loss value is less than or equal to a preset loss threshold, the first fusion feature vector is output.

5. The method of claim 1, wherein, Based on the first feature fusion vector, the predicted labels corresponding to the second voice information of each test sample data are obtained, including: Based on the first feature fusion vector, target feature data of the second voice information in each test sample data is extracted; Based on each target feature data and a preset parameter threshold, the predicted labels corresponding to the second voice information of each test sample data are obtained.

6. The method of claim 1, wherein, Based on each predicted label, the health status labels in each test sample data, and the number of categories of the health status labels, the accuracy of the initial body alienation disorder recognition model is determined, including: The health status labels in each test sample data and the predicted labels corresponding to each test sample data are compared to determine the recall rate of the body alienation disorder prediction result of the test sample data; Based on the recall rate and the number of categories of the health status labels, a non-weighted average recall rate is determined, which is the accuracy of the initial body alienation disorder recognition model.

7. A somatization disorder recognition device based on voice self-supervised learning, characterized by, The device includes: The first obtaining module is configured to obtain voice information to be verified of a target object; The first preprocessing module is configured to perform frame division and noise reduction processing on the voice information to be verified to obtain a plurality of frames of first voice information from which noise is removed; The second obtaining module is configured to input each frame of the first voice information into a target somatization disorder recognition model to obtain a first feature fusion vector of the voice information to be verified, wherein the target somatization disorder recognition model is a model trained based on a self-supervised contrastive coding model, and the first feature fusion vector is a feature vector that fuses the first voice information of each frame. The third obtaining module includes a target feature extraction unit and a first determination unit, wherein the target feature extraction unit is configured to extract target feature data of the voice information to be verified based on the first feature fusion vector, and the first determination unit is configured to determine a predicted identifier corresponding to the voice information to be verified based on the target feature data and a preset parameter threshold, wherein the predicted identifier is used to indicate whether the target object has somatization disorder. 8.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images