A multidimensional data screening method and system for predicting mental illness

Abstract

The present invention proposes a multidimensional data screening method and system for predicting mental illness. The multidimensional data screening method for predicting mental illness includes: extracting text data information, sound data information and video data information during the patient consultation process from the medical system; performing data screening on the text data information, sound data information and video data information respectively, obtaining key data information corresponding to the text data information, sound data information and video data information and their corresponding information evaluation parameters; using the information evaluation parameters corresponding to the text data information, sound data information and video data information to obtain the patient's comprehensive evaluation index parameters. The system includes modules corresponding to the steps of the method.

Description

Technical Field

[0001] The present invention proposes a multidimensional data screening method and system for predicting mental illness, belonging to the technical field of medical data information screening. Background Art

[0002] With the continuous development of medical technology, the treatment process of mental illness has become more and more complex and sophisticated. In the process, a large amount of data information is generated, including the patient's physiological data, psychological assessment results, drug use, treatment effect feedback, etc. This information is crucial for doctors to formulate and adjust treatment plans, but it also brings problems such as information overload and screening difficulties. Traditional data information screening methods mainly rely on the doctor's professional knowledge and experience, but this method has problems such as strong subjectivity and low efficiency.

[0003] In addition, with the development of big data and artificial intelligence technology, although it provides new means for data information screening, its application in the field of mental illness treatment is still relatively small, because patients need to observe and collect many types of data during the treatment process, including sound, movement, expression, etc., which makes the data relatively complex. When the data volume is large and complex, each type of data needs to be sent to the doctor for display separately, and the doctor needs to check them one by one, which will waste manpower and time, and also requires doctors to manually screen out valid data from a large amount of data. The degree of intelligence and efficiency are low. Therefore, the current data information screening technology still faces many challenges when processing complex and multi-source data information in the field of mental illness prediction. Summary of the invention

[0004] The present invention provides a multidimensional data screening method and system for predicting mental illness, which is used to solve the problems of low data screening efficiency and integration ability in the prior art. The technical solutions adopted are as follows:

[0005] A multidimensional data screening method for predicting mental illness, the multidimensional data screening method for predicting mental illness comprising:

[0006] Extract text data information, sound data information and video data information during the patient consultation process from the medical system;

[0007] Performing data screening on the text data information, the sound data information and the video data information respectively, and obtaining key data information corresponding to the text data information, the sound data information and the video data information and corresponding information evaluation parameters;

[0008] The comprehensive evaluation index parameters of the patient are obtained by using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information.

[0009] Furthermore, the text data information, the sound data information and the video data information are respectively screened to obtain key data information corresponding to the text data information, the sound data information and the video data information, including:

[0010] Preprocessing the text data information, screening the preprocessed text data information, and obtaining key text information in the text data information corresponding to the patient;

[0011] Preprocessing the sound data information, screening the preprocessed sound data information, and obtaining key sound information in the sound data information corresponding to the patient object;

[0012] The video data information is preprocessed, and the preprocessed video data information is screened to obtain key image information in the video data information corresponding to the patient.

[0013] Further, the text data information is preprocessed, and the preprocessed text data information is screened to obtain key text information in the text data information corresponding to the patient, including:

[0014] Performing text-clearing processing on the text data information, removing irrelevant characters, irrelevant punctuation marks and stop words in the text data information, and obtaining text data information after text-clearing processing corresponding to the text data information;

[0015] Performing word segmentation processing on the text data information after the text legibility processing to obtain the words, phrases and sentences corresponding to the text data information after the text legibility processing;

[0016] Performing part-of-speech tagging on the words, phrases and sentences corresponding to the text data information, and converting the words, phrases and sentences corresponding to the text data information into word embedding vectors;

[0017] Using a convolutional neural network model to extract text features from a word embedding vector corresponding to the text data information, and obtaining text feature information contained in the text data information, wherein the text feature information includes words, phrases, and sentence structures used to characterize characteristics of mental illness; and the words, phrases, and sentence structures used to characterize characteristics of mental illness are key text information in the text data information corresponding to the patient;

[0018] The text information evaluation parameters of the patient are obtained by using the key text information in the text data information corresponding to the patient.

[0019] Further, using key text information in the text data information corresponding to the patient to obtain the text information evaluation parameters of the patient includes:

[0020] Performing feature classification on the text feature information to obtain a vocabulary information set, a phrase information set, and a sentence structure information set;

[0021] The lexical feature information contained in the lexical information set is used to obtain a first text information evaluation parameter value corresponding to the lexical feature information; wherein the first text information evaluation parameter value is obtained by the following formula:

[0022] W 01 =ln(1+w x01 -w x02 )

[0023]

[0024]

[0025] Among them, W 01 represents the first text information evaluation parameter value; x01 represents the first parameter coefficient; w x02 represents the second parameter coefficient; n represents the number of words contained in the vocabulary information set; G represents the preset reference word distance; L i Indicates the starting position of the i-th word; L i+1 Indicates the starting position of the i+1th word; Z i represents the number of words contained in the i-th word; D represents the number of words in the sentence where the i-th word is located;

[0026] The phrase feature information included in the phrase information set is used to obtain a second text information evaluation parameter value corresponding to the phrase feature information; wherein the second text information evaluation parameter value is obtained by the following formula:

[0027] W 02 =ln(1+w x03 -w x04 )

[0028]

[0029]

[0030] Among them, W 02 represents the second text information evaluation parameter value; x03 represents the third parameter coefficient; w x04 represents the fourth parameter coefficient; m represents the number of phrases contained in the phrase information set; Z di Indicates the number of words contained in the i-th phrase; Z ci Indicates the number of words contained in the i-th phrase; D dIndicates the number of words in the sentence where the i-th phrase is located; J i Indicates the starting position of the i-th phrase; J i+1 Indicates the starting position of the i+1th phrase;

[0031] The structural feature information contained in the sentence structure information set is used to obtain a third text information evaluation parameter value corresponding to the structural feature information; wherein the third text information evaluation parameter value is obtained by the following formula:

[0032]

[0033] Among them, W 03 Indicates the third text information evaluation parameter value; N y N represents the number of sentences in the sentence structure information set that meet the sentence structure characteristics of mental illness; z represents the total number of sentences contained in the text data information; P represents the total proportion of words representing the characteristics of mental illness in the case where the sentences satisfy the sentence structure representing the characteristics of mental illness;

[0034] The patient's text information evaluation parameter is obtained by using the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value; wherein the text information evaluation parameter is obtained by the following formula:

[0035] W=λ 01 ·W 01 +λ 02 ·W 02 +λ 03 ·W 03

[0036] Where W represents the text information evaluation parameter; λ 01 , 02 and λ 03 They respectively represent the weight values ​​corresponding to the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value.

[0037] Further, the sound data information is preprocessed, and the preprocessed sound data information is screened to obtain key sound information in the sound data information corresponding to the patient object, including:

[0038] Performing sound signal cleaning processing on the sound data information to eliminate background noise and electromagnetic interference signals in the sound data information, and obtaining the sound data information after the sound signal cleaning processing;

[0039] Segmenting the sound data information after the sound signal cleaning process according to preset time intervals to obtain segmented sound data information;

[0040] Performing standardization processing on the segmented sound data information to obtain the standardized sound data information;

[0041] Inputting the standardized sound data information into a convolutional neural network model to obtain sound data features corresponding to the sound data information, wherein the sound data features include abnormal utterance features, intonation change features, and speech speed change features;

[0042] The sound information evaluation parameter of the patient is obtained by using the sound data feature corresponding to the sound data information.

[0043] Further, obtaining the patient's voice information evaluation parameter using the voice data feature corresponding to the voice data information includes:

[0044] Retrieving the sound data features corresponding to the sound data information of the patient;

[0045] The first sound information evaluation parameter value is obtained by using the abnormal sound feature in the sound data feature; wherein the first sound information evaluation parameter value is obtained by the following formula:

[0046]

[0047] Among them, Y 01 represents the first sound information evaluation parameter value; k 01 Indicates the number of abnormal sounds in the sound data feature; T 01i represents the duration of the ith abnormal sound; T represents the total duration of the sound data; P 11i represents the proportion of time that the vocabulary representing the characteristics of mental illness contains in the duration of the ith abnormal voice; P 12i represents the proportion of time that phrases representing characteristics of mental illness are contained in the duration of the ith abnormal vocalization; P 13i represents the proportion of the time that the sentence structure representing the characteristics of mental illness is contained in the duration of the i-th abnormal voice;

[0048] The second sound information evaluation parameter value is obtained by using the intonation change feature in the sound data feature; wherein the second sound information evaluation parameter value is obtained by the following formula:

[0049]

[0050] Among them, Y 02 represents the second sound information evaluation parameter value; k 02 Indicates the number of intonation changes in the sound data feature; T 02i represents the duration of the i-th intonation change; T xirepresents the duration of abnormal utterance contained in the duration of the ith intonation change; P 21i represents the proportion of time that the words representing the characteristics of mental illness are contained in the duration of the ith intonation change, excluding the abnormal vocalization content; P 22i represents the proportion of time that phrases representing the characteristics of mental illness occupy in the duration of the i-th intonation change, excluding the abnormal vocalization content; P 23i represents the proportion of the time that the sentence structure representing the characteristics of mental illness excluding the abnormal vocalization content in the duration of the i-th intonation change;

[0051] The third sound information evaluation parameter value is obtained by using the speech rate change feature in the sound data feature; wherein the third sound information evaluation parameter value is obtained by the following formula:

[0052]

[0053] Among them, Y 03 represents the third sound information evaluation parameter value; k 03 Indicates the number of times the speech rate changes in the sound data feature; T 03i represents the duration of the i-th speech rate change; P xi represents the total duration ratio of abnormal utterance and intonation change included in the duration of the ith speech rate change;

[0054] The first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value are used to obtain the patient's sound information evaluation parameter; wherein the sound information evaluation parameter is obtained by the following formula:

[0055] Y=α 03 ·Y 03 -α 01 ·Y 01 +α 02 ·Y 02

[0056] Where Y represents the sound information evaluation parameter; α 01 , α 02 and α 03 Respectively represent the weight values ​​corresponding to the first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value.

[0057] Further, the video data information is preprocessed, and the preprocessed video data information is screened to obtain key image information in the video data information corresponding to the patient, including:

[0058] Performing frame processing on the video data information to obtain a static image of each frame corresponding to the video data information;

[0059] Performing image quality optimization processing on the static image of each frame to obtain the static image after optimization processing; wherein the image quality optimization processing includes image denoising processing and contrast enhancement processing;

[0060] Inputting the optimized static image into a convolutional neural network model to obtain video data features corresponding to the video data information, wherein the video data features include abnormal facial expression features and abnormal body movement features;

[0061] The video information evaluation parameters of the patient are obtained using the video data features corresponding to the video data information.

[0062] Further, obtaining the video information evaluation parameter of the patient using the video data feature corresponding to the video data information includes:

[0063] Retrieving video data features corresponding to the video data information of the patient;

[0064] The first video information evaluation parameter value is obtained by using the abnormal facial expression feature in the video data feature; wherein the first video information evaluation parameter value is obtained by the following formula:

[0065]

[0066] Among them, S 01 represents the first video information evaluation parameter value; r 01 Indicates the total number of abnormal facial expressions; T y01i represents the duration of the ith abnormal facial expression; T z Indicates the total duration of the video data information; T x11i and T x12i They represent the duration of abnormal vocalization and tone change in the ith abnormal facial expression; T c01i represents the overlapping duration of abnormal vocalization and intonation change in the duration of the i-th abnormal facial expression;

[0067] The second video information evaluation parameter value is obtained by using the abnormal body movement feature in the sound data feature; wherein the second video information evaluation parameter value is obtained by the following formula:

[0068]

[0069] Among them, S 02 represents the second video information evaluation parameter value; r 02 Indicates the total number of abnormal limb movements; T y02i represents the duration of the ith abnormal limb movement; P yiIt indicates the proportion of time that abnormal facial expressions are included in the duration of the i-th abnormal limb movement;

[0070] The video information evaluation parameter of the patient is obtained by using the first video information evaluation parameter value and the second video information evaluation parameter value; wherein the video information evaluation parameter is obtained by the following formula:

[0071] S=β 01 ·S 01 +β 02 ·S 02

[0072] Among them, S represents the video information evaluation parameter; β 01 and β 02 Respectively represent the weight values ​​corresponding to the first video information evaluation parameter value and the second video information evaluation parameter value.

[0073] Further, the information evaluation parameters corresponding to the text data information, the sound data information and the video data information are used to obtain the comprehensive evaluation index parameters of the patient, including:

[0074] Extracting text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters of the patient;

[0075] The comprehensive model is used to obtain the comprehensive evaluation index parameters of the patient through text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters, wherein the comprehensive model structure is as follows:

[0076] Q=logW+logY+logS

[0077] Among them, Q represents the comprehensive evaluation index parameter of the patient; W represents the text information evaluation parameter; Y represents the sound information evaluation parameter; S represents the video information evaluation parameter.

[0078] A multidimensional data screening system for predicting mental illness, the multidimensional data screening system for predicting mental illness comprising:

[0079] A data information extraction module is used to extract text data information, sound data information and video data information during the patient consultation process from the medical system;

[0080] An evaluation parameter acquisition module, used to screen the text data information, the sound data information and the video data information respectively, and acquire key data information corresponding to the text data information, the sound data information and the video data information and their corresponding information evaluation parameters;

[0081] The comprehensive evaluation index parameter acquisition module is used to obtain the comprehensive evaluation index parameters of the patient using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information.

[0082] Beneficial effects of the present invention:

[0083] The multidimensional data screening method and system for predicting mental illness proposed by the present invention utilizes information of three data types, namely text, sound and video, at the same time. The above technical solution can more comprehensively and accurately reflect the patient's mental state. Different types of data can provide complementary information and reduce the possibility of information loss or misjudgment. The above technical solution realizes the automation of data screening and comprehensive evaluation, reduces the need for manual operation, and improves work efficiency. At the same time, automated processing can also reduce the interference of human factors and improve the objectivity of evaluation results. By combining multiple data information and calculating information evaluation parameters, the above technical solution can achieve more personalized evaluation. Different patients will show different symptoms or behavior patterns, and the above technical solution can make more accurate evaluations based on individual differences. The comprehensive evaluation index parameters can provide objective and quantitative reference information for doctors, assisting doctors in making more accurate diagnosis and treatment decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 is a flow chart of the method of the present invention;

[0085] Figure 2 The system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0086] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0087] The embodiment of the present invention proposes a multidimensional data screening method for predicting mental illness, such as Figure 1 As shown, the multidimensional data screening method applied to mental illness prediction includes:

[0088] S1. Extracting text data information, sound data information and video data information during the patient consultation process from the medical system;

[0089] S2, respectively screening the text data information, the sound data information and the video data information to obtain key data information corresponding to the text data information, the sound data information and the video data information and their corresponding information evaluation parameters;

[0090] S3. Obtain comprehensive evaluation index parameters of the patient using information evaluation parameters corresponding to the text data information, sound data information and video data information.

[0091] The working principle of the above technical solution is: to extract the text data information, sound data information and video data information generated by the patient during the consultation process from the medical system using technical means (such as data crawlers or API interfaces).

[0092] The extracted text data information, sound data information and video data information are screened respectively. The screening process involves natural language processing (NLP) technology to identify keywords or key sentences in text data, speech recognition technology to parse key information in sound data, and video analysis technology to extract key frames or behavior patterns in video data. Through the above technical solution, the key data information corresponding to each data type can be obtained, and the corresponding information evaluation parameters can be calculated or assigned.

[0093] Based on the information evaluation parameters of text data information, sound data information and video data information, the comprehensive evaluation index parameters of the patient are calculated through certain algorithms or models (such as weighted summation, machine learning model, etc.). The comprehensive evaluation index parameters are helpful for the comprehensive and objective quantitative evaluation of the patient's mental state and improve the accuracy of data evaluation and screening.

[0094] The effect of the above technical solution is: by using information of three data types, text, sound and video, the above technical solution can reflect the patient's mental state more comprehensively and accurately. Different types of data can provide complementary information and reduce the possibility of information loss or misjudgment. The above technical solution realizes the automation of data screening and comprehensive evaluation, reduces the need for manual operation, and improves work efficiency. At the same time, automated processing can also reduce the interference of human factors and improve the objectivity of evaluation results. By combining multiple data information and calculating information evaluation parameters, the above technical solution can achieve more personalized evaluation. Different patients will show different symptoms or behavior patterns, and the above technical solution can make more accurate evaluations based on individual differences. The comprehensive evaluation index parameters can provide doctors with objective and quantitative reference information to assist doctors in making more accurate diagnosis and treatment decisions. At the same time, it also has important guiding significance for the prediction and early intervention of mental illness.

[0095] In summary, this technical solution achieves a comprehensive, objective and personalized assessment of the patient's mental state by extracting multidimensional data from the medical system and performing screening and comprehensive evaluation, providing strong support for doctors' diagnosis and treatment work.

[0096] An embodiment of the present invention provides for respectively screening the text data information, the sound data information and the video data information to obtain key data information corresponding to the text data information, the sound data information and the video data information, including:

[0097] S201, preprocessing the text data information, screening the preprocessed text data information, and obtaining key text information in the text data information corresponding to the patient;

[0098] S202, preprocessing the sound data information, screening the preprocessed sound data information, and obtaining key sound information in the sound data information corresponding to the patient object;

[0099] S203: pre-process the video data information, screen the pre-processed video data information, and obtain key image information in the video data information corresponding to the patient.

[0100] The working principle of the above technical solution is: pre-processing text data information (such as medical records, patient descriptions, etc.), including removing irrelevant characters and punctuation marks, performing word segmentation, part-of-speech tagging, etc., for subsequent analysis. Using natural language processing (NLP) technology, such as keyword extraction and topic modeling, the pre-processed text data is analyzed to extract key text information related to the patient's condition, symptoms, psychological state, etc.

[0101] Preprocess the sound data information, such as noise reduction, echo removal, audio format conversion, etc., to improve the audio quality. Use speech recognition technology to convert the sound data into text, and then use NLP technology to extract key sound information, such as the patient's emotional expression, voice characteristics (such as speaking speed, pitch, etc.), etc. This information helps analyze the patient's psychological state.

[0102] Preprocess the video data information, including video format conversion, key frame extraction, etc., for subsequent analysis. Use video analysis technologies such as face recognition, behavior analysis, and motion recognition to extract key image information related to the patient's condition, behavior pattern, etc. from the video data.

[0103] The effect of the above technical solution is: by preprocessing and screening text, sound and video data information, it is possible to more accurately extract information related to the patient's condition and psychological state and reduce the interference of irrelevant information. Combining information of the three data types of text, sound and video can provide a more comprehensive understanding of the patient's condition, including language expression, emotional state, behavior patterns, etc., providing a richer basis for the doctor's diagnosis. The use of automated data preprocessing and screening technology can greatly reduce the workload of manual processing and improve the efficiency of data processing and analysis. The screened key data information can provide doctors with objective and quantitative reference information, assisting doctors in making more accurate diagnosis and treatment decisions.

[0104] In summary, this technical solution can accurately and efficiently extract key data information related to the patient's condition and psychological state by preprocessing and screening text, sound and video data information respectively, providing strong support for doctors' diagnosis and treatment.

[0105] An embodiment of the present invention pre-processes the text data information, screens the pre-processed text data information, and obtains key text information in the text data information corresponding to the patient, including:

[0106] S2011, performing text-clearing processing on the text data information, removing irrelevant characters, irrelevant punctuation marks and stop words in the text data information, and obtaining text data information after text-clearing processing corresponding to the text data information;

[0107] S2012, performing word segmentation processing on the text data information after the text legibility processing, and obtaining the words, phrases and sentences corresponding to the text data information after the text legibility processing;

[0108] S2013, performing part-of-speech tagging on the words, phrases and sentences corresponding to the text data information, and converting the words, phrases and sentences corresponding to the text data information into word embedding vectors;

[0109] S2014, using a convolutional neural network model to extract text features from the word embedding vector corresponding to the text data information, to obtain text feature information contained in the text data information, wherein the text feature information includes words, phrases, and sentence structures used to characterize characteristics of mental illness; and the words, phrases, and sentence structures used to characterize characteristics of mental illness are key text information in the text data information corresponding to the patient;

[0110] S2015. Obtain text information evaluation parameters of the patient using key text information in the text data information corresponding to the patient.

[0111] The working principle of the above technical solution is: the original text data information is preliminarily cleaned up, and irrelevant characters, irrelevant punctuation marks and stop words (such as common conjunctions, auxiliary words, etc.) are removed to reduce the impact of noise data on subsequent analysis. The text data information after clear processing is further decomposed into words, phrases and sentences, which is a common step in Chinese natural language processing and helps to understand the text content more carefully. The word groups, phrases and sentences after word segmentation are tagged with parts of speech, that is, the grammatical function of each word is determined. Converting these words, phrases and sentences into word embedding vectors helps computers understand and process text information. The word embedding vector is feature extracted using a convolutional neural network model (CNN). CNN can capture local and global patterns in the text and extract vocabulary, phrases and sentence structures related to the characteristics of mental illness. These feature information (such as specific vocabulary usage, grammatical structure, etc.) are usually related to the manifestations or symptoms of a certain mental illness, and are therefore regarded as key text information. Based on the extracted key text information, the patient's text information evaluation parameters are calculated or evaluated. The parameters reflect the degree or intensity of the mental illness characteristics reflected in the patient's text expression.

[0112] The effect of the above technical solution is: using the CNN model to extract text features can more accurately capture text features related to mental illness and improve the accuracy of information screening. The entire process is automated, reducing manual intervention, improving processing efficiency, and reducing the possibility of human error. Through in-depth analysis of the patient's text information, personalized key text information can be obtained, thereby making a more accurate and personalized assessment. The extracted key text information and text information evaluation parameters can provide doctors with objective and quantitative references, assisting doctors in making more accurate diagnoses and formulating more appropriate treatment plans.

[0113] In summary, this technical solution achieves effective screening of text data information and extraction of key text information through a series of automated and precise text processing and analysis steps, providing strong technical support for the prediction and assessment of mental illness.

[0114] An embodiment of the present invention uses key text information in the text data information corresponding to the patient to obtain the text information evaluation parameter of the patient, including:

[0115] Step A1, classifying the text feature information to obtain a vocabulary information set, a phrase information set and a sentence structure information set;

[0116] Step A2: using the vocabulary feature information contained in the vocabulary information set to obtain the first text information evaluation parameter value corresponding to the vocabulary feature information; wherein the first text information evaluation parameter value is obtained by the following formula:

[0117] W 01 =ln(1+w x01 -w x02 )

[0118]

[0119]

[0120] Among them, W 01 represents the first text information evaluation parameter value; x01 represents the first parameter coefficient; w x02 represents the second parameter coefficient; n represents the number of words contained in the vocabulary information set; G represents the preset reference word distance (the word distance is the number of words between two words); L i Indicates the starting position of the i-th word; L i+1 Indicates the starting position of the i+1th word; Z i represents the number of words contained in the i-th word; D represents the number of words in the sentence where the i-th word is located;

[0121] Step A3: using the phrase feature information contained in the phrase information set to obtain a second text information evaluation parameter value corresponding to the phrase feature information; wherein the second text information evaluation parameter value is obtained by the following formula:

[0122] W 02 =ln(1+w x03 -w x04 )

[0123]

[0124]

[0125] Among them, W 02 represents the second text information evaluation parameter value; x03 represents the third parameter coefficient; w x04 represents the fourth parameter coefficient; m represents the number of phrases contained in the phrase information set; Z di Indicates the number of words contained in the i-th phrase; Z ci Indicates the number of words contained in the i-th phrase; D d Indicates the number of words in the sentence where the i-th phrase is located; J i Indicates the starting position of the i-th phrase; J i+1 Indicates the starting position of the i+1th phrase;

[0126] Step A4: using the structural feature information contained in the sentence structure information set to obtain a third text information evaluation parameter value corresponding to the structural feature information; wherein the third text information evaluation parameter value is obtained by the following formula:

[0127]

[0128] Among them, W 03 Indicates the third text information evaluation parameter value; N y N represents the number of sentences in the sentence structure information set that meet the sentence structure characteristics of mental illness; z represents the total number of sentences contained in the text data information; P represents the total proportion of words representing the characteristics of mental illness in the case where the sentences satisfy the sentence structure representing the characteristics of mental illness;

[0129] Step A5: using the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value to obtain the patient's text information evaluation parameter; wherein the text information evaluation parameter is obtained by the following formula:

[0130] W=λ 01 ·W 01 +λ 02 ·W 02 +λ 03 ·W 03

[0131] Where W represents the text information evaluation parameter; λ 01 , 02 and λ 03 They respectively represent the weight values ​​corresponding to the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value.

[0132] The working principle of the above technical solution is: the text feature information is classified according to vocabulary, phrases and sentence structure to form three information sets: vocabulary information set, phrase information set and sentence structure information set. The first text information evaluation parameter value is calculated by a specific formula using the vocabulary feature information in the vocabulary information set, combined with the preset reference word distance, the starting position of the vocabulary, the number of words contained and the number of words in the sentence. Similarly, the second text information evaluation parameter value is calculated by a specific formula using the phrase feature information in the phrase information set, including the number of words contained in the phrase, the number of words occupied by the vocabulary, the starting position of the phrase and the number of words in the sentence. The third text information evaluation parameter value is calculated by a formula using the number of sentences that meet the characteristic structure of mental illness in the sentence structure information set, the total number of sentences in the text data information, and the proportion of mental illness characteristic words in a specific sentence structure. Finally, according to the three evaluation parameter values ​​and their corresponding weight values, the patient's text information evaluation parameter is calculated by a formula.

[0133] The effect of the above technical solution is: by classifying and calculating the text feature information in detail, the key features related to mental illness can be extracted more accurately. The evaluation parameter values ​​and evaluation parameters calculated by the formula can provide an objective and quantitative evaluation standard, reducing the influence of subjective factors. Since the evaluation parameters are calculated based on the individual patient's text data information, they can reflect the patient's unique language characteristics and realize personalized evaluation. The text information evaluation parameters can provide doctors with a powerful auxiliary tool to help doctors diagnose mental illness more accurately and improve the accuracy of diagnosis. By understanding the characteristics of the patient's text expression, doctors can formulate and adjust treatment plans more accurately.

[0134] On the other hand, by carefully classifying the text feature information and extracting the feature information at the vocabulary, phrase and sentence structure levels, the above technical solution can more accurately capture the key information in the patient's text information, thereby more accurately evaluating the patient's text information. The above technical solution not only takes into account the information at the vocabulary level, but also incorporates higher-level language units such as phrases and sentence structures. This comprehensive information extraction method helps to evaluate the patient's text information from multiple perspectives and improve the comprehensiveness of the evaluation. At the same time, the parameter coefficients (such as wx01, wx02, wx03, etc.) and weight values ​​(such as λ01, λ02, λ03) can be adjusted according to actual conditions, which can adapt to different evaluation needs and scenarios and has high flexibility. In addition, the evaluation parameter values ​​(such as W01, W02, W03) and the final text information evaluation parameters (W) in the above technical solution can effectively improve the accuracy and interpretability of the evaluation results, which is convenient for doctors and researchers to understand and analyze.

[0135] In summary, this technical solution achieves an in-depth understanding and objective evaluation of patients' text data information through detailed and precise calculations and analysis, providing effective technical support for the prediction and evaluation of mental illness.

[0136] An embodiment of the present invention pre-processes the sound data information, screens the pre-processed sound data information, and obtains key sound information in the sound data information corresponding to the patient object, including:

[0137] S2021, performing sound signal cleaning processing on the sound data information, eliminating background noise and electromagnetic interference signals in the sound data information, and obtaining the sound data information after the sound signal cleaning processing;

[0138] S2022, segmenting the sound data information after the sound signal cleaning process according to preset time intervals to obtain segmented sound data information;

[0139] S2023, performing standardization processing on the segmented sound data information to obtain the standardized sound data information;

[0140] S2024, inputting the standardized sound data information into a convolutional neural network model, and obtaining sound data features corresponding to the sound data information, wherein the sound data features include abnormal vocalization features (abnormal vocalization includes but is not limited to screaming, slurred speech, and hoarseness, etc.), intonation change features (intonation changes include but are not limited to sudden rise and fall of pitch, etc.), and speech rate change features (speech rate changes include but are not limited to sudden rise and fall of speech rate, speech disfluency, abnormal pauses, and language repetition, etc.);

[0141] S2025. Obtain a sound information evaluation parameter of the patient using the sound data features corresponding to the sound data information.

[0142] The working principle of the above technical solution is: the original sound data information is cleaned to eliminate the background noise and electromagnetic interference signals therein to ensure the effectiveness of subsequent analysis. The cleaned sound data information is segmented according to preset time intervals. Segmentation processing helps to capture local features in the sound and reduce the amount of calculation. The segmented sound data information is standardized, which usually involves adjusting the volume, eliminating channel differences, etc., so that different sound samples are comparable in subsequent analysis. The standardized sound data information is input into the convolutional neural network model. The convolutional neural network can automatically learn the features in the sound data and extract sound data features including abnormal vocalization features, intonation change features, and speech speed change features. According to the extracted sound data features, the patient's sound information evaluation parameters are calculated or evaluated.

[0143] The effect of the above technical solution is: through the sound signal cleaning process, the background noise and electromagnetic interference signals are effectively eliminated, the quality of the sound data is improved, and a more reliable basis is provided for subsequent feature extraction and analysis. The convolutional neural network model is used for sound data feature extraction, which can automatically learn and capture the complex features in the sound, including abnormal phonation, changes in intonation and changes in speech speed, etc., thereby improving the accuracy and efficiency of feature extraction. Through in-depth analysis of the patient's sound data information, personalized sound features can be obtained, so as to make more accurate and personalized assessments. The extracted sound data features and sound information evaluation parameters can provide doctors with objective and quantitative references, assist doctors in making more accurate diagnoses, and formulate more appropriate treatment plans. The entire processing flow is automated, reducing manual intervention, improving processing efficiency, and reducing the possibility of human errors.

[0144] In summary, this technical solution achieves effective screening of sound data information and extraction of key sound information through data preprocessing and feature extraction steps, providing strong technical support for the prediction and assessment of mental illness.

[0145] An embodiment of the present invention uses the sound data features corresponding to the sound data information to obtain the sound information evaluation parameters of the patient, including:

[0146] Step B1, retrieving the sound data features corresponding to the sound data information of the patient;

[0147] Step B2: using the abnormal sound feature in the sound data feature to obtain a first sound information evaluation parameter value; wherein the first sound information evaluation parameter value is obtained by the following formula:

[0148]

[0149] Among them, Y 01 represents the first sound information evaluation parameter value; k 01 Indicates the number of abnormal sounds in the sound data feature; T 01i represents the duration of the ith abnormal sound; T represents the total duration of the sound data; P 11i represents the proportion of time that the vocabulary representing the characteristics of mental illness contains in the duration of the ith abnormal voice; P 12i represents the proportion of time that phrases representing characteristics of mental illness are contained in the duration of the ith abnormal vocalization; P 13i represents the proportion of the time that the sentence structure representing the characteristics of mental illness is contained in the duration of the i-th abnormal voice;

[0150] Step B3: using the intonation change feature in the sound data feature to obtain a second sound information evaluation parameter value; wherein the second sound information evaluation parameter value is obtained by the following formula:

[0151]

[0152] Among them, Y 02 represents the second sound information evaluation parameter value; k 02 Indicates the number of intonation changes in the sound data feature; T 02i represents the duration of the i-th intonation change; T xi represents the duration of abnormal utterance contained in the duration of the ith intonation change; P 21i represents the proportion of time that the words representing the characteristics of mental illness are contained in the duration of the ith intonation change, excluding the abnormal vocalization content; P 22i represents the proportion of time that phrases representing the characteristics of mental illness occupy in the duration of the i-th intonation change, excluding the abnormal vocalization content; P 23i represents the proportion of the time that the sentence structure representing the characteristics of mental illness excluding the abnormal vocalization content in the duration of the i-th intonation change;

[0153] Step B4: using the speech rate change feature in the sound data feature to obtain a third sound information evaluation parameter value; wherein the third sound information evaluation parameter value is obtained by the following formula:

[0154]

[0155] Among them, Y 03 represents the third sound information evaluation parameter value; k 03 Indicates the number of times the speech rate changes in the sound data feature; T 03i represents the duration of the i-th speech rate change; P xi represents the total duration ratio of abnormal utterance and intonation change included in the duration of the ith speech rate change;

[0156] Step B5: using the first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value to obtain the patient's sound information evaluation parameter; wherein the sound information evaluation parameter is obtained by the following formula:

[0157] Y=α 03 ·Y 03 -α 01 ·Y 01 +α 02 ·Y 02

[0158] Where Y represents the sound information evaluation parameter; α01 , α 02 and α 03 Respectively represent the weight values ​​corresponding to the first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value.

[0159] The working principle of the above technical solution is: retrieve information related to the sound data features from the sound data information, and these features include abnormal vocalization features, intonation change features, and speech speed change features. According to the number of abnormal vocalizations in the abnormal vocalization features, the duration of each abnormal vocalization, the total duration of the sound data, and the time proportion of the vocabulary, phrases, and sentence structures that characterize the characteristics of mental illness in the abnormal vocalization content, the first sound information evaluation parameter value is calculated by a specific formula. The above technical solution reflects the frequency and characteristics of abnormal vocalizations in the patient's voice. The second sound information evaluation parameter value is calculated by a formula using the number of intonation changes in the intonation change features, the duration of each intonation change, the duration of abnormal vocalizations, and the time proportion of the vocabulary, phrases, and sentence structures that characterize the characteristics of mental illness in the intonation changes after removing the abnormal vocalization content. The above technical solution reflects the characteristics of intonation changes in the patient's voice and the degree of association with mental illness. According to the number of speech rate changes in the speech rate change characteristics, the duration of each speech rate change, and the total duration ratio of abnormal utterances and intonation changes in the speech rate change, the third sound information evaluation parameter value is calculated by a formula. The above technical scheme reflects the frequency and characteristics of speech rate changes in the patient's voice. Combining the above three evaluation parameter values ​​and their corresponding weight values, the patient's voice information evaluation parameter is calculated by a formula. The parameter is a comprehensive quantitative evaluation of the characteristics of mental illness in the patient's voice.

[0160] The effect of the above technical solution is: by comprehensively considering various sound data features such as abnormal vocalization, intonation changes and speech speed changes, the technical solution can more comprehensively and accurately evaluate the patient's sound information, and provide a multi-angle reference for the prediction and evaluation of mental illness. The sound information evaluation parameter values ​​and evaluation parameters calculated by the formula provide doctors with objective and quantitative evaluation standards, reduce the influence of subjective factors, and improve the accuracy and reliability of the evaluation. Since the sound data characteristics and evaluation parameters are calculated based on the sound data information of individual patients, they can reflect the unique language characteristics of patients, realize personalized evaluation, and help to formulate more accurate treatment plans. The sound information evaluation parameters can provide doctors with a powerful auxiliary tool to help doctors more accurately identify and understand the voice characteristics of patients. The entire evaluation process is automated and quantified, which improves the evaluation efficiency, while reducing the possibility of human errors, making the evaluation process more efficient and accurate.

[0161] On the other hand, the above technical solution realizes the quantification of the sound characteristics of mental illness by extracting abnormal vocalization features, intonation change features and speech speed change features from the sound data, and converting these features into specific evaluation parameter values. This quantification method helps to more accurately evaluate the relationship between the patient's voice information and mental illness. At the same time, traditional sound information evaluation often relies on the subjective judgment of doctors or experts, and there is a large degree of subjectivity and uncertainty. The above technical solution evaluates sound information through objective formulas and algorithms, reduces the influence of subjective factors, and improves the objectivity and accuracy of the evaluation. In addition, the above technical solution comprehensively considers the sound data characteristics of abnormal vocalization, intonation changes and speech speed changes, so that the patient's sound information can be comprehensively evaluated from multiple angles, improving the accuracy and effectiveness of data information acquisition, and avoiding the interference of invalid information.

[0162] The above technical solution can help doctors more accurately judge the patient's mental state and improve the accuracy of diagnosing mental illness by quantifying the characteristics of sound data and extracting key information. Compared with the traditional subjective evaluation method, the above technical solution is based on objective formulas and algorithms for evaluation, which reduces the interference of human factors and thus improves the accuracy of the evaluation results. At the same time, the above technical solution can capture subtle changes in sound data, such as abnormal vocalizations, changes in intonation and speech speed, thereby improving the sensitivity of the evaluation.

[0163] In summary, this technical solution uses the sound data features corresponding to the sound data information to obtain the patient's sound information evaluation parameters, providing an efficient, objective and comprehensive method for the prediction and evaluation of mental illness. At the same time, this technical solution achieves the accuracy of data screening and evaluation of the sound features of mental illness by quantifying the sound data features and extracting key information, improving the accuracy of diagnosis and the objectivity of evaluation, and has high practical application value and scientific research value.

[0164] An embodiment of the present invention pre-processes the video data information, screens the pre-processed video data information, and obtains key image information in the video data information corresponding to the patient, including:

[0165] S2031, performing frame processing on the video data information to obtain a static image of each frame corresponding to the video data information;

[0166] S2032, performing image quality optimization processing on the static image of each frame to obtain a static image after optimization processing; wherein the image quality optimization processing includes image denoising processing and contrast enhancement processing;

[0167] S2033, inputting the optimized static image into a convolutional neural network model to obtain video data features corresponding to the video data information, wherein the video data features include abnormal facial expression features and abnormal body movement features;

[0168] S2034. Obtain video information evaluation parameters of the patient using video data features corresponding to the video data information.

[0169] Specifically, abnormal facial expression features include but are not limited to:

[0170] Eye gaze: including fixed, erratic, dull or overactive eye gaze, which is related to emotional expression, attention or cognitive function of mental illness.

[0171] Mouth movements: such as rising or falling corners of the mouth, pursed lips or trembling lips, etc., are related to emotional state, anxiety or depression.

[0172] Facial muscle tension: The relaxation or tension of facial muscles reflects the patient's emotional state or the side effects of medication.

[0173] Abnormal body movement characteristics include but are not limited to:

[0174] Body posture: such as leaning forward, leaning back, or bending to the side, etc., are related to anxiety, depression, or delusional symptoms of mental illness.

[0175] Gestures: This includes the frequency, amplitude, and complexity of gestures, such as rapid and chaotic gestures that are associated with agitated or anxious states.

[0176] Motor coordination: Fluidity and coordination of movements, such as tremors, rigidity, or hyperactivity, are associated with movement disorders in psychiatric illnesses.

[0177] Behavioral pattern characteristics:

[0178] Activity level: This includes the amount and rate of activity of the patient, such as hyperactivity or hypoactivity, which is related to the excited or inhibited state of mental illness.

[0179] Social interaction: Changes in eye contact, facial expressions, and body language when communicating with others can reflect the patient's social skills and emotional expression.

[0180] Repetitive behaviors, such as repeated movements or rituals, are associated with mental illnesses such as obsessive-compulsive disorder and schizophrenia.

[0181] The working principle of the above technical solution is: decompose the video data information into separate frames, that is, obtain the static image of each frame corresponding to the video data information. This is done to analyze the video content frame by frame, so that each frame of the image can be processed independently later. The image quality of each frame of the static image is optimized, including image denoising and contrast enhancement. Image denoising aims to eliminate noise in the image and improve the clarity of the image; contrast enhancement is used to adjust the contrast of the image so that the details in the image are more prominent, which is convenient for subsequent feature extraction. The optimized static image is input into the convolutional neural network model. The convolutional neural network can automatically learn the features in the image and extract key features related to mental illness, such as abnormal facial expression features and abnormal limb movement features. According to the extracted video data features, the patient's video information evaluation parameters are calculated or evaluated. The above technical solution reflects the degree or intensity of the mental illness characteristics reflected by the patient in the video.

[0182] The effect of the above technical solution is: by performing frame processing and image quality optimization processing on the video data information, noise can be eliminated and image quality can be improved, thereby improving the accuracy of subsequent feature extraction. Using the convolutional neural network model for feature extraction, it is possible to automatically learn and capture key features in the video, such as abnormal facial expressions and abnormal body movements, reducing the need for manual intervention and subjective judgment. Through in-depth analysis of the patient's video data information, personalized image features can be obtained, thereby making more accurate and personalized assessments. The extracted video data features and video information evaluation parameters can provide doctors with objective and quantitative references, assisting doctors in making more accurate diagnoses and formulating more appropriate treatment plans. The entire processing flow is automated, reducing the time and cost of manual processing and improving processing efficiency.

[0183] In summary, this technical solution achieves effective screening and evaluation of key imaging information in patient video data information through data preprocessing and feature extraction, providing strong technical support for the prediction and evaluation of mental illness.

[0184] An embodiment of the present invention uses the video data features corresponding to the video data information to obtain the video information evaluation parameters of the patient, including:

[0185] Step C1, retrieving video data features corresponding to the video data information of the patient;

[0186] Step C2, using the abnormal facial expression feature in the video data feature to obtain a first video information evaluation parameter value; wherein the first video information evaluation parameter value is obtained by the following formula:

[0187]

[0188] Among them, S 01 represents the first video information evaluation parameter value; r 01 Indicates the total number of abnormal facial expressions; T y01i represents the duration of the ith abnormal facial expression; T z Indicates the total duration of the video data information; T x11i and T x12i They represent the duration of abnormal vocalization and tone change in the ith abnormal facial expression; T c01i represents the overlapping duration of abnormal vocalization and intonation change in the duration of the i-th abnormal facial expression;

[0189] Step C3, using the abnormal body movement feature in the sound data feature to obtain a second video information evaluation parameter value; wherein the second video information evaluation parameter value is obtained by the following formula:

[0190]

[0191] Among them, S 02 represents the second video information evaluation parameter value; r 02 Indicates the total number of abnormal limb movements; T y02i represents the duration of the ith abnormal limb movement; P yi It indicates the proportion of time that abnormal facial expressions are included in the duration of the i-th abnormal limb movement;

[0192] Step C4: using the first video information evaluation parameter value and the second video information evaluation parameter value to obtain the patient's video information evaluation parameter; wherein the video information evaluation parameter is obtained by the following formula:

[0193] S=β 01 ·S 01 +β 02 ·S 02

[0194] Among them, S represents the video information evaluation parameter; β 01 and β 02 Respectively represent the weight values ​​corresponding to the first video information evaluation parameter value and the second video information evaluation parameter value.

[0195] The working principle of the above technical solution is: retrieve the corresponding video data features from the patient's video data information, which mainly include abnormal facial expression features and abnormal limb movement features. According to the abnormal facial expression features, calculate the first video information evaluation parameter value (S01). The calculation process takes into account the total number of abnormal facial expressions (r01), the duration of each abnormal facial expression (Ty01i), the total duration of the video data information (Tz), and the duration of abnormal vocalization and tone changes in abnormal facial expressions (Tx11i, Tx12i) and their overlapping duration (Tc01i). These factors together reflect the frequency and duration of abnormal facial expressions and their degree of correlation with sound features.

[0196] According to the abnormal limb movement characteristics, the second video information evaluation parameter value (S02) is calculated. The total number of abnormal limb movements (r02), the duration of each abnormal limb movement (Ty02i), and the proportion of time containing abnormal facial expressions in the duration of abnormal limb movements (Pyi) are taken into account in the calculation. These indicators comprehensively reflect the frequency and duration of abnormal limb movements and the degree of correlation between them and abnormal facial expressions.

[0197] The first video information evaluation parameter value (S01) and the second video information evaluation parameter value (S02), as well as their respective corresponding weight values ​​(β01 and β02), are combined to calculate the patient's video information evaluation parameter (S). The parameter is a comprehensive quantitative evaluation of abnormal facial expressions and abnormal body movement features in the patient's video data.

[0198] The effect of the above technical solution is: the technical solution comprehensively considers the two video data features of abnormal facial expressions and abnormal limb movements, can evaluate the patient's video information more comprehensively and accurately, and provides a multi-angle reference for the prediction and evaluation of mental illness. The first video information evaluation parameter value, the second video information evaluation parameter value and the video information evaluation parameter calculated by the formula provide doctors with objective and quantitative evaluation standards, reduce the influence of subjective factors, and improve the accuracy and reliability of the evaluation. The entire evaluation process is automatically calculated based on algorithms and models, which improves the evaluation efficiency and reduces the time and cost of manual processing. Since the video data features and evaluation parameters are calculated based on the individual video data information of the patient, they can reflect the patient's unique video features, realize personalized evaluation, and help to formulate a more accurate treatment plan. The video information evaluation parameters can provide doctors with a powerful auxiliary tool to help doctors more accurately identify and understand the patient's video features, thereby effectively improving the accuracy and efficiency of data screening with higher effectiveness.

[0199] Abnormal facial expressions and body movements of patients with mental illness in videos are important clues for assessing their mental condition. The above technical solution provides key information for subsequent evaluation by retrieving the video data features corresponding to the patient's video data information, especially abnormal facial expressions and body movements. Traditional video information evaluation often relies on the subjective judgment of doctors or experts, which is highly subjective and uncertain. The above technical solution uses objective mathematical models and algorithms to extract and quantify the features of abnormal facial expressions and body movements, making the evaluation more objective and accurate. On the other hand, the manifestations of mental illness are multifaceted, and evaluation based on only a single dimension of information (such as facial expressions or body movements) may not be comprehensive enough. The above technical solution comprehensively considers the information of two dimensions, abnormal facial expressions and body movements, so that data information for comprehensive assessment of the patient's mental condition can be provided from multiple angles.

[0200] By extracting and quantifying key features in video data, the above technical solution can more accurately capture the abnormal manifestations of patients with mental illness, thereby improving the accuracy and sensitivity of diagnosis. Compared with traditional subjective evaluation methods, the above technical solution is based on objective formulas and algorithms for evaluation, which reduces the interference of human factors and makes the evaluation results more objective and consistent. Through automated feature extraction and quantification processes, the above technical solution can efficiently process large amounts of video data, improving the efficiency and degree of automation of evaluation. The above technical solution can not only be used for clinical diagnosis, but also provide a large amount of objective and accurate data support for scientific research, which is helpful for in-depth research on the pathogenesis and laws of mental illness.

[0201] In summary, this technical solution provides an efficient, objective and comprehensive method for predicting and evaluating mental illness by using the video data features corresponding to the video data information to obtain the patient's video information evaluation parameters. At the same time, this technical solution achieves accurate evaluation of the video information of patients with mental illness by extracting and quantifying the key features in the video data, thereby improving the accuracy of diagnosis and the objectivity of evaluation.

[0202] An embodiment of the present invention uses information evaluation parameters corresponding to the text data information, the sound data information, and the video data information to obtain comprehensive evaluation index parameters of the patient, including:

[0203] S301, extracting text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters of the patient;

[0204] S302, using the comprehensive model to obtain comprehensive evaluation index parameters of the patient through text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters, wherein the comprehensive model structure is as follows:

[0205] Q=logW+logY+logS

[0206] Among them, Q represents the comprehensive evaluation index parameter of the patient; W represents the text information evaluation parameter; Y represents the sound information evaluation parameter; S represents the video information evaluation parameter.

[0207] The working principle of the above technical solution is: extract corresponding text information evaluation parameters (W), sound information evaluation parameters (Y) and video information evaluation parameters (S) from the patient's text data information, sound data information and video data information respectively. These parameters are obtained through the respective data preprocessing and feature extraction steps, and reflect the mental health status or disease characteristics of the patient in different data types.

[0208] The extracted text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters are input into the comprehensive model. The comprehensive model is a pre-built mathematical model that calculates the comprehensive evaluation index parameters (Q) of the patient based on these parameters.

[0209] In the comprehensive model, text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters are respectively used as inputs of the model. Through certain algorithms and weight distribution, a comprehensive evaluation index parameter is finally obtained, which helps to comprehensively quantify and accurately represent the patient's mental health status or disease characteristics.

[0210] The effect of the above technical solution is: the technical solution combines three different types of data information, text, sound and video, and can more comprehensively evaluate the patient's mental health status or disease characteristics. Different types of data information can provide different observation angles and information, complement each other, and improve the accuracy and completeness of the evaluation. The comprehensive evaluation index parameter calculated by the comprehensive model is a quantitative indicator that can objectively reflect the patient's mental health status or disease characteristics. This quantitative evaluation method reduces the influence of subjective factors and improves the reliability and consistency of the evaluation.

[0211] Since the comprehensive evaluation index parameters are calculated based on the text, sound and video data information of the individual patient, they can reflect the unique characteristics of the patient and achieve personalized evaluation. This helps doctors understand the patient's condition more accurately and formulate more appropriate treatment plans. The comprehensive evaluation index parameters can provide doctors with a powerful auxiliary tool to help doctors understand the patient's mental health status or disease characteristics more comprehensively. The entire evaluation process is automatically calculated based on algorithms and models, which improves evaluation efficiency and reduces the time and cost of manual processing.

[0212] In summary, this technical solution provides a comprehensive, objective and efficient method for the prediction and assessment of mental illness by integrating the patient's text, sound and video data information and using a comprehensive model to calculate the comprehensive evaluation index parameters.

[0213] The embodiment of the present invention proposes a multidimensional data screening system for predicting mental illness. Figure 2 As shown, the multidimensional data screening system applied to mental illness prediction includes:

[0214] A data information extraction module is used to extract text data information, sound data information and video data information during the patient consultation process from the medical system;

[0215] An evaluation parameter acquisition module, used to screen the text data information, the sound data information and the video data information respectively, and acquire key data information corresponding to the text data information, the sound data information and the video data information and their corresponding information evaluation parameters;

[0216] The comprehensive evaluation index parameter acquisition module is used to obtain the comprehensive evaluation index parameters of the patient using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information.

[0217] The working principle of the above technical solution is: to extract the text data information, sound data information and video data information generated by the patient during the consultation process from the medical system using technical means (such as data crawlers or API interfaces).

[0218] The extracted text data information, sound data information and video data information are screened respectively. The screening process involves natural language processing (NLP) technology to identify keywords or key sentences in text data, speech recognition technology to parse key information in sound data, and video analysis technology to extract key frames or behavior patterns in video data. Through the above technical solution, the key data information corresponding to each data type can be obtained, and the corresponding information evaluation parameters can be calculated or assigned.

[0219] Based on the information evaluation parameters of text data information, sound data information and video data information, the comprehensive evaluation index parameters of the patient are calculated through certain algorithms or models (such as weighted summation, machine learning model, etc.). The comprehensive evaluation index parameters are helpful for the comprehensive and objective quantitative evaluation of the patient's mental state and improve the accuracy of data evaluation and screening.

[0220] The effect of the above technical solution is: by using information of three data types, text, sound and video, the above technical solution can reflect the patient's mental state more comprehensively and accurately. Different types of data can provide complementary information and reduce the possibility of information loss or misjudgment. The above technical solution realizes the automation of data screening and comprehensive evaluation, reduces the need for manual operation, and improves work efficiency. At the same time, automated processing can also reduce the interference of human factors and improve the objectivity of evaluation results. By combining multiple data information and calculating information evaluation parameters, the above technical solution can achieve more personalized evaluation. Different patients will show different symptoms or behavior patterns, and the above technical solution can make more accurate evaluations based on individual differences. The comprehensive evaluation index parameters can provide doctors with objective and quantitative reference information to assist doctors in making more accurate diagnosis and treatment decisions. At the same time, it also has important guiding significance for the prediction and early intervention of mental illness.

[0221] In summary, this technical solution achieves a comprehensive, objective and personalized assessment of the patient's mental state by extracting multidimensional data from the medical system and performing screening and comprehensive evaluation, providing strong support for doctors' diagnosis and treatment work.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multidimensional data screening method for predicting mental illness, characterized in that: The multidimensional data screening method applied to mental illness prediction includes: Extract text data information, sound data information and video data information during the patient consultation process from the medical system; Performing data screening on the text data information, the sound data information and the video data information respectively, and obtaining key data information corresponding to the text data information, the sound data information and the video data information and corresponding information evaluation parameters; Obtaining comprehensive evaluation index parameters of the patient using information evaluation parameters corresponding to the text data information, sound data information and video data information; The information evaluation parameters corresponding to the key data information corresponding to the text data information, the sound data information and the video data information include the following steps: Obtaining text information evaluation parameters of the patient using key text information in the text data information corresponding to the patient; Acquiring a sound information evaluation parameter of the patient using a sound data feature corresponding to the sound data information; Obtaining video information evaluation parameters of the patient using video data features corresponding to the video data information; Wherein, obtaining the text information evaluation parameters of the patient using the key text information in the text data information corresponding to the patient includes: Performing feature classification on the text feature information to obtain a vocabulary information set, a phrase information set, and a sentence structure information set; The lexical feature information contained in the lexical information set is used to obtain a first text information evaluation parameter value corresponding to the lexical feature information; wherein the first text information evaluation parameter value is obtained by the following formula: Among them, W 01 represents the first text information evaluation parameter value; x01 represents the first parameter coefficient; w x02 represents the second parameter coefficient; n represents the number of words contained in the vocabulary information set; G represents the preset reference word distance; L i Indicates the starting position of the i-th word; L i+1 Indicates the starting position of the i+1th word; Z i represents the number of words contained in the i-th word; D represents the number of words in the sentence where the i-th word is located; The phrase feature information included in the phrase information set is used to obtain a second text information evaluation parameter value corresponding to the phrase feature information; wherein the second text information evaluation parameter value is obtained by the following formula: Among them, W 02 represents the second text information evaluation parameter value; x03 represents the third parameter coefficient; w x04 represents the fourth parameter coefficient; m represents the number of phrases contained in the phrase information set; Z di Indicates the number of words contained in the i-th phrase; Z ci Indicates the number of words contained in the i-th phrase; D d Indicates the number of words in the sentence where the i-th phrase is located; J i Indicates the starting position of the i-th phrase; J i+1 Indicates the starting position of the i+1th phrase; The structural feature information contained in the sentence structure information set is used to obtain a third text information evaluation parameter value corresponding to the structural feature information; wherein the third text information evaluation parameter value is obtained by the following formula: Among them, W 03 Indicates the third text information evaluation parameter value; N y N represents the number of sentences in the sentence structure information set that meet the sentence structure characteristics of mental illness; z represents the total number of sentences contained in the text data information; P represents the total proportion of words representing the characteristics of mental illness in the case where the sentences satisfy the sentence structure representing the characteristics of mental illness; The patient's text information evaluation parameter is obtained by using the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value; wherein the text information evaluation parameter is obtained by the following formula: Where W represents the text information evaluation parameter; λ 01 , 02 and λ 03 Respectively represent the weight values ​​corresponding to the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value; At the same time, obtaining the patient's voice information evaluation parameters using the voice data features corresponding to the voice data information includes: Retrieving the sound data features corresponding to the sound data information of the patient; The first sound information evaluation parameter value is obtained by using the abnormal sound feature in the sound data feature; wherein the first sound information evaluation parameter value is obtained by the following formula: Among them, Y 01 represents the first sound information evaluation parameter value; k 01 Indicates the number of abnormal sounds in the sound data feature; T 01i represents the duration of the ith abnormal sound; T represents the total duration of the sound data; P 11i represents the proportion of time that the vocabulary representing the characteristics of mental illness contains in the duration of the ith abnormal voice; P 12i represents the proportion of time that phrases representing characteristics of mental illness are contained in the duration of the ith abnormal vocalization; P 13i represents the proportion of the time that the sentence structure representing the characteristics of mental illness is contained in the duration of the i-th abnormal voice; The second sound information evaluation parameter value is obtained by using the intonation change feature in the sound data feature; wherein the second sound information evaluation parameter value is obtained by the following formula: Among them, Y 02 represents the second sound information evaluation parameter value; k 02 Indicates the number of intonation changes in the sound data feature; T 02i represents the duration of the i-th intonation change; T xi represents the duration of abnormal utterance contained in the duration of the ith intonation change; P 21i represents the proportion of time that the words representing the characteristics of mental illness are contained in the duration of the ith intonation change, excluding the abnormal vocalization content; P 22i represents the proportion of time that phrases representing the characteristics of mental illness occupy in the duration of the i-th intonation change, excluding the abnormal vocalization content; P 23i represents the proportion of the time that the sentence structure representing the characteristics of mental illness excluding the abnormal vocalization content in the duration of the i-th intonation change; The third sound information evaluation parameter value is obtained by using the speech rate change feature in the sound data feature; wherein the third sound information evaluation parameter value is obtained by the following formula: Among them, Y 03 represents the third sound information evaluation parameter value; k 03 Indicates the number of times the speech rate changes in the sound data feature; T 03i represents the duration of the i-th speech rate change; P xi represents the total duration ratio of abnormal utterance and intonation change included in the duration of the ith speech rate change; The first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value are used to obtain the patient's sound information evaluation parameter; wherein the sound information evaluation parameter is obtained by the following formula: Where Y represents the sound information evaluation parameter; α 01 , α 02 and α 03 Respectively represent the weight values ​​corresponding to the first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value; Furthermore, obtaining the video information evaluation parameter of the patient using the video data feature corresponding to the video data information includes: Retrieving video data features corresponding to the video data information of the patient; The first video information evaluation parameter value is obtained by using the abnormal facial expression feature in the video data feature; wherein the first video information evaluation parameter value is obtained by the following formula: Among them, S 01 represents the first video information evaluation parameter value; r 01 Indicates the total number of abnormal facial expressions; T y01i represents the duration of the ith abnormal facial expression; T z Indicates the total duration of the video data information; T x11i and T x12i They represent the duration of abnormal vocalization and tone change in the ith abnormal facial expression; T c01i represents the overlapping duration of abnormal vocalization and intonation change in the duration of the i-th abnormal facial expression; The second video information evaluation parameter value is obtained by using the abnormal body movement feature in the video data feature; wherein the second video information evaluation parameter value is obtained by the following formula: Among them, S 02 represents the second video information evaluation parameter value; r 02 Indicates the total number of abnormal limb movements; T y02i represents the duration of the ith abnormal limb movement; P yi It indicates the proportion of time that abnormal facial expressions are included in the duration of the i-th abnormal limb movement; The video information evaluation parameter of the patient is obtained by using the first video information evaluation parameter value and the second video information evaluation parameter value; wherein the video information evaluation parameter is obtained by the following formula: Among them, S represents the video information evaluation parameter; β 01 and β 02 Respectively represent the weight values ​​corresponding to the first video information evaluation parameter value and the second video information evaluation parameter value; The comprehensive evaluation index parameters of the patient are obtained by using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information, including: Extracting text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters of the patient; The comprehensive model is used to obtain the comprehensive evaluation index parameters of the patient through text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters, wherein the comprehensive model structure is as follows: Among them, Q represents the comprehensive evaluation index parameter of the patient; W represents the text information evaluation parameter; Y represents the sound information evaluation parameter; S represents the video information evaluation parameter.

2. The multidimensional data screening method for predicting mental illness according to claim 1, characterized in that: The text data information, the sound data information and the video data information are respectively screened to obtain key data information corresponding to the text data information, the sound data information and the video data information, including: Preprocessing the text data information, screening the preprocessed text data information, and obtaining key text information in the text data information corresponding to the patient; Preprocessing the sound data information, screening the preprocessed sound data information, and obtaining key sound information in the sound data information corresponding to the patient object; The video data information is preprocessed, and the preprocessed video data information is screened to obtain key image information in the video data information corresponding to the patient.

3. The multidimensional data screening method for predicting mental illness according to claim 2, characterized in that: Perform data preprocessing on the text data information, screen the preprocessed text data information, and obtain key text information in the text data information corresponding to the patient, including: Performing text-clearing processing on the text data information, removing irrelevant characters, irrelevant punctuation marks and stop words in the text data information, and obtaining text data information after text-clearing processing corresponding to the text data information; Performing word segmentation processing on the text data information after the text legibility processing to obtain the words, phrases and sentences corresponding to the text data information after the text legibility processing; Performing part-of-speech tagging on the words, phrases and sentences corresponding to the text data information, and converting the words, phrases and sentences corresponding to the text data information into word embedding vectors; A convolutional neural network model is used to extract text features from the word embedding vector corresponding to the text data information to obtain text feature information contained in the text data information, wherein the text feature information includes words, phrases and sentence structures used to characterize characteristics of mental illness; and the words, phrases and sentence structures used to characterize characteristics of mental illness are the key text information in the text data information corresponding to the patient.

4. The multidimensional data screening method for predicting mental illness according to claim 2, characterized in that: Preprocess the sound data information, screen the preprocessed sound data information, and obtain key sound information in the sound data information corresponding to the patient object, including: Performing sound signal cleaning processing on the sound data information to eliminate background noise and electromagnetic interference signals in the sound data information, and obtaining the sound data information after the sound signal cleaning processing; Segmenting the sound data information after the sound signal cleaning process according to preset time intervals to obtain segmented sound data information; Performing standardization processing on the segmented sound data information to obtain the standardized sound data information; The standardized sound data information is input into a convolutional neural network model to obtain sound data features corresponding to the sound data information, wherein the sound data features include abnormal vocalization features, intonation change features, and speech speed change features.

5. The multidimensional data screening method for predicting mental illness according to claim 2, characterized in that: Perform data preprocessing on the video data information, screen the preprocessed video data information, and obtain key image information in the video data information corresponding to the patient, including: Performing frame processing on the video data information to obtain a static image of each frame corresponding to the video data information; Performing image quality optimization processing on the static image of each frame to obtain the static image after optimization processing; wherein the image quality optimization processing includes image denoising processing and contrast enhancement processing; The optimized static image is input into a convolutional neural network model to obtain video data features corresponding to the video data information, wherein the video data features include abnormal facial expression features and abnormal limb movement features.

6. A multidimensional data screening system for predicting mental illness, characterized in that: The multidimensional data screening system applied to mental illness prediction comprises: A data information extraction module is used to extract text data information, sound data information and video data information during the patient consultation process from the medical system; An evaluation parameter acquisition module, used to screen the text data information, the sound data information and the video data information respectively, and acquire key data information corresponding to the text data information, the sound data information and the video data information and their corresponding information evaluation parameters; A comprehensive evaluation index parameter acquisition module is used to obtain the comprehensive evaluation index parameters of the patient using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information; The information evaluation parameters corresponding to the key data information corresponding to the text data information, the sound data information and the video data information include the following steps: Obtaining text information evaluation parameters of the patient using key text information in the text data information corresponding to the patient; Acquiring a sound information evaluation parameter of the patient using a sound data feature corresponding to the sound data information; Obtaining video information evaluation parameters of the patient using video data features corresponding to the video data information; Wherein, obtaining the text information evaluation parameters of the patient using the key text information in the text data information corresponding to the patient includes: Performing feature classification on the text feature information to obtain a vocabulary information set, a phrase information set, and a sentence structure information set; The lexical feature information contained in the lexical information set is used to obtain a first text information evaluation parameter value corresponding to the lexical feature information; wherein the first text information evaluation parameter value is obtained by the following formula: Among them, W 01 represents the first text information evaluation parameter value; x01 represents the first parameter coefficient; w x02 represents the second parameter coefficient; n represents the number of words contained in the vocabulary information set; G represents the preset reference word distance; L i Indicates the starting position of the i-th word; L i+1 Indicates the starting position of the i+1th word; Z i represents the number of words contained in the i-th word; D represents the number of words in the sentence where the i-th word is located; The phrase feature information included in the phrase information set is used to obtain a second text information evaluation parameter value corresponding to the phrase feature information; wherein the second text information evaluation parameter value is obtained by the following formula: Among them, W 02 represents the second text information evaluation parameter value; x03 represents the third parameter coefficient; w x04 represents the fourth parameter coefficient; m represents the number of phrases contained in the phrase information set; Z di Indicates the number of words contained in the i-th phrase; Z ci Indicates the number of words contained in the i-th phrase; D d Indicates the number of words in the sentence where the i-th phrase is located; J i Indicates the starting position of the i-th phrase; J i+1 Indicates the starting position of the i+1th phrase; The structural feature information contained in the sentence structure information set is used to obtain a third text information evaluation parameter value corresponding to the structural feature information; wherein the third text information evaluation parameter value is obtained by the following formula: Among them, W 03 Indicates the third text information evaluation parameter value; N y N represents the number of sentences in the sentence structure information set that meet the sentence structure characteristics of mental illness; z represents the total number of sentences contained in the text data information; P represents the total proportion of words representing the characteristics of mental illness in the case where the sentences satisfy the sentence structure representing the characteristics of mental illness; The patient's text information evaluation parameter is obtained by using the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value; wherein the text information evaluation parameter is obtained by the following formula: Where W represents the text information evaluation parameter; λ 01 , 02 and λ 03 Respectively represent the weight values ​​corresponding to the first text information evaluation parameter value, the second text information evaluation parameter value and the third text information evaluation parameter value; At the same time, obtaining the patient's voice information evaluation parameters using the voice data features corresponding to the voice data information includes: Retrieving the sound data features corresponding to the sound data information of the patient; The first sound information evaluation parameter value is obtained by using the abnormal sound feature in the sound data feature; wherein the first sound information evaluation parameter value is obtained by the following formula: Among them, Y 01 represents the first sound information evaluation parameter value; k 01 Indicates the number of abnormal sounds in the sound data feature; T 01i represents the duration of the ith abnormal sound; T represents the total duration of the sound data; P 11i represents the proportion of time that the vocabulary representing the characteristics of mental illness contains in the duration of the ith abnormal voice; P 12i represents the proportion of time that phrases representing characteristics of mental illness are contained in the duration of the ith abnormal vocalization; P 13i represents the proportion of the time that the sentence structure representing the characteristics of mental illness is contained in the duration of the i-th abnormal voice; The second sound information evaluation parameter value is obtained by using the intonation change feature in the sound data feature; wherein the second sound information evaluation parameter value is obtained by the following formula: Among them, Y 02 represents the second sound information evaluation parameter value; k 02 Indicates the number of intonation changes in the sound data feature; T 02i represents the duration of the i-th intonation change; T xi represents the duration of abnormal utterance contained in the duration of the ith intonation change; P 21i represents the proportion of time that the words representing the characteristics of mental illness are contained in the duration of the ith intonation change, excluding the abnormal vocalization content; P 22i represents the proportion of time that phrases representing the characteristics of mental illness occupy in the duration of the i-th intonation change, excluding the abnormal vocalization content; P 23i represents the proportion of the time that the sentence structure representing the characteristics of mental illness excluding the abnormal vocalization content in the duration of the i-th intonation change; The third sound information evaluation parameter value is obtained by using the speech rate change feature in the sound data feature; wherein the third sound information evaluation parameter value is obtained by the following formula: Among them, Y 03 represents the third sound information evaluation parameter value; k 03 Indicates the number of times the speech rate changes in the sound data feature; T 03i represents the duration of the i-th speech rate change; P xi represents the total duration ratio of abnormal utterance and intonation change included in the duration of the ith speech rate change; The first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value are used to obtain the patient's sound information evaluation parameter; wherein the sound information evaluation parameter is obtained by the following formula: Where Y represents the sound information evaluation parameter; α 01 , α 02 and α 03 Respectively represent the weight values ​​corresponding to the first sound information evaluation parameter value, the second sound information evaluation parameter value and the third sound information evaluation parameter value; Furthermore, obtaining the video information evaluation parameter of the patient using the video data feature corresponding to the video data information includes: Retrieving video data features corresponding to the video data information of the patient; The first video information evaluation parameter value is obtained by using the abnormal facial expression feature in the video data feature; wherein the first video information evaluation parameter value is obtained by the following formula: Among them, S 01 represents the first video information evaluation parameter value; r 01 Indicates the total number of abnormal facial expressions; T y01i represents the duration of the ith abnormal facial expression; T z Indicates the total duration of the video data information; T x11i and T x12i They represent the duration of abnormal vocalization and tone change in the ith abnormal facial expression; T c01i represents the overlapping duration of abnormal vocalization and intonation change in the duration of the i-th abnormal facial expression; The second video information evaluation parameter value is obtained by using the abnormal body movement feature in the video data feature; wherein the second video information evaluation parameter value is obtained by the following formula: Among them, S 02 represents the second video information evaluation parameter value; r 02 Indicates the total number of abnormal limb movements; T y02i represents the duration of the ith abnormal limb movement; P yi It indicates the proportion of time that abnormal facial expressions are included in the duration of the i-th abnormal limb movement; The video information evaluation parameter of the patient is obtained by using the first video information evaluation parameter value and the second video information evaluation parameter value; wherein the video information evaluation parameter is obtained by the following formula: Among them, S represents the video information evaluation parameter; β 01 and β 02 Respectively represent the weight values ​​corresponding to the first video information evaluation parameter value and the second video information evaluation parameter value; The comprehensive evaluation index parameters of the patient are obtained by using the information evaluation parameters corresponding to the text data information, the sound data information and the video data information, including: Extracting text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters of the patient; The comprehensive model is used to obtain the comprehensive evaluation index parameters of the patient through text information evaluation parameters, sound information evaluation parameters and video information evaluation parameters, wherein the comprehensive model structure is as follows: Among them, Q represents the comprehensive evaluation index parameter of the patient; W represents the text information evaluation parameter; Y represents the sound information evaluation parameter; S represents the video information evaluation parameter.

Images