Artificial intelligence-based psychological disease triage method and system

Abstract

The invention discloses an artificial intelligence psychological disease triage method and system, and the method comprises the steps: collecting original data, including texts, voices and videos, of a target object; performing triage on the original data based on a pre-trained psychological disease triage model to obtain an emotional state of the target object; wherein the training process of the psychological disease triage model is as follows: denoising collected training data to remove abnormal fluctuation caused by target object activity or external interference; in the de-noised data, combining key emotion biomarkers and carrying out multi-modal mapping on the text, the voice and the video to form high-order emotion features; and classifying the extracted emotion feature data through a deep learning model based on a cloud AI algorithm, and classifying the extracted emotion feature data into joy, satisfaction, anxiety or sadness emotion categories.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical triage, and specifically relates to an artificial intelligence mental illness triage method and system. Background Art

[0002] Social progress and development have accelerated people's pace of life and intensified social competition. These changes will inevitably have a significant impact on individual mental health. In this era, it is particularly important to conduct efficient and accurate assessment of mental health, which is the premise for understanding the mental health status of the people and providing effective intervention. In recent years, artificial intelligence and big data mining technologies such as machine learning and deep learning have gradually been applied to the field of mental health, bringing about innovations in mental health assessment methods and giving rise to the emerging field of "intelligent mental health assessment". Artificial intelligence is a science that studies and develops methods, technologies and application systems for simulating and extending human intelligence. Machine learning is the most important technical means of artificial intelligence, which aims to explore and model complex high-dimensional interactions between a large number of variables. By applying artificial intelligence technology to obtain and analyze data, and using machine learning methods to characterize and model the relationship between features and psychological states, intelligent mental health assessment can assist and, to a certain extent, replace manual assessment. At the same time, intelligent mental health assessment has greatly broadened the assessment forms and data analysis methods of traditional mental health assessment methods (such as scale method, interview method, etc.), enabling researchers to obtain multimodal data based on more simulated task scenarios for collaborative analysis and modeling, and achieve more efficient and accurate assessment.

[0003] Patent CN111653361A discloses a psychological examination system for employees based on speech emotion recognition, including a psychological examination system and a medical system. The psychological examination system includes a target object layer, a core examination business layer, a nursing basic business layer, and a nursing data resource layer. The psychological examination system collects information through speech recognition, collects, stores, transmits and processes information, and performs psychological examinations and health management by collecting information from users. The medical system includes a medical target object layer, a medical core business layer, a medical basic service layer, and a medical data resource layer. The core examination business layer of the invention includes a psychological examination, through which psychological diagnosis of employees is performed, and the mental health management provided can manage patients and help patients recover better psychologically. The speech recognition system is used to recognize the patient's speech and determine the patient's psychological problems through the patient's speech.

[0004] Patent CN 112117002A discloses a new intelligent psychological assessment intervention system and method combined with virtual reality technology. The new intelligent psychological assessment intervention system combined with virtual reality technology includes: psychological data acquisition module, main control module, psychological data processing module, VR scene selection module, psychological test module, assessment model construction module, psychological state assessment module, assessment report generation module, psychological treatment module, cloud storage module, and update display module. The invention associates the abnormal psychological state determined based on physiological signals with the abnormal risk factors determined based on the scale through the psychological data processing module, provides objective data basis for the target object to adjust psychologically, and assists the target object to improve its psychological state; at the same time, the psychological state assessment module eliminates the influence of individual subjective factors on data collection, which helps to more accurately assess the mental health state.

[0005] Patent CN111784134A discloses that the present invention provides a data-driven multimodal academic level prediction method, including taking the student-related information collected in the early stage as a training set for quantitative input, and using the regression analysis method to analyze and calculate the influence of various factors on the final academic level, and obtain the factor correlation coefficient; according to the factor correlation coefficient, adjust the data in proportion, and use the algorithm model of the support vector machine to predict the grade; compare the predicted grade with the actual grade, adjust the algorithm model of the support vector machine and adjust the factor correlation coefficient until the error between the coupled predicted grade and the actual grade is less than or equal to the preset value; obtain the student-related information of the student to be tested containing various factors, and obtain the predicted grade of the student to be tested according to the final factor correlation coefficient and the algorithm model of the support vector machine. The implementation of this invention can solve the problems existing in the existing support vector machine prediction, reduce the subsequent work machine memory and operation time, and ensure the accuracy of prediction.

[0006] However, at present, intelligent mental health assessment can only be used for rough screening and cannot be used directly for diagnosis. However, strengthening the refinement and targeting of the assessment will help improve the accuracy of the assessment, while providing more abundant information to help doctors conduct further clinical evaluation and diagnosis. Summary of the invention

[0007] In view of the above-mentioned prior art, the purpose of the present invention is to provide a more stable artificial intelligence mental illness triage method and system. The main process is to obtain the voice, text and video data of the target object, and then obtain the text content of the target object's speech through voice recognition and text emotion recognition, and then perform multimodal mapping of the text content, voice and video to form high-order emotional features to build a mental illness triage model, and then identify the psychological state of the target object through the mental illness triage model. Compared with the model that directly recognizes emotions through voice, the present invention is more mature and practical, and eliminates the problem of the difficulty of few voice annotation tools. Compared with directly identifying psychological states through text, the present invention is more reliable and more explanatory. This method of inferring the psychological state of the target object by the changes in the target object's emotions over a period of time is more acceptable to the public. The identified psychological state can ultimately enable AI to conduct more professional psychological counseling or issue early warnings to the target object, etc., In a first aspect, the present invention provides an artificial intelligence method for triaging mental illness, comprising collecting raw data of a target subject, including text, voice, and video; The original data is triaged based on the pre-trained mental illness triage model to obtain the emotional state of the target object; wherein the training process of the mental illness triage model is as follows: De-noising the collected training data to remove abnormal fluctuations caused by target object activities or external interference; In the denoised data, key emotion biomarkers are combined and multimodal mapping of the text, speech and video is performed to form high-order emotion features; The cloud-based AI algorithm classifies the extracted emotional feature data through a deep learning model and categorizes it into emotion categories such as joy, satisfaction, worry, or sadness.

[0008] Preferably, in the denoised data, key emotion biomarkers are combined and the text, voice and video are multimodally mapped to form high-order emotion features, including: Based on automatic speech recognition technology, a modality adaptation unit is constructed for speech recognition tasks to achieve feature representation of both speech and text modalities; Construct a fine-tuning instruction set for cross-modal videos and fine-tune the fine-tuning instruction set; Modality mapping is done based on pre-trained encoders, large language models, and fine-tuning instruction sets.

[0009] Preferably, the mapping of the modalities further includes: pre-training a universal prediction module based on text data; freezing the pre-trained universal prediction module, and training an embedder, wherein the universal prediction module and the embedder are partial modules of an encoder for multimodal alignment.

[0010] Preferably, the modality mapping model structure is constructed as an embedder , pre-trained encoder , predictor and the parameters of the entire model are expressed as ; where e represents the embedded function, Indicates that this function can accept any input, represents the parameters of the embedding function, which are learned during the training process and are used to adjust the embedding vector to best represent the input data, f represents the encoder, represents the parameters of the encoder, h represents the encoder, Represents the parameters of the encoder; Constructing a model of the target modality ;in, Indicates that the parameter is Model is a parameter vector, given by , , It consists of three parts, , , Corresponding to the parameters of the embedder, encoder and predictor respectively; The mapping model between the source mode and the target mode is obtained as ; in, represents the optimal set of parameters found during the optimization process, which minimizes the loss function; Indicates finding the parameter that minimizes the following expression The value of; It means to sum all samples. is the total number of samples; represents the loss function, which measures the model The predicted value for the i-th sample With actual value The difference between.

[0011] Preferably, the raw data of the target object collected includes: Obtain psychological emotional representation data of target subjects based on emotional value and activation level; According to the psychological emotion representation data of the target object based on the emotional value and activation level, the pre-set psychological emotion standard data of the target object based on the emotional value and activation level are indexed; The indexed target object psychological emotion standard data based on the emotional value and the activation level is obtained, and the target object psychological emotion standard data based on the emotional value and the activation level is retrieved.

[0012] Preferably, the AI ​​algorithm is one of decision tree, logistic regression, support vector machine, and recurrent neural network. The algorithm adopts an adaptive learning mechanism, that is, the model is continuously updated according to the feedback of the target object, so that the emotion recognition ability of the system can dynamically adapt to individual differences.

[0013] In a second aspect, the present invention also provides an artificial intelligence mental illness triage system, comprising Acquisition module: used to collect the original data of the target object, including text, voice and video; Model training module: triage the original data based on the pre-trained mental illness triage model to obtain the emotional state of the target object; wherein the model training module includes: Denoising unit: used to denoise the collected training data and remove abnormal fluctuations caused by target object activities or external interference; Modality mapping unit: used to combine key emotion biomarkers in the denoised data and perform multimodal mapping of the text, voice and video to form high-order emotion features; Classification unit: used for cloud-based AI algorithms to classify the extracted emotional feature data through deep learning models and classify them into joy, satisfaction, worry or sadness emotion categories.

[0014] In a third aspect, the present invention further provides an artificial intelligence mental illness triage device, comprising: Memory for storing computer programs; A processor is used to implement the steps of the artificial intelligence mental illness triage method as described in the first aspect when executing the computer program.

[0015] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the artificial intelligence mental illness triage method as in the first aspect are implemented.

[0016] 1. The present invention can collect chat content data of a single target object through the dialogue system to establish a psychological state tendency model of the target object, so as to achieve timely diagnosis, early treatment, and behavior prediction, and reduce the occurrence of tragic events such as suicide and endangerment to social public security. It can assist hospitals and public security departments in preventing disasters before they happen, and enhance social harmony and stability, and personal family stability and happiness.

[0017] 2. In the present invention, in order to facilitate the training of the psychological state recognition model, we only need to predict the psychological state of the target object according to the changes in the emotional state of the target object. Here, we do not use the direct text to predict the psychological state, mainly because the psychological state of the target object cannot be simply predicted by a sentence. In order to more accurately predict the psychological state of the target object, we decided to use the emotions shown by the target object to infer the psychological state of the target object. In addition, we do not need to annotate the psychological state of the speech here, because the labor cost of speech annotation is too high, and there is no better speech annotation tool at present.

[0018] 3. The present invention gives the recognition result of the psychological state according to the speaker's emotional fluctuations, making the result of the psychological state analysis more interpretable. The present invention is highly practical and operable. The data used to train the psychological recognition model only needs to be labeled with the psychological category to which it belongs. The present invention has a wide range of applications and can be used for psychological state recognition in scenarios such as intelligent psychological customer service and psychological counseling robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 Flowchart provided for this application; Figure 2 This is the system structure diagram provided for this application. DETAILED DESCRIPTION

[0021] As required, detailed embodiments of the present invention are disclosed herein, but it should be understood that the disclosed embodiments are merely exemplary of the present invention, and the present invention may be implemented in different and alternative forms. The accompanying drawings may not be drawn to scale, and certain features may be exaggerated or reduced to show the details of specific components. Therefore, the specific structural and functional details disclosed herein should not be understood as having a limiting meaning, but only as a representative basis to teach those skilled in the art to adopt the present invention differently. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0022] Example:

[0023] like Figure 1 As shown, this embodiment provides an artificial intelligence mental illness triage method, including: S10, collecting original data of the target object, including text, voice and video; Obtain target object psychological emotion representation data based on emotional value and activation level; index pre-set target object psychological emotion standard data based on emotional value and activation level according to target object psychological emotion representation data based on emotional value and activation level; obtain the indexed target object psychological emotion standard data based on emotional value and activation level, and retrieve the target object psychological emotion standard data based on emotional value and activation level.

[0024] It should be noted that the target object is asked questions related to the screening of psychological problems based on emotional value and activation level. Before the target object answers, the high-definition camera set at the front end of the target object obtains the face and neck muscle group images of the target object in real time when being asked questions. At the same time, the text content and voice content of the answer are recorded, which are used as the psychological and emotional representation data of the target object. Then, based on the pre-set target object psychological and emotional standard data, a preliminary judgment is made on which category of direct representation data the target object's psychological emotions belong to, and it is recorded in the target object's original data, including text, voice and video. Then, the next step of the mental illness triage model is used for further analysis.

[0025] S20, triage the original data based on the pre-trained mental illness triage model to obtain the emotional state of the target object; wherein the training process of the mental illness triage model is as follows: S21, denoising the collected training data to remove abnormal fluctuations caused by target object activities or external interference; Mainly adopt such as 1) Kalman filter: This is a recursive algorithm used to estimate the state of a process from noisy measurements. It consists of two main steps, prediction and update, and is suitable for state estimation of linear dynamic systems. 2) Sliding average filter: This is a basic time series analysis technique that smoothes data by calculating the average value of a local set of data points to reduce random fluctuations and noise. 3) Outlier removal filter: A filtering technique based on statistical methods that reduces data errors and improves the accuracy of analysis by identifying and removing outliers or noise. 4) Median filter: This is a nonlinear smoothing filter that is particularly suitable for removing salt and pepper noise. It eliminates isolated noise points by replacing the value of a point with the median of the values ​​of each point in a field. This type of existing denoising processing method removes abnormal fluctuations caused by the activities of the target object or external interference, and avoids affecting the objective judgment of the target object by the psychological disease triage model.

[0026] S22, in the denoised data, combining key emotion biomarkers and performing multimodal mapping of the text, speech and video to form high-order emotion features; In this step, the key emotion biomarkers are first combined, that is, the direct representation data of which category the psychological emotion of the target object belongs to is preliminarily judged in step S10, and the emotion biomarker combination is used as the basis for subsequent judgment. When the subsequent judgment result is inconsistent with the preliminary judgment result, it is necessary to re-judge once to avoid misdiagnosis. Then, text, voice and video are multimodally mapped to ensure that the emotional characteristics reflected in the three modalities of text, voice and video are consistent.

[0027] In modality mapping alignment, the main idea is to align text and speech first, and then align video with text and speech. The specific steps are as follows: text data is used as the source modality and speech data is used as the target modality. Pre-train the predictor based on text data because this data is richer than other modalities. The modality mapping model structure is constructed as an embedder. , pre-trained encoder , predictor and the parameters of the entire model are expressed as ; where e represents the embedded function, Indicates that this function can accept any input, represents the parameters of the embedding function, which are learned during the training process and are used to adjust the embedding vector to best represent the input data, f represents the encoder, represents the parameters of the encoder, h represents the encoder, Represents the parameters of the encoder; The embedder maps the input data to a shared input embedding space, and the encoder extracts features from the embedded input. The predictor is a linear layer that maps the encoder output to the label space. ;in, Indicates that the parameter is Model is a parameter vector, given by , , It consists of three parts, , , Corresponding to the parameters of the embedder, encoder, and predictor, respectively; both the embedder and the predictor are specifically redesigned to fit the input and label space of the target task. The flexibility of this architecture enables end-to-end training on the target task, simply by fine-tuning all parameters of the target model by minimizing the task-specific loss on a given training dataset: The mapping model between the source mode and the target mode is obtained as ; in, represents the optimal set of parameters found during the optimization process, which minimizes the loss function; Indicates finding the parameter that minimizes the following expression The value of; It means to sum all samples. is the total number of samples; represents the loss function, which measures the model The predicted value for the i-th sample With actual value The difference between.

[0028] By learning directly from the target supervision in this way, the model is encouraged to learn knowledge that helps distinguish the target data. Since the pre-trained model already contains discriminative knowledge from the source domain, cross-modal transfer naturally expects that the knowledge in the source and target domains is similar in some ways, so that the knowledge in the source domain can be reused to promote target learning. We then conduct experiments to show that this similarity depends on the modality, providing high-order sentiment features and formalizing knowledge differences.

[0029] S23. The cloud-based AI algorithm classifies the extracted emotional feature data through a deep learning model and classifies it into joy, satisfaction, worry, or sadness emotion categories.

[0030] The role of the artificial intelligence (AI) algorithm is not to enhance the representation, but to project new data only into this shared space. Using an encoder represents a compromise between alignment performance and complexity, as it minimizes the number of parameters that need to be tuned. This approach allows the integration of new modalities at low cost and ensures good performance even in the absence of large aligned datasets.

[0031] Compared with traditional triage systems based on a single modality (usually text), this mental illness triage model significantly improves the comprehensiveness and accuracy of data processing by integrating multiple data types (such as text, images, and voice). This multimodal processing capability enables the medical triage system to understand the patient's condition more comprehensively and make more accurate triage decisions.

[0032] In addition, the self-learning ability of the mental illness triage model is also a major improvement. Through deep learning and data analysis, the medical triage system can continuously learn from new medical cases to optimize and adjust its triage algorithm. This ability enables the medical triage system to adapt to the ever-changing medical environment and case types, maintaining its long-term effectiveness and accuracy.

[0033] The cloud computing platform includes high-speed data transmission, big data storage, and data encryption and protection. Among them, high-speed data transmission ensures that the data uploaded from the terminal can be quickly and securely transmitted to the cloud. Big data storage is used to provide sufficient storage space and processing power to cope with a large amount of concurrent case data. Data encryption and protection are used to ensure patient privacy and ensure that data is fully encrypted during transmission and storage. High-speed data transmission uses a dedicated protocol for real-time data transmission to improve data processing efficiency. Big data storage uses a distributed file system to store large amounts of data. Data encryption and protection uses public and private keys to encrypt and decrypt data. The cloud computing platform also includes a distributed computing module and a data management module. The distributed computing module is used to split multiple patient data into several sub-data for calculation and processing. The data management module is used to manage the configuration and monitoring of patient data resources and tasks.

[0034] The training algorithm of the multimodal large language model adopts database-enhanced human feedback reinforcement learning training, and assists the reasoning of the multimodal large language model with accurate knowledge in the medical knowledge database; each trajectory of the database-enhanced human feedback reinforcement learning training will include the query object output in the database by the multimodal large language model according to the analysis result of the multimodal input, the return result of the database is integrated into the input of the multimodal large language model to obtain the medical result, and uniformly fine-tuned in the reinforcement learning stage through the idea of ​​hierarchical reinforcement learning; the return result of the database is integrated with the analysis result of the multimodal input through a preset prompt format.

[0035] It can automatically interpret medical records and patient descriptions into data that can be recognized by the large language model. The data analysis module uses the large language model to process and analyze the data, and then uses the self-learning ability of the data optimization module to improve the performance of the large language model. Finally, the results are fed back to the doctor through the target object interaction module. Compared with the human experience and judgment of the existing system, the present invention uses a large language model to improve the automation and intelligence level of the system, and quickly provide triage suggestions, which greatly improves work efficiency; and the present invention reduces dependence on personal experience. The present invention helps to improve the overall quality of medical services, reduce patient waiting time, and thus improve patient experience.

[0036] Specifically, the machine learning and big data analysis module includes feature extraction, training model, real-time analysis, evaluation module and recording module. Feature extraction is used to extract key features from data obtained from biosensor units and speech recognition systems. The training model is used to train models from known mental illness cases. Real-time analysis is used to analyze patients' real-time data. The evaluation module is used to perform instant mental state assessment. The recording module is used to record patient data and mental state.

[0037] like Figure 2As shown, an artificial intelligence mental illness triage device includes: A memory 30, for storing computer programs; The processor 31 is used to implement the steps of the artificial intelligence mental illness triage method mentioned in the above embodiment when executing the computer program.

[0038] The artificial intelligence mental illness triage device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0039] Among them, the processor 31 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 31 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 31 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 31 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 31 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0040] The memory 30 may include one or more computer-readable storage media, which may be non-transitory. The memory 30 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 30 is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor 31, the relevant steps of the artificial intelligence mental illness triage method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 30 may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to the artificial intelligence mental illness triage method, etc.

[0041] In some embodiments, the artificial intelligence mental illness triage device may also include a display screen, an input and output interface, a communication interface, a power supply, and a communication bus.

[0042] Those skilled in the art will understand that Figure 2 The structure shown in does not constitute a limitation of the artificial intelligence mental illness triage device and may include more or fewer components than shown in the figure.

[0043] The artificial intelligence mental illness triage device provided in the embodiment of the present application includes a memory 30 and a processor 31. When the processor 31 executes the program stored in the memory 30, it can implement the artificial intelligence mental illness triage method.

[0044] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0045] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory), ROM, random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0046] The above is a detailed introduction to an artificial intelligence mental illness triage method, device, equipment and medium provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0047] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. An artificial intelligence mental illness triage method, characterized by: Collect the original data of the target object, including text, voice and video; triage the raw data based on a pre-trained mental illness triage model to derive the emotional state of the target object; The training process of the mental illness triage model is as follows: De-noising the collected training data to remove abnormal fluctuations caused by target object activities or external interference; In the denoised data, key emotion biomarkers are combined and multimodal mapping of the text, speech and video is performed to form high-order emotion features; The cloud-based AI algorithm classifies the extracted emotional feature data through a deep learning model and categorizes it into emotion categories such as joy, satisfaction, worry, or sadness.

2. The artificial intelligence mental illness triage method according to claim 1, characterized in that: In the denoised data, key emotion biomarkers are combined and the text, voice and video are multimodally mapped to form high-order emotion features, including: Pre-train predictors based on text data; Based on automatic speech recognition technology, an embedder and encoder are built for speech recognition tasks to achieve feature representation of both speech and text modalities; Construct a fine-tuning instruction set for cross-modal videos and fine-tune the fine-tuning instruction set; The video is modally mapped with speech and text.

3. The artificial intelligence mental illness triage method according to claim 2, characterized in that: Further including: Constructing the modality mapping model structure as an embedder , pre-trained encoder , predictor and the parameters of the entire model are expressed as ; where e represents the embedded function, Indicates that this function can accept any input, represents the parameters of the embedding function, which are learned during the training process and are used to adjust the embedding vector to best represent the input data, f represents the encoder, represents the parameters of the encoder, h represents the encoder, Represents the parameters of the encoder; Constructing a model of the target modality ;in, Indicates that the parameter is Model is a parameter vector, given by , , It consists of three parts, , , Corresponding to the parameters of the embedder, encoder and predictor respectively; The mapping model between the source mode and the target mode is obtained as ； in, represents the optimal set of parameters found during the optimization process, which minimizes the loss function; Indicates finding the parameter that minimizes the following expression The value of; It means to sum all samples. is the total number of samples; represents the loss function, which measures the model The predicted value for the i-th sample With actual value The difference between.

4. The artificial intelligence mental illness triage method according to claim 1, characterized in that: The raw data of the target object to be collected includes: Obtain psychological emotional representation data of target subjects based on emotional value and activation level; According to the psychological emotion representation data of the target object based on the emotional value and activation level, the pre-set psychological emotion standard data of the target object based on the emotional value and activation level are indexed; The indexed target object psychological emotion standard data based on the emotional value and the activation level is obtained, and the target object psychological emotion standard data based on the emotional value and the activation level is retrieved.

5. According to the artificial intelligence mental illness triage method described in claim 1, the AI ​​algorithm is one of a decision tree, logistic regression, support vector machine, and recurrent neural network.

6. An artificial intelligence mental illness triage system, including Acquisition module: used to collect the original data of the target object, including text, voice and video; Model training module: triage the raw data based on the pre-trained mental illness triage model to obtain the emotional state of the target object; The model training module includes: Denoising unit: used to denoise the collected training data and remove abnormal fluctuations caused by target object activities or external interference; Modality mapping unit: used to combine key emotion biomarkers in the denoised data and perform multimodal mapping of the text, voice and video to form high-order emotion features; Classification unit: used for cloud-based AI algorithms to classify the extracted emotional feature data through deep learning models and classify them into joy, satisfaction, worry or sadness emotion categories.

7. An artificial intelligence mental illness triage device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the artificial intelligence mental illness triage method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the artificial intelligence mental illness triage method as described in any one of claims 1 to 5.

Images