Student mental health early warning method and system based on multi-modal data fusion

Abstract

The invention discloses a multi-modal data fusion student mental health early warning method and system. The method relates to the technical field of property detection, and comprises the following steps: collecting student psychological health data, analyzing the student psychological health data, and comprehensively analyzing, optimizing and adjusting the student psychological health early warning method. The student psychological health data is acquired and processed, the student psychological health data is analyzed, the student psychological health data image quality evaluation index and the student psychological health data fusion network volatility evaluation index are obtained, and the student psychological health early warning method is comprehensively analyzed, optimized and adjusted. The accuracy of the student psychological health early warning data quality is improved, and the problem that the accuracy of the student psychological health early warning data quality is insufficient in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of property detection technology, and in particular to a student mental health early warning method and system based on multimodal data fusion. Background Art

[0002] With the rapid development of social economy and the continuous changes in education models, students' mental health issues have become increasingly prominent. Colleges and universities and primary and secondary schools are facing huge challenges in students' mental health education. Traditional mental health assessment methods have problems such as low real-time performance and poor evaluation effect of single-modal data. Therefore, quickly and accurately assessing students' mental health status has become an important task of mental health education in colleges and universities. The development of big data and artificial intelligence technology has provided new technical means for mental health early warning. Multimodal data fusion uses multiple data sources such as psychological test data, social network data, learning behavior data, etc. for comprehensive evaluation, which improves the accuracy and real-time performance of the evaluation. For example, methods based on deep learning algorithms and ecological instantaneous assessment theory can accurately assess students' mental health status in a smart learning environment.

[0003] The existing multimodal data fusion student mental health early warning methods and systems use AI multimodal mental health risk early warning, combining audio and video acquisition with AI multimodal analysis to analyze students' emotions and psychological states; through multimodal data-driven student mental health education and monitoring, the problems of low efficiency, narrow coverage, high subjectivity in evaluation results, and difficulty in real-time monitoring in traditional mental health education and assessment methods are solved; through multimodal data-based student psychological crisis early warning, multimodal data collection is realized, such as scenario simulation test modules, questionnaire modules, physiological parameter monitoring modules, personal behavior monitoring modules, etc.

[0004] For example, the mental health warning method announced in the invention patent application with announcement number: CN110729049B includes: obtaining audio and video data of multiple students listening to classes in different subjects within a preset first time period; wherein the audio and video data of classroom listening to classes include emotional information and body movements; according to the emotional information of each student, obtaining a first curve of each student's emotion changing over time; according to the body movements of each student, obtaining a second curve of each student's body movements changing over time; according to the first curve and / or the second curve of each student, judging whether the student has a psychological abnormality; when a psychological abnormality exists, generating a first warning prompt information; and sending the first warning prompt information to the terminal.

[0005] For example, the invention patent announcement with announcement number: CN116453656B discloses a psychological health assessment and early warning system and method, including: using assessment questions to conduct psychological health assessment tests on target users to obtain multiple test results, and during the test, collecting physiological parameters of the target users for stress analysis to obtain multiple stress analysis results, and judging whether they are greater than a preset stress threshold, respectively. If the proportion of unqualified stress analysis results is greater than a preset proportion threshold, an early warning is issued. If not, an assessment mode is determined based on qualified stress analysis results, and the results are input into a corresponding psychological health assessment model to obtain psychological health assessment results. When the psychological health assessment results are unqualified, an early warning is issued.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] This method has the problem of insufficient quality of student mental health warning data. Due to the influence of electromagnetic interference and light intensity during the multimodal data collection process, the accuracy of student mental health warning data is insufficient. When processing multimodal data, the synchronization of student mental health warning data is delayed, and the quality accuracy of student mental health warning data is insufficient. Summary of the invention

[0008] The embodiments of the present application solve the problem of insufficient quality accuracy of student mental health warning data in the prior art by providing a student mental health warning method and system with multimodal data fusion, thereby improving the quality accuracy of student mental health warning data.

[0009] The embodiment of the present application provides a student mental health early warning method of multimodal data fusion, including the following steps: collecting and processing student mental health data; analyzing the student mental health data to obtain the student mental health data image quality evaluation index and the student mental health data fusion network volatility evaluation index; comprehensively analyzing to obtain the student mental health data fusion reliability evaluation index; comparing and analyzing the student mental health data image quality evaluation index with the first threshold of the student mental health data image quality evaluation index, and optimizing the student mental health data image quality method; comparing and analyzing the student mental health data fusion network volatility evaluation index with the student mental health data fusion network volatility evaluation index second threshold, and optimizing the student mental health data fusion network volatility method; comparing and analyzing the student mental health data fusion reliability evaluation index with the student mental health data fusion reliability evaluation index comprehensive threshold, and optimizing and adjusting the student mental health early warning method; the specific steps of obtaining the student mental health data image quality evaluation index are: obtaining the image frame number of the preset student mental health data image quality detection segment through a network camera; obtaining the image noise of the preset student mental health data image quality detection point through a noise testing device; obtaining the image resolution of the preset student mental health data image quality detection point through a camera; obtaining the network bandwidth of the preset student mental health data image quality detection point through a router; obtaining the preset Assume the image complexity entropy value of the student mental health data image quality detection point; obtain the electromagnetic intensity of the preset student mental health data image quality detection point through a spectrum analyzer; obtain the light intensity of the preset student mental health data image quality detection point through a light intensity meter; the student mental health data image quality data includes image frame number, image noise, image resolution, network bandwidth, image complexity entropy value, electromagnetic intensity and light intensity; obtain the student mental health data image quality evaluation index according to the student mental health data image quality data analysis; the specific steps of obtaining the student mental health data fusion network volatility evaluation index are: obtain the preset student mental health data fusion network volatility detection index through a network quality analyzer The network jitter of the point is obtained; the network bandwidth of the preset student mental health data fusion network volatility detection point is obtained through a router; the image transmission speed of the preset student mental health data fusion network volatility detection point is obtained through a network performance monitoring tool; the electromagnetic strength of the preset student mental health data fusion network volatility detection point is obtained through a spectrum analyzer; the network delay of the preset student mental health data fusion network volatility detection point is obtained through a ping tool; the student mental health data fusion network volatility data includes network jitter, network bandwidth, image transmission speed, electromagnetic strength and network delay; the student mental health data fusion network volatility evaluation index is obtained according to the student mental health data fusion network volatility data analysis;The specific steps of obtaining the student mental health data fusion reliability evaluation index through comprehensive analysis are: obtaining the image contrast of the preset student mental health data fusion reliability detection point through image processing software; obtaining the student mental health data fusion reliability evaluation index through comprehensive analysis of the student mental health data image quality evaluation index, image contrast and student mental health data fusion network volatility evaluation index. ;

[0010] Furthermore, the specific steps of collecting and processing student mental health data are: collecting student mental health raw data through cameras and sensors; cleaning and denoising the student mental health raw data to obtain student mental health data, and the student mental health data includes student mental health data image quality data and student mental health data fusion network volatility data.

[0011] Furthermore, the specific steps of the method for optimizing the image quality of student mental health data are as follows: if the image quality assessment index of student mental health data is greater than or equal to the first threshold of the image quality assessment index of student mental health data, there is no need to optimize the image quality method of student mental health data; if the image quality assessment index of student mental health data is lower than the first threshold of the image quality assessment index of student mental health data, use the enhancement tool in the image processing software to improve the contrast, brightness or color saturation of the image, use Gaussian filtering to reduce noise in the image, use a sharpening filter to enhance the edges and details of the image, and use bilinear interpolation to improve the resolution of the image.

[0012] Furthermore, the specific steps of the method for optimizing the student mental health data fusion network volatility are: if the student mental health data fusion network volatility assessment index is lower than the second threshold of the student mental health data fusion network volatility assessment index, there is no need to optimize the student mental health data fusion network volatility method; if the student mental health data fusion network volatility assessment index is greater than or equal to the second threshold of the student mental health data fusion network volatility assessment index, use traffic shaping technology to prioritize key student mental health data fusion data streams, use encryption technology to protect the security of data transmission, use load balancing technology to disperse network traffic, and use CDN to cache data on a server close to the user.

[0013] Furthermore, the specific steps of optimizing and adjusting the student mental health early warning method are: extracting the comprehensive threshold of the student mental health data fusion reliability assessment index in the multimodal data fusion student mental health database, and comparing the student mental health data fusion reliability assessment index with the comprehensive threshold of the student mental health data fusion reliability assessment index; if the student mental health data fusion reliability assessment index is greater than or equal to the comprehensive threshold of the student mental health data fusion reliability assessment index, the student mental health early warning method will not be optimized and adjusted.

[0014] Furthermore, the optimization and adjustment of the student mental health early warning method also includes: if the student mental health data fusion reliability assessment index is less than the comprehensive threshold of the student mental health data fusion reliability assessment index, using the Bayesian network to improve data fusion, sending student mental health data accuracy early warning information to relevant personnel, introducing a data redundancy mechanism through multi-source data fusion, fusing data of different modalities together in the data preprocessing stage, converting image and text data into the same feature space, and then performing joint processing.

[0015] The embodiment of the present application provides a student mental health early warning system with multimodal data fusion, including a student mental health data collection module, a student mental health data analysis module, a comprehensive analysis module and an optimization and adjustment student mental health early warning module: a student mental health data collection module: used to collect and process student mental health data; a student mental health data analysis module: used to analyze student mental health data, obtain a student mental health data image quality assessment index and obtain a student mental health data fusion network volatility assessment index; a comprehensive analysis module: used to comprehensively analyze and obtain a student mental health data fusion reliability assessment index; an optimization and adjustment student mental health early warning module: used to compare and analyze the student mental health data image quality assessment index with a first threshold of the student mental health data image quality assessment index, and optimize the student mental health data image quality method; compare and analyze the student mental health data fusion network volatility assessment index with a second threshold of the student mental health data fusion network volatility assessment index, and optimize the student mental health data fusion network volatility method; compare and analyze the student mental health data fusion reliability assessment index with a comprehensive threshold of the student mental health data fusion reliability assessment index, and optimize the student mental health early warning method.

[0016] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0017] 1. By collecting and processing students' mental health data, analyzing the students' mental health data, obtaining the student mental health data image quality assessment index and the student mental health data fusion network volatility assessment index, comprehensively analyzing and optimizing the student mental health early warning method, and improving the quality and accuracy of the student mental health early warning data.

[0018] 2. By analyzing the students' mental health data, we can obtain the student mental health data image quality evaluation index and the student mental health data fusion network volatility evaluation index, thereby ensuring that the collected data is accurate and reliable. Understanding the volatility of the data fusion network helps to optimize the data transmission process, reduce data loss and delay, and thus improve the effect of data fusion.

[0019] 3. Through comprehensive analysis and optimization of student mental health early warning methods, based on accurate data analysis, we can more effectively identify mental health risks and issue early warnings in a timely manner so that intervention measures can be taken, thereby more accurately identifying mental health risks, reducing false alarms and missed alarms, and improving the reliability of early warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flow chart of a student mental health early warning method based on multimodal data fusion provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a student mental health data fusion reliability evaluation index function provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of a student mental health early warning system based on multimodal data fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The embodiment of the present application solves the problem of insufficient quality accuracy of student mental health warning data in the prior art by providing a student mental health warning method and system with multimodal data fusion. By collecting and processing student mental health data, analyzing the student mental health data, obtaining the student mental health data image quality evaluation index and the student mental health data fusion network volatility evaluation index, the student mental health data fusion method is comprehensively analyzed and optimized to improve the quality accuracy of student mental health warning data.

[0024] The technical solution in the embodiment of the present application is to solve the above-mentioned problem of insufficient quality and accuracy of student mental health warning data. The overall idea is as follows:

[0025] By collecting and processing students' mental health data, analyzing the data, obtaining the student mental health data image quality assessment index and the student mental health data fusion network volatility assessment index, comprehensively analyzing and optimizing the student mental health early warning method, and improving the quality and accuracy of student mental health early warning data.

[0026] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0027] like Figure 1As shown, it is a flow chart of the student mental health early warning method of multimodal data fusion provided in an embodiment of the present application. The method is applied to a student mental health early warning device of multimodal data fusion, and the method includes the following steps: collecting and processing student mental health data; analyzing the student mental health data to obtain the student mental health data image quality evaluation index and the student mental health data fusion network volatility evaluation index; comprehensively analyzing to obtain the student mental health data fusion reliability evaluation index; comparing and analyzing the student mental health data image quality evaluation index with the first threshold of the student mental health data image quality evaluation index, and optimizing the student mental health data image quality method; comparing and analyzing the student mental health data fusion network volatility evaluation index with the second threshold of the student mental health data fusion network volatility evaluation index, and optimizing the student mental health data fusion network volatility method; comparing and analyzing the student mental health data fusion reliability evaluation index with the comprehensive threshold of the student mental health data fusion reliability evaluation index, and optimizing and adjusting the student mental health early warning method.

[0028] Furthermore, the specific steps of collecting and processing students' mental health data are: collecting students' original mental health data through cameras and sensors; cleaning and denoising the students' original mental health data to obtain students' mental health data, wherein the students' mental health data include students' mental health data image quality data and students' mental health data fusion network volatility data.

[0029] Furthermore, the specific steps for obtaining the student mental health data image quality evaluation index are: obtaining the number of image frames of a preset student mental health data image quality detection segment through a network camera; obtaining the image noise of a preset student mental health data image quality detection point through a noise testing device; obtaining the image resolution of a preset student mental health data image quality detection point through a camera; obtaining the network bandwidth of a preset student mental health data image quality detection point through a router; obtaining the image complexity entropy value of a preset student mental health data image quality detection point through image processing software; obtaining the electromagnetic intensity of a preset student mental health data image quality detection point through a spectrum analyzer; obtaining the light intensity of a preset student mental health data image quality detection point through a light intensity meter; the student mental health data image quality data includes the number of image frames, image noise, image resolution, network bandwidth, image complexity entropy value, electromagnetic intensity and light intensity; and obtaining the student mental health data image quality evaluation index based on the student mental health data image quality data analysis.

[0030] In this embodiment, the specific method for analyzing and obtaining the student mental health data image quality evaluation index is as follows: ; ; ; ; ; ;

[0031] The preset student mental health data image quality detection points are numbered in sequence. Indicates The number of the student mental health data image quality detection point under the student mental health data image quality detection segment, , Represents the total number of image quality detection points for student mental health data.

[0032] Divide the preset student mental health data image quality time into student mental health data image quality detection segments of the same length, Indicates the number of the image quality detection segment of the student mental health data, , Represents the total number of image quality detection segments of student mental health data.

[0033] Indicates The student mental health data image quality assessment index of the student mental health data image quality detection points.

[0034] Indicates Electromagnetic intensity correction coefficient at each student mental health data image quality detection point.

[0035] Indicates Light intensity correction coefficient at each student mental health data image quality detection point.

[0036] Indicates The number of image frames under the image quality detection segment of student mental health data.

[0037] It represents the standard value of the number of image frames, which is the preset standard value of the number of image frames obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the number of image frames under a certain student mental health data image quality detection segment and the standard value of the number of image frames, the worse the image quality of the student mental health data.

[0038] Indicates Image noise under image quality detection points of student mental health data.

[0039] Represents the image noise threshold, which is the preset image noise threshold obtained from the multimodal data fusion student mental health database. It means that if the image noise at a certain student mental health data image quality detection point is greater than the image noise threshold, the worse the student mental health data image quality is.

[0040] Indicates Image resolution at each student mental health data image quality detection point.

[0041] Represents the image resolution threshold, which is the preset image resolution threshold obtained from the multimodal data fusion student mental health database, and represents the image resolution threshold at a certain student mental health data image quality detection point.

[0042] Indicates The network bandwidth at the image quality detection point of student mental health data.

[0043] Represents the network bandwidth threshold, which is the preset network bandwidth threshold obtained from the multimodal data fusion student mental health database, and represents the network bandwidth threshold at a certain student mental health data image quality detection point.

[0044] Indicates Image complexity coefficient of student mental health data at image quality detection points.

[0045] Indicates Electromagnetic intensity at the image quality detection point of students' mental health data.

[0046] It represents the standard value of electromagnetic intensity, which is the preset standard value of electromagnetic intensity obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the electromagnetic intensity at a certain student mental health data image quality detection point and the standard value of electromagnetic intensity, the worse the quality of the student mental health data image.

[0047] Indicates Light intensity at the image quality detection point of student mental health data.

[0048] It represents the standard value of light intensity, which is the preset standard value of light intensity obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the light intensity at a certain student mental health data image quality detection point and the standard value of light intensity, the worse the quality of the student mental health data image.

[0049] Indicates Image complexity entropy value of student mental health data at image quality detection point.

[0050] It represents the standard value of image complexity entropy, which is the preset standard value of image complexity entropy obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the image complexity entropy value at a certain student mental health data image quality detection point and the standard value of image complexity entropy, the worse the image quality of the student mental health data.

[0051] The preset image frame number obtained from the multimodal data fusion student mental health database affects the weight factor.

[0052] The preset image noise impact weight factor is obtained from the multimodal data fusion student mental health database.

[0053] The preset image resolution impact weight factor obtained from the multimodal data fusion student mental health database.

[0054] It is the preset network bandwidth impact weight factor obtained from the multimodal data fusion student mental health database.

[0055] The preset image complexity coefficient influence weight factor obtained from the multimodal data fusion student mental health database.

[0056] It is the preset electromagnetic intensity correction factor obtained from the multimodal data fusion student mental health database.

[0057] is the preset light intensity correction factor obtained from the multimodal data fusion student mental health database.

[0058] The preset image frame number influence weight factor, the preset image noise influence weight factor, the preset image resolution influence weight factor, the preset network bandwidth influence weight factor and the preset image complexity coefficient influence weight factor respectively represent the numerical values ​​of the influence of image frame number, image noise, image resolution, network bandwidth and image complexity coefficient on the image quality assessment index of students' mental health data.

[0059] When in use, the preset image frame number influence weight factor, preset image noise influence weight factor, preset image resolution influence weight factor, preset network bandwidth influence weight factor and preset image complexity coefficient influence weight factor can be directly obtained from the multimodal data fusion student mental health database according to the image contrast value, and the corresponding relationship can be a preset mapping relationship. For example, a mapping set of image contrast and its corresponding preset image frame number influence weight factor, preset image noise influence weight factor, preset image resolution influence weight factor, preset network bandwidth influence weight factor and preset image complexity coefficient influence weight factor is constructed, and the real-time image contrast value is input into the mapping set to obtain the preset image frame number influence weight factor, preset image noise influence weight factor, preset image resolution influence weight factor, preset network bandwidth influence weight factor and preset image complexity coefficient influence weight factor, and the mapping relationship can be one-to-one or many-to-one.

[0060] The preset light intensity correction factor and the preset electromagnetic intensity correction factor respectively represent the numerical values ​​of the influence of light intensity and electromagnetic intensity on the image quality evaluation index of students' mental health data.

[0061] When in use, the preset light intensity correction factor and the preset electromagnetic intensity correction factor can be directly obtained from the multimodal data fusion student mental health database according to the ambient temperature value, and the corresponding relationship can be a preset mapping relationship. For example, a mapping set of ambient temperature and its corresponding preset light intensity correction factor and preset electromagnetic intensity correction factor is constructed, and the real-time ambient temperature value is input into the mapping set to obtain the preset light intensity correction factor and the preset electromagnetic intensity correction factor. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship.

[0062] Light intensity affects the brightness and contrast of the image. Electromagnetic intensity affects the quality of wireless network signals, thereby affecting the transmission of student mental health data image data. Image frame rate refers to the number of images displayed per second in a video or animation, which accumulates as the number of frames increases, affecting the image quality of student mental health data. Image resolution refers to the number and density of pixels in an image. The higher the image resolution, the more details it usually contains and the more susceptible it is to image noise. Image noise is more obvious in high-resolution images. Network bandwidth refers to the rate at which the network can transmit data. The higher the network bandwidth, the faster the transmission of high-resolution, high-frame-rate image data, while low-bandwidth networks lead to delays and packet loss in image transmission. Image complexity entropy is an indicator of image complexity, reflecting the randomness and uncertainty of information in the image. Image complexity entropy is related to image details, texture, and image noise. Images with high complexity entropy values ​​usually contain more information.

[0063] Electromagnetic intensity affects the quality of wireless network signals, thus affecting the transmission of image data. The image quality assessment of student mental health data depends on the image data transmitted over the network. Electromagnetic interference leads to unstable data transmission. The larger the square difference between the electromagnetic intensity and the standard value of the electromagnetic intensity, the negative correlation exists between the square difference between the electromagnetic intensity and the standard value of the electromagnetic intensity and the image quality assessment index of the student mental health data, and the lower the image quality assessment index of the student mental health data; light intensity affects the brightness and contrast of the image. The larger the square difference between the light intensity and the standard value of the light intensity, the negative correlation exists between the square difference between the light intensity and the standard value of the light intensity and the image quality assessment index of the student mental health data, and the lower the image quality assessment index of the student mental health data; the number of image frames affects the smoothness of the video or animation. The higher the number of image frames, the smoother the image sequence, which helps to improve the image quality assessment index. The number of image frames is positively correlated with the image quality assessment index of the student mental health data, and the higher the image quality assessment index of the student mental health data; Figure The higher the image noise, the lower the clarity and recognizability of the image. There is a negative correlation between image noise and the image quality assessment index of student mental health data, and the lower the image quality assessment index of student mental health data; the higher the image resolution, the more details it usually contains. There is a positive correlation between image resolution and the image quality assessment index of student mental health data, and the higher the image quality assessment index of student mental health data; the higher the network bandwidth, the faster the high-resolution and high-frame image data can be transmitted. There is a positive correlation between network bandwidth and the image quality assessment index of student mental health data, and the higher the image quality assessment index of student mental health data; images with high complexity entropy values ​​contain more information, but are also more susceptible to image noise. The larger the square of the difference between the image complexity entropy value and the standard value of the image complexity entropy value, the negative correlation between the square of the difference between the image complexity entropy value and the standard value of the image complexity entropy value and the image quality assessment index of student mental health data, and the lower the image quality assessment index of student mental health data.

[0064] Furthermore, the specific steps for obtaining the student mental health data fusion network volatility assessment index are: obtaining the network jitter of the preset student mental health data fusion network volatility detection point through a network quality analyzer; obtaining the network bandwidth of the preset student mental health data fusion network volatility detection point through a router; obtaining the image transmission speed of the preset student mental health data fusion network volatility detection point through a network performance monitoring tool; obtaining the electromagnetic strength of the preset student mental health data fusion network volatility detection point through a spectrum analyzer; obtaining the network delay of the preset student mental health data fusion network volatility detection point through a ping tool; the student mental health data fusion network volatility data includes network jitter, network bandwidth, image transmission speed, electromagnetic strength and network delay; and obtaining the student mental health data fusion network volatility assessment index based on the student mental health data fusion network volatility data analysis.

[0065] In this embodiment, the specific method for analyzing and obtaining the student mental health data fusion network volatility evaluation index is as follows: ; ; ; ; ;

[0066] The preset student mental health data fusion network volatility detection points are numbered in sequence. Indicates the number of the fluctuation detection point of the student mental health data fusion network, , Represents the total number of volatility detection points in the student mental health data fusion network.

[0067] Indicates The student mental health data fusion network volatility assessment index of the student mental health data fusion network volatility detection point.

[0068] Indicates The electromagnetic intensity correction coefficient of the student mental health data fusion network volatility detection point.

[0069] Indicates The network delay correction coefficient of the student mental health data fusion network volatility detection point.

[0070] Indicates The network jitter under the network volatility detection point of the student mental health data fusion.

[0071] It represents the network jitter threshold, which is the preset network jitter threshold obtained from the multimodal data fusion student mental health database. It means that if the network jitter at a student mental health data fusion network volatility detection point is greater than the network jitter threshold, the student mental health data fusion network volatility is greater.

[0072] Indicates The network bandwidth under the network volatility detection point of the fusion network mental health data of individual students.

[0073] It represents the network bandwidth threshold, which is the preset network bandwidth threshold obtained from the multimodal data fusion student mental health database. It means that if the network bandwidth under a student mental health data fusion network volatility detection point is larger, the student mental health data fusion network volatility is smaller.

[0074] Indicates The image transmission speed of the student mental health data fusion network under the volatility detection point.

[0075] It represents the image transmission speed threshold, which is the preset image transmission speed threshold obtained from the multimodal data fusion student mental health database. It means that if the image transmission speed at a student mental health data fusion network volatility detection point is faster, the student mental health data fusion network volatility is smaller.

[0076] Indicates The electromagnetic intensity under the fluctuation detection point of the student mental health data fusion network.

[0077] It represents the standard value of electromagnetic intensity, which is the preset standard value of electromagnetic intensity obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the electromagnetic intensity at a student mental health data fusion network volatility detection point and the standard value of electromagnetic intensity, the greater the volatility of the student mental health data fusion network.

[0078] Indicates The network delay under the network volatility detection point of the fusion of student mental health data.

[0079] It represents the standard value of network delay, which is the preset standard value of network delay obtained from the multimodal data fusion student mental health database. It means that the greater the deviation between the network delay at a student mental health data fusion network volatility detection point and the standard value of network delay, the greater the volatility of the student mental health data fusion network.

[0080] It is the preset network jitter impact weight factor obtained from the multimodal data fusion student mental health database.

[0081] It is the preset network bandwidth impact weight factor obtained from the multimodal data fusion student mental health database.

[0082] The influence weight factor of the preset image transmission speed obtained from the multimodal data fusion student mental health database.

[0083] It is the preset electromagnetic intensity correction factor obtained from the multimodal data fusion student mental health database.

[0084] The preset network delay correction factor is obtained from the multimodal data fusion student mental health database.

[0085] The preset network jitter influence weight factor, the preset network bandwidth influence weight factor and the preset image transmission speed influence weight factor respectively represent the numerical values ​​of the influence of network bandwidth, network jitter and image transmission speed on the student mental health data fusion network volatility assessment index.

[0086] When in use, the preset network jitter impact weight factor, preset network bandwidth impact weight factor and preset image transmission speed impact weight factor can be directly obtained from the multimodal data fusion student mental health database according to the value of network load balancing, and the corresponding relationship can be a pre-set mapping relationship. For example, a mapping set of network load balancing and its corresponding preset network jitter impact weight factor, preset network bandwidth impact weight factor and preset image transmission speed impact weight factor is constructed, and the real-time network load balancing value is input into the mapping set to obtain the preset network jitter impact weight factor, preset network bandwidth impact weight factor and preset image transmission speed impact weight factor, and the mapping relationship can be one-to-one or many-to-one.

[0087] The preset electromagnetic intensity correction factor and the preset network delay correction factor respectively represent the numerical values ​​of the influence of electromagnetic intensity and network delay on the student mental health data fusion network volatility assessment index.

[0088] When in use, the preset electromagnetic intensity correction factor and the preset network delay correction factor can be directly obtained from the multimodal data fusion student mental health database according to the value of the network topology, and the corresponding relationship can be a preset mapping relationship. For example, a mapping set of the network topology and its corresponding preset electromagnetic intensity correction factor and preset network delay correction factor is constructed, and the real-time network topology value is input into the mapping set to obtain the preset electromagnetic intensity correction factor and the preset network delay correction factor. The mapping relationship can be a one-to-one correspondence or a many-to-one relationship.

[0089] Electromagnetic intensity refers to the intensity of electromagnetic waves, which interferes with the student mental health data fusion network. The higher the electromagnetic intensity, the greater the network delay, the lower the network bandwidth and the increased network jitter. This is because electromagnetic interference will affect the quality of student mental health data transmission, resulting in the loss or retransmission of student mental health data, thereby affecting the performance of the student mental health data fusion network. Network delay refers to the time required for student mental health data to travel from the sender to the receiver. Network delay is affected by network bandwidth, network jitter and image transmission speed. The higher the network bandwidth, the lower the delay provided because more student mental health data can be transmitted in a short time. Network jitter will also affect the delay because jitter will cause uncertainty in the transmission time of student mental health data. Image transmission speed will also affect the delay because the faster the student mental health data transmission speed, the shorter the time it takes for the student mental health data to reach the destination.

[0090] The electromagnetic intensity interferes with the student mental health data fusion network signal, thereby affecting the stability of the student mental health data fusion network connection. The student mental health data fusion relies on wireless network transmission. Electromagnetic interference may cause instability in the data transmission of the student mental health data fusion network. The greater the square difference between the electromagnetic intensity and the standard value of the electromagnetic intensity, the positive correlation exists between the square difference between the electromagnetic intensity and the standard value of the electromagnetic intensity and the student mental health data fusion network volatility assessment index. The greater the student mental health data fusion network volatility assessment index. In the process of student mental health data fusion, the network delay increases, resulting in uncertainty in the arrival time of the student mental health data fusion data packet, thereby affecting the real-time and accuracy of the student mental health data fusion. The network delay is positively correlated with the student mental health data fusion network volatility assessment index. The greater the student mental health data fusion network volatility assessment index. Network jitter refers to the student mental health The uncertainty of health data fusion network transmission time, increased network jitter, and instability of student mental health data packet transmission time affect the real-time and accuracy of data fusion. There is a positive correlation between network jitter and student mental health data fusion network volatility assessment index, and the larger the student mental health data fusion network volatility assessment index is; network bandwidth refers to the rate at which the network can transmit data. The lower the network bandwidth, the more congestion the student mental health data transmission will cause, thereby increasing network delay and jitter, affecting the efficiency and quality of data fusion. There is a negative correlation between network bandwidth and student mental health data fusion network volatility assessment index, and the larger the student mental health data fusion network volatility assessment index is; the faster the image transmission speed, the higher the efficiency of student mental health data fusion. There is a negative correlation between image transmission speed and student mental health data fusion network volatility assessment index, and the smaller the student mental health data fusion network volatility assessment index is.

[0091] Furthermore, the specific steps of comprehensive analysis to obtain the student mental health data fusion reliability evaluation index are: obtaining the image contrast of the preset student mental health data fusion reliability detection point through image processing software; and obtaining the student mental health data fusion reliability evaluation index through comprehensive analysis of the student mental health data image quality evaluation index, image contrast and student mental health data fusion network volatility evaluation index.

[0092] In this embodiment, the specific method for analyzing and obtaining the student mental health data fusion reliability evaluation index is as follows: ; ;

[0093] The preset student mental health data fusion reliability detection points are numbered in sequence. Indicates the number of the reliability detection point of the student mental health data fusion, , Represents the total number of reliability detection points for student mental health data fusion.

[0094] Represents the reliability evaluation index of student mental health data fusion.

[0095] Indicates The student mental health data image quality assessment index of the student mental health data image quality detection points.

[0096] Indicates Image contrast at the reliability detection point of student mental health data fusion.

[0097] Represents the image contrast threshold, which is the preset image contrast threshold obtained from the multimodal data fusion student mental health database. It means that if the image contrast at a student mental health data fusion network volatility detection point is greater than the image contrast threshold, the reliability of the student mental health data fusion is greater.

[0098] Indicates The student mental health data fusion network volatility assessment index of the student mental health data fusion network volatility detection point.

[0099] It is the influencing weight factor of the preset student mental health data image quality assessment index obtained from the multimodal data fusion student mental health database.

[0100] The preset image contrast impact weight factor obtained from the multimodal data fusion student mental health database.

[0101] It is the impact weight factor of the preset student mental health data fusion network volatility assessment index obtained from the multimodal data fusion student mental health database.

[0102] The preset student mental health data image quality assessment index influencing weight factor, the preset image contrast influencing weight factor and the preset student mental health data fusion network volatility assessment index influencing weight factor respectively represent the numerical values ​​of the influence coefficients of the student mental health data image quality assessment index, image contrast and student mental health data fusion network volatility assessment index on the student mental health data fusion reliability assessment index.

[0103] When in use, the preset student mental health data image quality assessment index influence weight factor, the preset image contrast influence weight factor and the preset student mental health data fusion network volatility assessment index influence weight factor can be directly obtained from the multimodal data fusion student mental health database according to the value of the packet loss rate, and the corresponding relationship can be a pre-set mapping relationship. For example, a mapping set of the packet loss rate and its corresponding preset student mental health data image quality assessment index influence weight factor, preset image contrast influence weight factor and preset student mental health data fusion network volatility assessment index influence weight factor is constructed, and the real-time packet loss rate value is input into the mapping set to obtain the preset student mental health data image quality assessment index influence weight factor, preset image contrast influence weight factor and preset student mental health data fusion network volatility assessment index influence weight factor, and the mapping relationship can be one-to-one or many-to-one.

[0104] Table 1 is an example table of student mental health data fusion reliability evaluation index. The example parameters in Table 1 only take the parameters under one student mental health data fusion reliability detection point for example. Set to 0.4, weight factor Set to 0.3, weight factor Set to 0.3, The image contrast threshold is 2, as shown in Table 1.

[0105] Table 1 Example of reliability evaluation index of student mental health data fusion

[0106] From Table 1 and Figure 2It can be seen that the image quality evaluation index of student mental health data measures the overall quality of the image. Image contrast is an important aspect of the image quality of student mental health data, reflecting the brightness difference between different areas in the image. The higher the image contrast, the better the visual effect, the easier it is to be identified and analyzed, and the higher the image quality evaluation index of student mental health data; the image quality evaluation index of student mental health data is the visual quality of the image itself, while the network volatility evaluation index of student mental health data fusion is the influence of network volatility during data fusion. The image quality evaluation index of student mental health data is used to evaluate image data related to student mental health. Image quality affects the accuracy and reliability of data fusion. Low image contrast leads to more noise and errors in the data fusion process, thereby increasing the network volatility evaluation index of student mental health data fusion.

[0107] High-quality image data is more likely to provide accurate information. The higher the image quality assessment index of student mental health data, the higher the reliability of data fusion. It is positively correlated with the student mental health data fusion reliability assessment index, and the larger the student mental health data fusion reliability assessment index is; image contrast is a key parameter of image quality, which reflects the brightness difference between different areas in the image. The higher the image contrast, the easier it is to analyze and understand, so it can improve the accuracy of data fusion. It is positively correlated with the student mental health data fusion reliability assessment index, and the larger the student mental health data fusion reliability assessment index is; the student mental health data fusion network volatility assessment index is the influence of network volatility in the data fusion process. The larger the student mental health data fusion network volatility assessment index is, the more instability and errors there are in the data fusion process, thereby reducing the reliability of data fusion. It is negatively correlated with the student mental health data fusion reliability assessment index, and the smaller the student mental health data fusion reliability assessment index is.

[0108] like Figure 2 As shown, it is a schematic diagram of the student mental health data fusion reliability evaluation index function provided in an embodiment of the present application; x is the positive half axis of the horizontal axis, and y is the positive half axis of the vertical axis.

[0109] Curve a indicates that if the image contrast is set to a fixed value of 1, the student mental health data fusion network volatility assessment index is set to a fixed value of 1, the student mental health data image quality assessment index is x, and the student mental health data fusion reliability assessment index is y, the student mental health data fusion reliability assessment index increases with the increase of the student mental health data image quality assessment index.

[0110] Furthermore, the specific steps of optimizing the method for optimizing the image quality of student mental health data are as follows: if the image quality evaluation index of student mental health data is greater than or equal to the first threshold of the image quality evaluation index of student mental health data, there is no need to optimize the image quality method of student mental health data; if the image quality evaluation index of student mental health data is lower than the first threshold of the image quality evaluation index of student mental health data, use the enhancement tool in the image processing software to improve the contrast, brightness or color saturation of the image, use Gaussian filtering to reduce the noise in the image, use a sharpening filter to enhance the edges and details of the image, and use bilinear interpolation to improve the resolution of the image.

[0111] In this embodiment, if the student mental health data image quality assessment index is 2 at this time, the first threshold value of the student mental health data image quality assessment index is 4, and it is assumed that the contrast, brightness, and color saturation of the original image are C, L, and S, respectively. The values ​​after improving the contrast, brightness, and color saturation are respectively new contrast = C * (1 + 0.2) = 1.2C, new brightness = L * (1 + 0.1) = 1.1L, and new color saturation = S * (1 + 0.15) = 1.15S. A Gaussian filter is used to reduce noise. The effect of the Gaussian filter depends on its standard deviation. It is assumed that a Gaussian filter with a standard deviation of 1 is used. Then, a sharpening filter is used to enhance edges and details. The sharpening filter is a Laplace filter, and its effect depends on the weight of the filter. Finally, bilinear interpolation is used to improve the image resolution. Bilinear interpolation is an algorithm that performs linear interpolation in two directions to calculate the value of new pixels.

[0112] Furthermore, the specific steps of optimizing the student mental health data fusion network volatility method are as follows: if the student mental health data fusion network volatility assessment index is lower than the second threshold of the student mental health data fusion network volatility assessment index, then there is no need to optimize the student mental health data fusion network volatility method; if the student mental health data fusion network volatility assessment index is greater than or equal to the second threshold of the student mental health data fusion network volatility assessment index, then use traffic shaping technology to prioritize key student mental health data fusion data streams, use encryption technology to protect the security of data transmission, use load balancing technology to disperse network traffic, and use CDN to cache data on a server close to the user.

[0113] In this embodiment, if the student mental health data fusion network volatility assessment index is 2 at this time, the second threshold value of the student mental health data fusion network volatility assessment index is 1, assuming that the critical data flow is 10MB, the non-critical data flow is 20MB, the network bandwidth is 100Mbps, the encryption algorithm is AES-256, there are 3 load balancing servers, and there are 5 CDN nodes. First, the traffic shaping technology will be used to assign a higher priority to the critical data flow. Assuming that the priority of the critical data flow is set to 80% and the priority of the non-critical data flow is set to 20%, the critical data flow will occupy 80Mbps of bandwidth and the non-critical data flow will occupy 20Mbps of bandwidth. The AES-256 encryption algorithm is used to encrypt the transmitted data. Assuming that the encrypted data size is 1.1 times the original data size, the encrypted data size of the critical data flow is 11MB, and the encrypted data size of the non-critical data flow is 22MB. Then, load balancing technology is used to disperse the network traffic to 3 servers. Assuming that each server can handle 30Mbps of traffic, each server will process 10Mbps of critical data flow and 10Mbps of non-critical data flow. Finally, CDN will be used to cache data on servers close to users. Assuming that each CDN node can cache 5MB of data, then 5 CDN nodes can cache a total of 25MB of data. Through the above measures, the volatility of the student mental health data fusion network can be optimized, the stable transmission of key data streams can be ensured, and the security of data can be improved.

[0114] Furthermore, the specific steps for optimizing and adjusting the student mental health early warning method are as follows: extracting the comprehensive threshold of the student mental health data fusion reliability assessment index in the multimodal data fusion student mental health database, and comparing the student mental health data fusion reliability assessment index with the comprehensive threshold of the student mental health data fusion reliability assessment index; if the student mental health data fusion reliability assessment index is greater than or equal to the comprehensive threshold of the student mental health data fusion reliability assessment index, the student mental health early warning method will not be optimized and adjusted.

[0115] Furthermore, the optimization and adjustment of the student mental health early warning method also includes: if the student mental health data fusion reliability assessment index is less than the comprehensive threshold of the student mental health data fusion reliability assessment index, using the Bayesian network to improve data fusion, sending student mental health data accuracy early warning information to relevant personnel, introducing a data redundancy mechanism through multi-source data fusion, fusing data of different modalities together in the data preprocessing stage, converting image and text data into the same feature space, and then performing joint processing.

[0116] In this embodiment, if the student mental health data fusion reliability evaluation index is 2 at this time, and the comprehensive threshold of the student mental health data fusion reliability evaluation index is 3, a student mental health data fusion process is simulated, and the Bayesian network is used to improve the quality of data fusion, and the reliability of the data is enhanced by multi-source data fusion. In the data preprocessing stage, the image and text data are fused together and converted to the same feature space for joint processing. It is assumed that the image feature is a vector with a length of 128 and the text feature is a vector with a length of 128. Use a simple linear transformation to convert these two feature vectors to the same feature space, and then use the Bayesian network for data fusion. Use convolutional neural network to extract image features, and use word embedding technology to convert text data into vector representation. Convert image features and text features to the same feature space, use Bayesian network to fuse image features and text features, and improve data reliability through multi-source data fusion. If the reliability evaluation index after data fusion is still lower than the threshold, send student mental health data accuracy warning information to relevant personnel, indicating that the student mental health data is not accurate enough at this time. After acquiring the data, you can also use convolutional neural networks and word embedding technology to fuse the data to improve the accuracy of mental health warnings. Suppose there is a group of student photos, including students' facial expressions, which can reflect the students' emotional state. Use convolutional neural networks to extract the features of these images, identify patterns in the images, such as smiles, frowns, etc., and convert these patterns into numerical features. Collect student text data, such as diaries, social media posts, etc., which contain students' language expressions and reflect students' emotions and psychological states. Use word embedding technology such as Word2Vec to convert these texts into vector representations. These vectors can capture the semantic information in the text. Use linear transformations such as principal component analysis PCA to fuse image features and text features, and send the fused data as student psychological warning information to relevant personnel.

[0117] like Figure 3As shown, it is a structural schematic diagram of the student mental health early warning system of multimodal data fusion provided in the embodiment of the present application. The student mental health early warning system of multimodal data fusion provided in the embodiment of the present application includes: a student mental health data collection module, a student mental health data analysis module, a comprehensive analysis module and an optimization and adjustment student mental health early warning module: a student mental health data collection module: used to collect and process student mental health data; a student mental health data analysis module: used to analyze student mental health data, obtain the student mental health data image quality evaluation index and obtain the student mental health data fusion network volatility evaluation index; a comprehensive analysis module: used to comprehensively analyze and obtain the student mental health data fusion reliability evaluation index; an optimization and adjustment student mental health early warning module: used to compare and analyze the student mental health data image quality evaluation index with the first threshold of the student mental health data image quality evaluation index, and optimize the student mental health data image quality method; compare and analyze the student mental health data fusion network volatility evaluation index with the second threshold of the student mental health data fusion network volatility evaluation index, and optimize the student mental health data fusion network volatility method; compare and analyze the student mental health data fusion reliability evaluation index with the student mental health data fusion reliability evaluation index comprehensive threshold, and optimize the student mental health early warning method.

[0118] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0123] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A student mental health early warning method based on multimodal data fusion, characterized in that: The following steps are involved: Collect and process student mental health data; Analyze the students' mental health data to obtain the image quality evaluation index of the students' mental health data and the fusion network volatility evaluation index of the students' mental health data; Comprehensive analysis resulted in the reliability evaluation index of student mental health data fusion; Compare and analyze the student mental health data image quality assessment index with the first threshold of the student mental health data image quality assessment index to optimize the student mental health data image quality method; Compare and analyze the student mental health data fusion network volatility evaluation index with the second threshold of the student mental health data fusion network volatility evaluation index, and optimize the student mental health data fusion network volatility method; Compare and analyze the student mental health data fusion reliability assessment index and the comprehensive threshold of the student mental health data fusion reliability assessment index to optimize and adjust the student mental health early warning method; The specific steps of obtaining the student mental health data image quality evaluation index are as follows: Obtain the number of image frames of a preset student mental health data image quality detection segment through a network camera; The image noise of the preset student mental health data image quality detection point is obtained through the noise testing equipment; The image resolution of the preset student mental health data image quality detection point is obtained through the camera; Obtain the network bandwidth of the preset student mental health data image quality detection point through the router; The image complexity entropy value of the preset student mental health data image quality detection point is obtained through image processing software; The electromagnetic intensity of the preset student mental health data image quality detection point is obtained through a spectrum analyzer; The light intensity of the preset student mental health data image quality detection point is obtained through a light intensity meter; The student mental health data image quality data includes image frame number, image noise, image resolution, network bandwidth, image complexity entropy value, electromagnetic intensity and light intensity; According to the data analysis of student mental health data image quality, the student mental health data image quality evaluation index is obtained; The specific steps of obtaining the student mental health data fusion network volatility evaluation index are as follows: The network jitter of the preset student mental health data fusion network volatility detection point is obtained through the network quality analyzer; Obtain the network bandwidth of the preset student mental health data fusion network volatility detection point through the router; The image transmission speed of the preset student mental health data fusion network volatility detection point is obtained through the network performance monitoring tool; The electromagnetic intensity of the fluctuation detection point of the preset student mental health data fusion network is obtained through the spectrum analyzer; The network delay of the preset student mental health data fusion network fluctuation detection point is obtained through the ping tool; The student mental health data is fused with network volatility data including network jitter, network bandwidth, image transmission speed, electromagnetic intensity and network delay; According to the analysis of student mental health data fusion network volatility data, the student mental health data fusion network volatility assessment index is obtained; The specific steps of obtaining the student mental health data fusion reliability evaluation index through comprehensive analysis are as follows: The image contrast of the preset student mental health data fusion reliability detection point is obtained through image processing software; The student mental health data fusion reliability evaluation index is obtained through comprehensive analysis of the student mental health data image quality evaluation index, image contrast and student mental health data fusion network volatility evaluation index.

2. The student mental health early warning method based on multimodal data fusion as claimed in claim 1, characterized in that: The specific steps of collecting and processing students' mental health data are as follows: Collect raw data on students’ mental health through cameras and sensors; The original student mental health data is cleaned and denoised to obtain the student mental health data, which includes the student mental health data image quality data and the student mental health data fusion network volatility data.

3. The student mental health early warning method based on multimodal data fusion as claimed in claim 1, characterized in that: The specific steps of the method for optimizing the image quality of student mental health data are as follows: If the student mental health data image quality assessment index is greater than or equal to the first threshold of the student mental health data image quality assessment index, there is no need to optimize the student mental health data image quality method; If the student mental health data image quality assessment index is lower than the first threshold of the student mental health data image quality assessment index, use the enhancement tools in the image processing software to improve the contrast, brightness or color saturation of the image, use Gaussian filtering to reduce the noise in the image, use sharpening filters to enhance the edges and details of the image, and use bilinear interpolation to improve the resolution of the image.

4. The student mental health early warning method based on multimodal data fusion as claimed in claim 1, characterized in that: The specific steps of the method for optimizing student mental health data fusion network volatility are as follows: If the student mental health data fusion network volatility assessment index is lower than the second threshold of the student mental health data fusion network volatility assessment index, there is no need to optimize the student mental health data fusion network volatility method; If the student mental health data fusion network volatility assessment index is greater than or equal to the second threshold of the student mental health data fusion network volatility assessment index, traffic shaping technology is used to prioritize the key student mental health data fusion data stream, encryption technology is used to protect the security of data transmission, load balancing technology is used to disperse network traffic, and CDN is used to cache data on a server close to the user.

5. The student mental health early warning method based on multimodal data fusion as claimed in claim 1, characterized in that: The specific steps of optimizing and adjusting the student mental health early warning method are: Extract the comprehensive threshold of student mental health data fusion reliability evaluation index from the multimodal data fusion student mental health database, and compare the student mental health data fusion reliability evaluation index with the comprehensive threshold of student mental health data fusion reliability evaluation index; If the student mental health data fusion reliability assessment index is greater than or equal to the comprehensive threshold of the student mental health data fusion reliability assessment index, the student mental health early warning method will not be optimized and adjusted.

6. The student mental health early warning method based on multimodal data fusion as claimed in claim 1, characterized in that: The method for optimizing and adjusting the student's mental health early warning method also includes: If the student mental health data fusion reliability assessment index is less than the comprehensive threshold of the student mental health data fusion reliability assessment index, the Bayesian network is used to improve data fusion, and student mental health data accuracy warning information is sent to relevant personnel. The data redundancy mechanism is introduced through multi-source data fusion. Data of different modalities are fused together in the data preprocessing stage, and the image and text data are converted into the same feature space, and then jointly processed.

7. A student mental health early warning system based on multimodal data fusion, characterized in that: It includes the module for collecting students' mental health data, the module for analyzing students' mental health data, the module for comprehensive analysis and the module for optimizing and adjusting students' mental health early warning: Student mental health data collection module: used to collect and process student mental health data; Analyzing students' mental health data module: used to analyze students' mental health data, obtain the image quality evaluation index of students' mental health data and obtain the fusion network volatility evaluation index of students' mental health data; Comprehensive analysis module: used for comprehensive analysis to obtain the reliability evaluation index of student mental health data fusion; Optimize and adjust the student mental health early warning module: used to compare and analyze the student mental health data image quality assessment index with the first threshold of the student mental health data image quality assessment index, and optimize the student mental health data image quality method; compare and analyze the student mental health data fusion network volatility assessment index with the second threshold of the student mental health data fusion network volatility assessment index, and optimize the student mental health data fusion network volatility method; compare and analyze the student mental health data fusion reliability assessment index with the comprehensive threshold of the student mental health data fusion reliability assessment index, and optimize and adjust the student mental health early warning method.

Images