Teenager non-suicide self-injury behavior prediction system based on multi-modal data fusion

Abstract

A teenager non-suicide self-injury behavior prediction system based on multi-modal data fusion comprises a data collection module, a data preprocessing module, a feature fusion module, a model training and optimizing module, a model evaluation module, a prediction and intervention module, a user interface module, a data security and privacy protection module and the like. According to the invention, through a highly integrated multi-level architecture, full-process management from data acquisition, preprocessing and feature fusion to model training, evaluation and prediction intervention is realized, and all levels cooperate with each other, so that the accuracy of data, the effectiveness of a model and the reliability of a system are ensured. Through continuous user feedback and a system iteration module, the system can continuously adapt to changing requirements and environments, more accurate and timely prediction and intervention services are provided, and the system has important significance for improving prediction accuracy, early identifying risks, optimizing resource allocation and providing personalized intervention.

Description

Technical Field

[0001] This invention belongs to the field of adolescent psychological research technology, specifically involving a prediction system for non-suicidal self-harm behavior in adolescents based on multimodal data fusion. Background Technology

[0002] Non-suicidal self-harm is prevalent among adolescents, with an estimated 17% of adolescents worldwide having engaged in at least one self-harming behavior. NSSI behaviors are closely associated with a variety of mental disorders (such as depression, anxiety, and borderline personality disorder) and adverse consequences (such as suicide attempts, academic failure, and social impairment). Identification and prevention of NSSI behaviors are crucial for improving the mental health and quality of life of adolescents. Currently, the identification of NSSI behaviors mainly relies on assessments by mental health professionals, including interviews, psychological tests, and behavioral observations. This approach has the following limitations: (1) Resource constraints; the supply of professional mental health services often cannot meet the demand, especially in resource-scarce areas; (2) Time consumption; traditional assessment methods are time-consuming, which may lead to delayed intervention; (3) Subjectivity; professional assessments may be influenced by personal experience, biases, and fatigue, leading to inconsistencies in assessment results; (4) Limited coverage; traditional methods are difficult to cover all potentially at-risk adolescents, especially those who are unwilling to seek help.

[0003] In recent years, the application of artificial intelligence and machine learning technologies in the field of mental health has gradually increased. These technologies can process and analyze large amounts of data, identify patterns and correlations, and provide new possibilities for the prediction and intervention of mental health problems. However, existing research and applications mainly focus on the analysis of single data sources (such as text or physiological data), lacking the comprehensive utilization of multimodal data. Multimodal data fusion refers to combining data from different sources and types (such as text, physiological, behavioral, and environmental data) to obtain more comprehensive and accurate information. In the field of NSSI behavior prediction, multimodal data fusion faces the following challenges: First, data heterogeneity, as data from different modalities have different characteristics and formats, making direct integration difficult; second, feature extraction and fusion, effectively extracting features from heterogeneous data and achieving feature fusion is a technical challenge; third, data privacy and security, as multimodal data often contains sensitive information, and how to conduct data fusion and analysis while protecting privacy is an important ethical and legal issue; and finally, social and ethical considerations, as the prediction and intervention of NSSI behavior involves not only technical issues but also social and ethical considerations, such as how to ensure the fairness and non-discrimination of prediction results, how to protect the privacy and autonomy of adolescents, and how to take cultural and social differences into account during the prediction and intervention process. Summary of the Invention

[0004] Given the limitations of existing NSSI behavior prediction methods and the challenges of multimodal data fusion, this paper develops a method for predicting adolescent NSSI behavior based on multimodal feature information fusion to improve the accuracy and efficiency of prediction.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The system for predicting non-suicidal self-harm behaviors in adolescents based on multimodal data fusion includes a data collection module, a data preprocessing module, a feature fusion module, a model training and optimization module, a model evaluation module, a prediction and intervention module, a user interface module, a data security and privacy protection module, a system scalability and maintenance module, a fault tolerance mechanism and anomaly handling module, a user feedback and system iteration module, a multimodal data reinforcement learning module, a model interpretability and transparency module, a real-time data processing and analysis module, and a prediction result visualization and reporting module. The data collection module includes automated data collection and data annotation. Automated data collection uses automated scripts and APIs to collect data regularly from sources such as social media platforms, wearable devices, and school information systems. Data annotation annotates unstructured data to create training datasets. The data preprocessing module includes natural language processing, physiological data preprocessing, and behavioral data preprocessing. Natural language processing performs word segmentation, part-of-speech tagging, and sentiment analysis on text data. Physiological data preprocessing is used to perform signal filtering and noise reduction, outlier detection, and feature extraction on physiological monitoring data. Behavioral data preprocessing is used to perform time series analysis on behavioral logs to extract activity patterns and usage habits. The feature fusion module effectively integrates data features from different modalities to build a comprehensive user profile, including deep feature fusion and cross-modal association learning; The model training and optimization module is used to build and optimize prediction models, including transfer learning, ensemble learning, and hyperparameter optimization. The model evaluation module is used to evaluate the performance and reliability of the model to ensure its effectiveness in practical applications, including performance index calculation, model interpretability, and real-time performance monitoring. The prediction and intervention module is used to provide dynamic risk assessments and personalized intervention recommendations based on model prediction results; The user interface module provides an intuitive and user-friendly interface, including interactive visualizations and multilingual support. The data security and privacy protection module ensures that the system complies with relevant regulations when processing sensitive data, protecting user privacy and data security, including compliance checks and privacy protection technologies; The system scalability maintenance module ensures that the system can scale as demand changes and data volume grows, while remaining easy to maintain and update, including modular design and cloud service integration; Fault tolerance mechanism and exception handling module are used to improve the stability and reliability of the system under abnormal conditions and ensure continuous and effective operation. The user feedback and system iteration module continuously collects user feedback to drive system improvement and optimization, thereby enhancing user experience and system performance. The multimodal data augmentation learning module is used to improve the generalization ability and robustness of the model through data augmentation techniques, and enhance its adaptability to diverse data, including data augmentation and adversarial training. The Model Interpretability and Transparency module is designed to provide a deeper understanding of the model's internal decision-making processes, enhancing the model's credibility and acceptability, including the model interpretation framework and report production. The real-time data processing and analysis module is used to realize the real-time processing and analysis of high-speed, continuous data streams, and provide timely predictions and responses; The visualization and reporting module for forecast results presents forecast results and intervention recommendations in an intuitive and easy-to-understand format to support decision-making and action; a dynamic report generation tool is developed to present forecast results and intervention recommendations in graphical and tabular form, making it easier for professionals to understand and apply them.

[0006] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention significantly improves the predictive accuracy of NSSI behavior in adolescents through multimodal feature information fusion; it can identify adolescents at risk of self-harm at an early stage, providing the possibility for timely intervention, and reduces reliance on professional assessment, thus optimizing the allocation of mental health resources; this invention provides personalized intervention measures based on the prediction results, improving the targeting and effectiveness of intervention; this invention provides an intuitive user interface, facilitating use by professionals, ensuring data security and privacy protection, and complying with legal and regulatory requirements.

[0007] The prediction system of this invention, through a highly integrated multi-layered architecture, achieves end-to-end management from data acquisition, preprocessing, and feature fusion to model training, evaluation, and predictive intervention. Each layer collaborates with the others to ensure data accuracy, model effectiveness, and system reliability. Simultaneously, the system prioritizes user experience and data security, providing a practical and secure solution through a user-friendly interface and stringent privacy protection measures. Through continuous user feedback and system iteration modules, the system can constantly adapt to changing needs and environments, providing more accurate and timely prediction and intervention services. Detailed Implementation

[0008] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0009] The present invention will be further described in detail below with reference to the embodiments.

[0010] Specific embodiment 1 of the multimodal data fusion-based prediction system for non-suicidal self-harm behavior in adolescents provided by this invention: The system for predicting non-suicidal self-harm behaviors in adolescents based on multimodal data fusion includes a data collection module, a data preprocessing module, a feature fusion module, a model training and optimization module, a model evaluation module, a prediction and intervention module, a user interface module, a data security and privacy protection module, a system scalability and maintenance module, a fault tolerance mechanism and anomaly handling module, a user feedback and system iteration module, a multimodal data reinforcement learning module, a model interpretability and transparency module, a real-time data processing and analysis module, and a prediction result visualization and reporting module. The data collection module includes automated data collection and data annotation. Automated data collection regularly gathers data from sources such as social media platforms, wearable devices, and school information systems through APIs and web scraping technologies. Wearable devices can acquire physiological indicators of adolescents, such as heart rate, sleep patterns, and activity levels. School information systems can collect data such as academic performance, attendance, and mental health assessments. Data annotation includes manual annotation and semi-automatic annotation. Manual annotation is performed on the collected unstructured data, such as social media text and images, to create training datasets. Semi-automatic annotation uses preliminary models to assist in annotation, thereby improving efficiency and accuracy.

[0011] The data preprocessing module cleans, standardizes, and performs preliminary analysis on raw data to ensure data quality and extract useful information. This includes natural language processing (NLP), physiological data preprocessing, and behavioral data preprocessing. NLP performs word segmentation, part-of-speech tagging, and sentiment analysis on text data. Word segmentation and part-of-speech tagging break down text into lexical units and assign them their parts of speech. Sentiment analysis assesses the emotional tendency of the text and identifies potential negative emotions. Physiological data preprocessing performs signal filtering and denoising, outlier detection, and feature extraction on physiological monitoring data. Signal filtering removes noise and interference from physiological data. Outlier detection identifies and handles abnormal data points to ensure data consistency. Feature extraction extracts key features from physiological signals, such as heart rate variability. Behavioral data preprocessing performs time-series analysis on behavioral logs to extract activity patterns and usage habits. This includes time-series analysis, pattern recognition, and data standardization. Time-series analysis analyzes the temporal patterns of behavioral data to identify regularities and anomalies. Pattern recognition extracts features of daily activities and usage habits. Data standardization converts behavioral data from different sources into a unified format.

[0012] The feature fusion module effectively integrates data features from different modalities to build a comprehensive user profile. This includes deep feature fusion and cross-modal association learning. Deep feature fusion uses the attention mechanism and multimodal fusion network in deep learning to dynamically adjust the weights of features from different modalities. Cross-modal association learning uses graph neural networks (GNNs) to establish associations between data from different modalities.

[0013] The model training and optimization module is used to build and optimize predictive models to achieve high accuracy and robustness in behavior prediction. It includes transfer learning, ensemble learning, and hyperparameter optimization. Transfer learning utilizes models pre-trained on large datasets and fine-tunes them to adapt to specific data in the domain. Ensemble learning combines multiple machine learning algorithms to improve the robustness and accuracy of the model and improves the overall prediction accuracy through the combination of different models. Hyperparameter optimization uses methods such as grid search and Bayesian optimization to find the optimal model parameters, evaluates the model's performance under different data splits, and prevents overfitting.

[0014] The model evaluation module assesses the model's performance and reliability to ensure its effectiveness in practical applications. This includes performance metric calculation, model interpretability, and real-time performance monitoring. Performance metrics include accuracy, recall, F1 score, ROC curve, and AUC value, comprehensively measuring the model's predictive ability. ROC curve and AUC value evaluate the model's classification performance at different thresholds. Model interpretability introduces interpretability tools, such as SHAP values ​​or LIME methods, to improve the interpretability of model prediction results. SHAP values ​​quantify the contribution of each feature to the prediction result, enhancing model transparency. LIME methods provide local explanations of individual predictions, helping to understand the model's decision-making process. Real-time performance monitoring includes an online evaluation and alert system. Prior to evaluation, the system continuously monitors the model's performance on real-time data, promptly identifying and correcting performance degradation. When abnormal performance is detected, the alert system automatically triggers alarms and intervention measures.

[0015] The prediction and intervention module provides dynamic risk assessments and personalized intervention recommendations based on model predictions. This includes dynamic risk assessments and personalized recommendations for intervention measures. The dynamic risk assessment is continuously updated based on the latest data to reflect the current situation and categorizes risk levels into different grades to facilitate appropriate action. The personalized recommendations for intervention measures utilize reinforcement learning algorithms to optimize and adjust intervention strategies based on individual feedback and intervention effectiveness. Based on the assessment results, appropriate mental health resources and support services are recommended, and feedback on intervention effectiveness is collected to further refine the intervention strategies.

[0016] The user interface module provides an intuitive and user-friendly interface, facilitating understanding and use of system functions by professionals and users. This includes interactive visualizations and multilingual support. The interactive visualizations feature data dashboards, visualization tools, and interactive controls. Data dashboards display key indicators and trends, supporting in-depth analysis. Visualization tools use charts, heatmaps, and other formats to visually present data and forecasts. Interactive controls allow users to customize views, filter data, and explore details. Multilingual support supports multiple languages ​​to adapt to different regions and cultural backgrounds, considering cultural differences and providing content that aligns with local customs and expectations.

[0017] The data security and privacy protection module ensures that the system complies with relevant regulations when processing sensitive data, protecting user privacy and data security. This includes compliance checks and privacy protection technologies. The compliance checks can be performed regularly to ensure that the system complies with data protection regulations such as GDPR and HIPAA. The privacy protection technologies employ differential privacy, homomorphic encryption, and other techniques to perform data analysis while protecting privacy.

[0018] The system features a scalable maintenance module, ensuring it can expand with changing demands and increasing data volume while remaining easy to maintain and update. This includes modular design and cloud service integration. Modular design divides the system into modules that can be independently developed, tested, and deployed, improving maintenance efficiency. Furthermore, it defines clear interfaces between modules, ensuring compatibility and collaboration between components. Cloud service integration includes cloud computing platforms, containerized deployment, and automated operations and maintenance. The cloud computing platform utilizes cloud services such as AWS and Azure to provide elastic computing and storage resources. Containerized deployment uses technologies such as Docker and Kubernetes to achieve rapid deployment and scaling. Automated operations and maintenance, through CI / CD pipelines, enables continuous code integration and deployment, reducing human error.

[0019] Fault tolerance and exception handling modules are used to improve the stability and reliability of the system under abnormal conditions, ensuring continuous and effective operation. These modules include a fault tolerance module and an exception handling module. The fault tolerance module prevents data loss and service interruption through backup and redundancy design; it automatically switches to a backup system when a component fails, maintaining service continuity. Potential faults are considered in the design to ensure the system can operate smoothly under abnormal conditions. The exception handling module monitors the system's operating status in real time, quickly identifies abnormal situations, and records detailed error information, supporting subsequent analysis and problem solving. When an anomaly occurs, it automatically takes corrective measures to restore normal operation.

[0020] The user feedback and system iteration module continuously collects user feedback to drive system improvement and optimization, enhancing user experience and system performance. User feedback collection utilizes multiple channels (such as in-app feedback and surveys) to gather user opinions and suggestions. The collected feedback is categorized and analyzed to identify common problems and improvement opportunities. System iteration updates are performed regularly based on feedback and performance monitoring results, updating and optimizing system functions periodically. The update content and improvement measures for each version are clearly recorded to ensure traceability. Users are promptly notified and update instructions are provided during system updates.

[0021] The multimodal data augmentation learning module enhances the model's generalization ability and robustness through data augmentation techniques, improving its adaptability to diverse data. This includes data augmentation and adversarial training. Data augmentation uses methods such as synonym replacement, random insertion, and deletion to enrich the text dataset; it also adds techniques like simulated noise and time warping to enhance the diversity of physiological signal data, generating synthetic behavioral sequences to simulate behavioral patterns in different scenarios. Adversarial training generates realistic synthetic data to enhance the model's learning ability; perturbations are added during training to improve the model's resistance to noise and anomalies.

[0022] The Model Interpretability and Transparency module provides a deeper understanding of the model's internal decision-making process, enhancing the model's credibility and acceptability. It includes model interpretation frameworks and report generation, such as TensorBoard or Weights & Biases, to provide transparency and interpretability of the model training process. It regularly generates reports detailing the model's performance, strengths, and limitations, and uses flowcharts and diagrams to visually illustrate the model's decision-making path.

[0023] The real-time data processing and analysis module enables real-time processing and analysis of high-speed, continuous data streams, providing timely predictions and responses. It utilizes streaming data processing technologies such as Apache Kafka or Apache Flink to achieve rapid processing and analysis of real-time data. It allows setting thresholds for key indicators, real-time detection and alerting of anomalies, and timely notifications to relevant personnel via SMS, email, and other means. The visualization and reporting module for forecast results presents forecasts and intervention recommendations in an intuitive and easy-to-understand format to support decision-making and action. It develops dynamic report generation tools to display forecast results and intervention recommendations in graphical and tabular formats, facilitating professional understanding and application. For example, automated reporting automatically generates reports containing key information based on forecast results. It supports multiple report formats (such as PDF and HTML) to meet the needs of different scenarios. Interactive charts and data views are embedded in the reports to enhance the user experience. Result visualization includes trend charts, heatmaps, and map visualizations. Trend charts show the changing trends of risk levels, helping to identify long-term patterns; heatmaps highlight high-risk areas and time periods, supporting rapid identification and response; and map visualizations display data at a geographical level, identifying regional risks and characteristics.

[0024] It should be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0025] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A prediction system for non-suicidal self-harm behavior in adolescents based on multimodal data fusion, characterized in that: It includes modules for data collection, data preprocessing, feature fusion, model training and optimization, model evaluation, prediction and intervention, user interface, data security and privacy protection, system scalability and maintenance, fault tolerance and exception handling, user feedback and system iteration, multimodal data reinforcement learning, model interpretability and transparency, real-time data processing and analysis, and visualization and reporting of prediction results. The data collection module includes automated data collection and data annotation. Automated data collection uses automated scripts and APIs to collect data regularly from sources such as social media platforms, wearable devices, and school information systems. Data annotation annotates unstructured data to create training datasets. The data preprocessing module includes natural language processing, physiological data preprocessing, and behavioral data preprocessing. Natural language processing performs word segmentation, part-of-speech tagging, and sentiment analysis on text data. Physiological data preprocessing is used to perform signal filtering and noise reduction, outlier detection, and feature extraction on physiological monitoring data. Behavioral data preprocessing is used to perform time series analysis on behavioral logs to extract activity patterns and usage habits. The feature fusion module effectively integrates data features from different modalities to build a comprehensive user profile, including deep feature fusion and cross-modal association learning; The model training and optimization module is used to build and optimize prediction models, including transfer learning, ensemble learning, and hyperparameter optimization. The model evaluation module is used to evaluate the performance and reliability of the model to ensure its effectiveness in practical applications, including performance index calculation, model interpretability, and real-time performance monitoring. The prediction and intervention module is used to provide dynamic risk assessments and personalized intervention recommendations based on model prediction results; The user interface module provides an intuitive and user-friendly interface, including interactive visualizations and multilingual support. The data security and privacy protection module ensures that the system complies with relevant regulations when processing sensitive data, protecting user privacy and data security, including compliance checks and privacy protection technologies; The system scalability maintenance module ensures that the system can scale as demand changes and data volume grows, while remaining easy to maintain and update, including modular design and cloud service integration; Fault tolerance mechanism and exception handling module are used to improve the stability and reliability of the system under abnormal conditions and ensure continuous and effective operation. The user feedback and system iteration module continuously collects user feedback to drive system improvement and optimization, thereby enhancing user experience and system performance. The multimodal data augmentation learning module is used to improve the generalization ability and robustness of the model through data augmentation techniques, and enhance its adaptability to diverse data, including data augmentation and adversarial training. The Model Interpretability and Transparency module is designed to provide a deeper understanding of the model's internal decision-making processes, enhancing the model's credibility and acceptability, including the model interpretation framework and report production. The real-time data processing and analysis module is used to realize the real-time processing and analysis of high-speed, continuous data streams, and provide timely predictions and responses; The visualization and reporting module for forecast results presents forecast results and intervention recommendations in an intuitive and easy-to-understand format to support decision-making and action; Develop dynamic report generation tools to present prediction results and intervention recommendations in graphical and tabular form, making them easier for professionals to understand and apply.