A social risk perception method based on a large model and multi-modal data

Abstract

The present application relates to a kind of social risk perception methods based on large model and multi-modal data, the method comprises: collecting multi-modal data, including policy and public data etc.;According to preset label type, data is labeled;According to policy and public data etc. Construct social risk perception knowledge base;Build base model;Using social risk perception knowledge base trains model, and the social risk perception model is obtained by training;The data collected are input into social risk perception model and are risk forecasted, and the risk data of each event and risk prediction report are obtained;Enrich knowledge base, and the prediction result is self-corrected by confidence learning.The present application establishes mathematical model by combining large model and multi-modal data, enhances the early prediction of social risk, provides more extensive understanding for using large model to predict social risk, so that social risk can be changed from experience governance to large model big data wisdom governance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data mining, and particularly relates to a social risk perception method based on a large model and multi-modal data. BACKGROUND

[0002] The current society is facing challenges and changes in economy, environment, technology and other aspects. Such changes have led to an increase in the diversity and complexity of social risks, which require more accurate and comprehensive methods to identify and respond to. The social risk perception system is an artificial intelligence technology based on big data, which is used to analyze and calculate the social risk index in a region, identify risk events that need to be focused on and provide processing suggestions. It is of great significance to the development and stability of society. This technology uses a large model to analyze various data such as social security incidents, traffic accidents, and hot news events to obtain the risk situation of the entire region. It can monitor and analyze potential social risk factors and provide early warnings and processing suggestions to help managers make quick decisions before or at the early stage of the problem, thereby reducing losses and impacts. Through timely data analysis and effective risk management, the social risk perception system helps to prevent social instability events and ensures social order and national security. SUMMARY

[0003] In order to improve the scientificity and accuracy of social management and promote the stability and sustainable development of society, the present application provides a social risk perception method based on a large model and multi-modal data.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] A social risk perception method based on a large model and multi-modal data, the method comprising:

[0006] S1, collecting multi-modal data, including policy and public data, statistical data and survey data, news reports and social hotspots;

[0007] S2, labeling news reports and social hotspots according to preset label types;

[0008] S3, constructing a social risk perception knowledge base according to policy and public data, statistical data and survey data, and labeled news reports and social hotspots, for model training;

[0009] S4, constructing a basic model composed of a voice feature extraction module, a text feature extraction module and a large model;

[0010] S5, training the model using the social risk perception knowledge base, obtaining a social risk perception model, and periodically performing incremental training to optimize the model;

[0011] S6, input the collected data into the social risk perception model for risk prediction, to obtain risk data and a risk prediction report of each event;

[0012] S7, constantly enrich the knowledge base, and perform self-correction on the prediction result through confidence learning, so that the prediction result is more accurate.

[0013] As a further technical solution of the application, in step S1, the collected multi-modal data comprises:

[0014] Policy and public data: laws and regulations, policy changes, legislative activities and planning documents;

[0015] Statistical data and survey data: population growth, GDP growth rate, age distribution, employment, education, etc. have a multi-dimensional and complex relationship with social risk, and the collection channels are the National Bureau of Statistics and Baidu Index, etc.

[0016] News reports and social hotspots: news articles and hot video and comment content, number of forwarded comments;

[0017] As a further technical solution of the application, in step S2, the step of labeling the news reports and social hotspots according to the preset label type comprises:

[0018] Labeling the news reports and social hotspots, the label labeling work is only performed at the initial training, and the inference result of the model is used as the label for the subsequent collected new data; the specific label content is as follows:

[0019] Time label: including event occurrence time and data publishing time;

[0020] Nature label: the events contained in the data are divided into risk events and positive events, and the grades are divided into high, medium and low;

[0021] Emotion label: the overall emotional atmosphere of the data comments, if there is no comment, it is none;

[0022] Heat label: record the interaction of the data, including the number of comments and forwards, and the heat is obtained by adding the weight of the number of comments and the number of forwards, and the weight is the number of comments + the number of forwards * 2;

[0023] Trend label: the heat development trend of the event is calculated according to the heat information of the same event in the future preset time in the collected information;

[0024] Measures label: for positive events, generate measures to expand the impact, such as widely disseminating through news media, social platforms, inviting public figures to participate in publicity, giving policy support, etc.; for risk events, generate prevention measures, such as establishing a response team, strengthening public opinion monitoring, and preventing rumors from spreading, etc.

[0025] As a further technical solution of the present invention, step S3, which involves constructing a social risk perception knowledge base based on policy and public data, statistical data and survey data, as well as labeled news reports and social hotspots, for model training, includes the following steps:

[0026] To improve the efficiency and performance of the model, enabling it to provide stable prediction capabilities even with scarce or interfering data, this invention organizes and summarizes the multimodal data collected in step S1 and the labels obtained in step S2 into structured data, constructing a social risk perception knowledge base for model training. News reports and social hot topics typically contain a large amount of redundant information, requiring the extraction of summaries from a large model. These summaries are then used as knowledge base content, only used during model retrieval. Risk prediction utilizes the specific content and labels stored under the summaries. During data retrieval, the summaries are matched first, and then the specific content stored under those summaries is analyzed to confirm if it matches the desired content. This effectively controls the size of the knowledge base and the retrieval speed.

[0027] As a further technical solution of the present invention, step S4, which involves constructing a basic model consisting of a speech feature extraction module, a text feature extraction module, and a large model, includes the following steps:

[0028] Let A={I1,…,I n} represents the collected raw data sequence, where I represents a data point and n represents the sequence length. The model consists of the following components: an audio feature extraction module (Audio_Encoder), a text feature extraction module (Text_Encoder), and a large model (LLM).

[0029] Text Feature Extraction Module: The Text_Encoder module is based on the Transformer architecture and utilizes self-attention mechanism and context-sensitive embedding technology to effectively capture the rich semantic information of each word in the text;

[0030] After inputting text data I, the Text_Encoder module will convert the text of I into a dense vector representation I. t These vectors capture the semantics and features of the text, which can be analyzed by large models;

[0031] Speech Feature Extraction Module: The speech feature extraction module Audio_Encoder is composed of multiple stacked Transformer blocks. Each Transformer block consists of a multi-head self-attention mechanism and a feedforward neural network. In the self-attention mechanism, the features at each time step interact with all time steps of the entire sequence to generate attention weights, which enables the model to model dependencies over a long period of time.

[0032] Speech is input into the speech feature extraction module in the form of raw waveform. After multiple layers of Transformer encoding, the input is gradually transformed into a high-dimensional hidden vector containing rich semantic information. The high-dimensional hidden vector captures the spectral pattern, semantic content and contextual information of the audio, and is the core input for subsequent large-scale model speech analysis.

[0033] Large Model: The Large Model LLM is a 32-layer Transformer decoder model with 4096 hidden layers and a total of 7 billion parameters. It is used to process speech features extracted by the Audio Encoder and text features extracted by the Text Encoder. Based on the set task, it analyzes the semantic information contained in the input features and the emotional information contained in the audio features, providing richer dimensions for the model's risk prediction.

[0034] As a further technical solution of the present invention, step S5, which involves training a model using a social risk perception knowledge base, includes the following steps:

[0035] Large models are further optimized during training using the DPO loss function, where L is the loss function. DPO as follows:

[0036] Where D represents the dataset, which contains triplet data. , where x is the input sequence, and These are the labeled good and bad responses, respectively. This represents all data in dataset D. Seeking expectations, The model to be optimized has parameter θ. Indicates using The initial reference model, where log is the logarithm symbol, σ represents the sigmoid function, and β is a hyperparameter;

[0037] After obtaining the initial social risk perception model through pre-training, it is necessary to continuously use new data to strengthen the model during use in order to continuously optimize the model. Due to the continuous increase of new data, continuing to use the entire knowledge base for model training will lead to increased training time and wasted resources. Therefore, an incremental training method is adopted in subsequent model training. First, the data of the most recent month is taken, and then the same number of data are randomly sampled from historical data to form an incremental dataset. Then, this dataset is used for incremental training to save resources and optimize the model.

[0038] As a further technical solution of the present invention, the step of inputting the collected data into a social risk perception model to perform risk prediction and obtain risk data for each event and a risk prediction report includes:

[0039] The social risk perception system collects new data every day and analyzes and processes each piece of new data. First, the model extracts core information such as summary, popularity, sentiment and time for each piece of new data. Then, it queries the knowledge base through the data summary to identify the same events. By merging the same events, it tracks the trend of popularity and sentiment changes of events based on time sequence.

[0040] For the events collected daily, the model will sort them according to popularity, level and sentiment changes, prioritize the analysis of the risk events and positive events with the highest popularity, and provide development trend predictions and coping strategies.

[0041] The daily analysis results will be compiled into a risk prediction report, which includes a summary of the day's risk events, trends in sentiment and popularity, and corresponding coping strategies. The risk prediction report will also combine historical trends and changes in current data to assess potential future risk directions, helping users make informed decisions about the future development trends and countermeasures for key events.

[0042] As a further technical solution of the present invention, step S7, which involves continuously enriching the knowledge base and performing self-correction on the prediction results through confidence learning to make the prediction results more accurate, includes the following steps:

[0043] The trained model is deployed to a real-world environment for testing, and prediction services are provided through API interfaces and web applications. In the early stages of algorithm testing, the prediction results need to be manually verified to identify and record any unqualified data in the model's prediction results. Then, a confidence learning task is added for incremental training.

[0044] Confidence learning is a machine learning method that aims to provide a confidence estimate of the model's output. Confidence represents how confident the model is in its output, and is described by a probability value between 0 and 1. The higher the confidence, the more confident the model is in the output.

[0045] The confidence learning task designed in this application refers to telling the model the proportion of unqualified labels in the training data during incremental training, and letting the model score all the labels of the data to find unqualified labels. After multiple training sessions, when the proportion of unqualified labels in the model results is less than one in a thousand, manual verification is no longer performed. However, each time the model self-checks, it is set that there is erroneous data in the data. Then, the data with confidence scores lower than the preset threshold are removed, and the remaining data are merged into the knowledge base.

[0046] Compared with existing technologies, the beneficial effects of this invention are: by combining large models and multimodal data to establish mathematical models, this invention enhances the early prediction of social risks, provides a broader understanding of using large models for social risk prediction, and makes it possible to shift social risk governance from experience-based governance to big data-driven intelligent governance based on large models. This helps to improve the scientificity and accuracy of social management and promotes social stability and sustainable development. Attached Figure Description

[0047] Figure 1 This is a flowchart of a social risk perception method based on large models and multimodal data.

[0048] Figure 2 This is a diagram of the risk prediction network structure in an embodiment of the present invention. Detailed Implementation

[0049] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0050] Please see Figure 1 This invention provides a social risk perception method based on large models and multimodal data, the method comprising:

[0051] S1. Collect multimodal data. Multimodal data includes various types of data such as text, images, sound, and video. This allows for the construction of more accurate models and provides a more comprehensive perspective for predicting the likelihood and trend of risk occurrence. Currently, the input of large models is generally text. To obtain more reference data, this invention, in addition to traditional text information, will also collect video data and extract audio to enrich the data source. Below is an example of the collected data;

[0052] Policy and public data: Laws and regulations, policy changes, legislative activities and planning documents, play a guiding role in the direction of social risks;

[0053] Statistical and survey data: Population growth, GDP growth rate, age distribution, employment status, education status, etc., have a multi-dimensional and complex relationship with social risks. Data collection channels include the National Bureau of Statistics and Baidu Index.

[0054] News reports and social hot topics: News articles and trending videos along with their comments, number of reposts and comments; audio data is extracted from trending videos during use; text and video news reports can provide detailed information about specific events or trends; data sources: Xinhua News Agency, People's Daily Online, etc., using the platform's public RSS feed to subscribe to news updates; Weibo, Douyin videos, etc., obtained through official developer tools or APIs to avoid directly downloading or storing data and infringing copyright; and the collected data is anonymized to remove user privacy information (such as usernames, location information, etc.).

[0055] All data in this invention comes from legal and publicly available sources. The principles of legality, compliance, and privacy protection are always upheld during the collection and use of data. We strictly abide by relevant laws and regulations as well as the terms of use of the data source platforms to ensure that user privacy and intellectual property rights are not infringed. The collected data is used only for internal analysis and will not be used for commercial purposes or dissemination.

[0056] S2. Label news reports and social hot topics according to preset label types;

[0057] After collecting the required data, it is necessary to label and organize the data. Firstly, policy and public data, statistical data, and survey data do not require labeling, as their multi-dimensional and complex relationships with social risks are automatically analyzed by the model during learning. Secondly, news reports and social hotspots are labeled. Labeling is only performed during initial training; subsequent data will use the model's inference results as labels. Specific labeling content is as follows:

[0058] Time tags: include the time of event occurrence and the time of data release;

[0059] Nature Labels: The events contained in the data are divided into risk events and positive events, and the levels are divided into high, medium and low.

[0060] Sentiment tag: The overall sentiment of the data comments; if there are no comments, then none.

[0061] Popularity tag: Records the interaction of data, including the number of comments and reposts. Popularity is calculated by adding the two in a weighted manner, with the weight being the number of comments + the number of reposts * 2.

[0062] Trend tags: Calculate the trend of an event's popularity within a preset timeframe based on the popularity information of the same event collected from the data.

[0063] Measures are categorized as follows: For positive events that amplify their impact, measures include widespread dissemination through news media and social media platforms, inviting public figures to participate in promotion, and providing policy support; for risk events that require preventative measures, measures include establishing response teams, strengthening public opinion monitoring, and preventing the spread of rumors.

[0064] S3. Construct a social risk perception knowledge base based on policy and public data, statistical data and survey data, as well as labeled news reports and social hotspots, for model training;

[0065] To improve the efficiency and performance of the model, enabling it to provide stable prediction capabilities even with scarce or interfering data, this invention organizes and summarizes the multimodal data collected in step S1 and the labels obtained in step S2 into structured data, constructing a social risk perception knowledge base for model training. News reports and social hot topics typically contain a large amount of redundant information, requiring the extraction of summaries from a large model. These summaries are then used as knowledge base content, only used during model retrieval. Risk prediction utilizes the specific content and labels stored under the summaries. During data retrieval, the summaries are matched first, and then the specific content stored under those summaries is analyzed to confirm if it matches the desired content. This effectively controls the size of the knowledge base and the retrieval speed.

[0066] S4. Construct the basic model, which consists of three parts: speech feature extraction module, text feature extraction module, and large model;

[0067] Let A={I1,…,I n} represents the collected raw data sequence, where I represents a data point and n represents the sequence length. The model consists of the following components: an audio feature extraction module (Audio_Encoder), a text feature extraction module (Text_Encoder), and a large LLM model (see appendix). Figure 2 ;

[0068] Text Feature Extraction Module: The Text_Encoder module is based on the Transformer architecture and utilizes self-attention mechanism and context-sensitive embedding technology to effectively capture the rich semantic information of each word in the text. The pre-trained Text_Encoder module can generate representations that deeply understand language. After being fine-tuned for specific tasks, these representations can be effectively applied to various natural language understanding tasks and have excellent text feature extraction capabilities.

[0069] After inputting text data I, the Text_Encoder module will convert the text of I into a dense vector representation I. t These vectors capture the semantics and features of the text, which can be analyzed by large models;

[0070] Speech Feature Extraction Module: The speech feature extraction module Audio_Encoder is composed of multiple stacked Transformer blocks. Each Transformer block consists of a multi-head self-attention mechanism and a feedforward neural network. In the self-attention mechanism, the features at each time step interact with all time steps of the entire sequence to generate attention weights, which enables the model to model dependencies over a long period of time.

[0071] Speech is input into the speech feature extraction module in the form of raw waveform. After multiple layers of Transformer encoding, the input is gradually transformed into a high-dimensional hidden vector containing rich semantic information. The high-dimensional hidden vector captures the spectral pattern, semantic content and contextual information of the audio, and is the core input for subsequent large-scale model speech analysis.

[0072] Large Model: The Large Model LLM is a 32-layer Transformer decoder model with 4096 hidden layers and a total of 7 billion parameters. It is used to process speech features extracted by the Audio Encoder and text features extracted by the Text Encoder. Based on the set task, it analyzes the semantic information contained in the input features and the emotional information contained in the audio features, providing richer dimensions for the model's risk prediction.

[0073] S5. Use the social risk perception knowledge base to train the model, obtain the social risk perception model, and perform incremental training to optimize the model regularly.

[0074] As a further technical solution of the present invention, step S5, which involves training a model using a social risk perception knowledge base, includes the following steps:

[0075] During training, the large model employs the Direct Preference Optimization (DPO) loss function to further optimize the model and conform to human preferences. The core idea of ​​DPO is to directly adjust model parameters using preference data, bypassing the fitting of explicit reward models and complex reinforcement learning optimization processes. This simplified training method not only improves training efficiency but also avoids the instabilities common in traditional human feedback reinforcement learning methods. The DPO loss function L... DPO as follows:

[0076] ;

[0077] Where D represents the dataset, which contains triplet data. , where x is the input sequence, and These are the labeled good and bad responses, respectively. This represents all data in dataset D. Seeking expectations, The model to be optimized has parameter θ. Indicates using The initial reference model, where log is the logarithm symbol, σ represents the sigmoid function, and β is a hyperparameter;

[0078] After obtaining the initial social risk perception model through pre-training, it is necessary to continuously use new data to strengthen the model during use in order to continuously optimize the model. Due to the continuous increase of new data, continuing to use the entire knowledge base for model training will lead to increased training time and wasted resources. Therefore, an incremental training method is adopted in subsequent model training. First, the data of the most recent month is taken, and then the same number of data are randomly sampled from historical data to form an incremental dataset. Then, this dataset is used for incremental training to save resources and optimize the model.

[0079] S6. Input the collected data into the social risk perception model to make risk predictions and obtain risk data and risk prediction reports for each event.

[0080] New data is collected daily and analyzed one by one. First, the model extracts core information such as summary, popularity, sentiment and time for each new data. Then, it queries the knowledge base through the data summary to identify the same events. By merging the same events, it tracks the popularity and sentiment trends of the events based on the time sequence.

[0081] During the processing, the model can quickly search historical data to determine whether the events in the data are recurring old events, or provide more accurate trend predictions and countermeasures for new events based on the historical records of similar events.

[0082] For the events collected daily, the model will sort them according to popularity, level and sentiment changes, and prioritize the analysis of the risk events and positive events with the highest popularity, as well as other events with high-risk characteristics (such as rapidly rising popularity and negative public opinion, frequent related events, and a large number of participants), and provide development trend predictions and coping strategies.

[0083] The daily analysis results will be compiled into a risk prediction report, which includes a summary of the day's risk events, trends in sentiment and popularity, and corresponding coping strategies. The risk prediction report will also combine historical trends and changes in current data to assess potential future risk directions, helping users make informed decisions about the future development trends and countermeasures for key events.

[0084] S7. Continuously enrich the knowledge base and use confidence learning to self-correct the prediction results, making the prediction results more accurate.

[0085] The trained model is deployed to a real-world environment for testing, and prediction services are provided through API interfaces and web applications. In the early stages of algorithm testing, the prediction results need to be manually verified to identify and record any unqualified data in the model's prediction results. Then, a confidence learning task is added for incremental training.

[0086] Confidence learning is a machine learning method that aims to provide a confidence estimate of the model's output. Confidence represents how confident the model is in its output, and is described by a probability value between 0 and 1. The higher the confidence, the more confident the model is in the output.

[0087] The confidence learning task designed in this invention refers to telling the model the proportion of unqualified labels in the training data during incremental training, so that the model scores the labels of all data and finds the unqualified labels. After multiple training sessions, when the proportion of unqualified labels in the model results is less than one in a thousand, manual verification is no longer performed. However, each time the model self-checks, it is set that there is erroneous data in the data. Then, the data with confidence scores lower than the preset threshold are removed, and the remaining data are merged into the knowledge base.

[0088] It should be noted that, in this document, the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0089] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A social risk perception method based on large models and multimodal data, characterized in that, The method includes: S1. Collect multimodal data, including policy and public data, statistical data and survey data, news reports and social hotspots; S2. Label news reports and social hot topics according to preset label types; S3. Construct a social risk perception knowledge base based on policy and public data, statistical data and survey data, as well as labeled news reports and social hotspots, for model training; S4. Construct the basic model, which consists of three parts: speech feature extraction module, text feature extraction module, and large model; S5. Use the social risk perception knowledge base to train the model, obtain the social risk perception model, and perform incremental training to optimize the model regularly. S6. Input the collected data into the social risk perception model to make risk predictions and obtain risk data and risk prediction reports for each event. S7. Continuously enrich the knowledge base and use confidence learning to self-correct the prediction results, making the prediction results more accurate. In step S7, the steps of continuously enriching the knowledge base and using confidence learning to self-correct prediction results to make the predictions more accurate include: deploying the trained model to a real-world environment for testing, and providing prediction services through API interfaces and web applications; in the early stages of algorithm testing, the prediction results need to be manually verified to identify and record unqualified data in the model's prediction results, and then a confidence learning task is added for incremental training; confidence learning is a machine learning method whose goal is to provide a confidence estimate for the model's output, where confidence represents the model's confidence in its output; the confidence learning task refers to telling the model the proportion of unqualified labels in the training data during incremental training, allowing the model to score all data labels and identify unqualified labels. After multiple training iterations, when the proportion of unqualified results in the model's results is less than one in a thousand, manual verification is no longer performed, but each time the model self-checks, it is set that there is erroneous data in the data, and then data with confidence scores below a preset threshold are removed, and the remaining data are merged into the knowledge base; In S5, the steps for training the social risk perception model using a social risk perception knowledge base include: Large models are further optimized during training using the DPO loss function, where L is the loss function. DPO as follows: ； Where D represents the dataset, which contains triples (x, y) w ,y l ), where x is the input sequence, y w and y l These are the labeled good and bad responses, respectively. This represents all data (x, y) in dataset D. w ,y l Find the expected value, P. θ The model to be optimized has parameters θ and P. ref Indicated by P θ The initial reference model is defined by log, where log represents the logarithm, σ represents the sigmoid function, and β is a hyperparameter.

2. The social risk perception method based on large models and multimodal data according to claim 1, characterized in that, In S1, the collected multimodal data includes: Policy and public data: laws and regulations, policy changes, legislative activities, and planning documents; Statistical and survey data: population growth, GDP growth rate, age distribution, employment, and education; News reports and social hot topics: news articles and trending videos, along with their commentary content and the number of reposts and comments.

3. The social risk perception method based on large models and multimodal data according to claim 1, characterized in that, In S2, the steps for labeling news reports and social hotspots according to preset label types include: News reports and social hot topics are labeled. Labeling is only performed during initial training; subsequent data collection will use the model's inference results as labels. Specific label content is as follows: Time tags: include the time of event occurrence and the time of data release; Nature Labels: The events contained in the data are divided into risk events and positive events, and the levels are divided into high, medium and low. Sentiment tag: The overall sentiment of the data comments; if there are no comments, then none. Popularity tag: Records the interaction of data, including the number of comments and reposts. Popularity is calculated by adding the two in a weighted manner, with the weight being the number of comments + the number of reposts * 2. Trend tags: Calculate the trend of an event's popularity within a preset timeframe based on the popularity information of the same event collected from the data. Measures labeled: Measures to amplify the impact of positive events, and preventive measures to address risky events.

4. The social risk perception method based on large models and multimodal data according to claim 3, characterized in that, In S3, a social risk perception knowledge base is constructed based on policy and public data, statistical data and survey data, as well as labeled news reports and social hotspots. The steps used for model training include: The multimodal data collected in S1 and the tags obtained in S2 are organized and summarized into structured data to construct a social risk perception knowledge base for model training. News reports and social hot topics contain a lot of redundant information, so it is necessary to extract summaries through a large model first, and then use the extracted summaries as the content of the knowledge base. The summaries are only used when the model is searching, and the specific content and tags stored under the summaries are used when predicting risks. When searching data, the summary is matched first, and then the specific content stored under the summary is analyzed to confirm whether it is the required content, which effectively controls the size of the knowledge base and the search speed.

5. The social risk perception method based on large models and multimodal data according to claim 3, characterized in that, In S4, the steps for building the basic model, which consists of three parts: a speech feature extraction module, a text feature extraction module, and a large model, include: Let A={I1,…,I n } represents the collected raw data sequence, where I represents a data point and n represents the sequence length. The model consists of the following components: an audio feature extraction module (Audio_Encoder), a text feature extraction module (Text_Encoder), and a large model (LLM). Text Feature Extraction Module: The Text_Encoder module is based on the Transformer architecture and effectively captures the rich semantic information of each word in the text by using self-attention mechanism and context-sensitive embedding technology. After inputting text data I, the Text_Encoder module will convert the text of I into a dense vector representation I; Speech Feature Extraction Module: The speech feature extraction module Audio_Encoder is composed of multiple stacked Transformer blocks. Each Transformer block consists of a multi-head self-attention mechanism and a feedforward neural network. In the self-attention mechanism, the features at each time step interact with all time steps of the entire sequence to generate attention weights. Speech is input into the speech feature extraction module in the form of raw waveform. After multiple layers of Transformer encoding, the input is gradually transformed into a high-dimensional hidden vector containing rich semantic information. The high-dimensional hidden vector captures the spectral pattern, semantic content and contextual information of the audio, and is the core input for subsequent large-scale model speech analysis. Large Model: The Large Model LLM is a 32-layer Transformer decoder model with 4096 hidden layers and a total of 7 billion parameters. It is used to process speech features extracted by Audio Encoder and text features extracted by Text Encoder, and analyzes the semantic information contained in the input features and the emotional information contained in the audio features according to the set task.

6. The social risk perception method based on large models and multimodal data according to claim 3, characterized in that, In S6, the steps of inputting the collected data into the social risk perception model for risk prediction, and obtaining risk data and risk prediction reports for each event, include: New data is collected daily and analyzed one by one. First, the model extracts the summary, popularity, sentiment and time of each new data. Then, it queries the knowledge base through the data summary to identify the same events. By merging the same events, it tracks the trend of popularity and sentiment changes of the events based on the time sequence. For the events collected daily, the model will sort them according to popularity, level and sentiment changes, prioritize the analysis of the risk events and positive events with the highest popularity, and provide development trend predictions and coping strategies. The daily analysis results will be compiled into a risk prediction report, which includes a summary of the day's risk events, trends in sentiment and popularity, and corresponding coping strategies. The risk prediction report will also combine historical trends and changes in current data to assess potential future risk directions.

Images