Financial activity public opinion analysis method and system based on multi-modal data analysis

Abstract

The application discloses a financial activity public opinion analysis method and system based on multi-modal data analysis, relates to the technical field of multi-modal financial public opinion monitoring, and comprises the following steps: collecting a multi-modal public opinion data set related to a target financial activity, extracting four types of financial behavior structural features, namely, income description, risk expression, fund path and behavior guidance, through structured analysis; identifying invariant and variable features through a pre-constructed illegal financial activity template; respectively carrying out similarity and migration risk analysis to obtain two types of risk analysis results; and finally fusing the public opinion risk analysis results. The application solves the technical problem that traditional public opinion monitoring is difficult to cope with the evasion means of illegal financial activities, such as rhetoric migration, concept replacement and scene packaging, and the accuracy and timeliness of the analysis results are significantly insufficient, effectively breaks the evasion and identification behaviors of illegal financial activities, and improves the accuracy and timeliness of financial public opinion risk analysis.

Description

Technical Field

[0001] This invention relates to the field of multimodal financial public opinion monitoring technology, and in particular to a method and system for analyzing public opinion on financial activities based on multimodal data analysis. Background Technology

[0002] With the widespread application of big data resource services, public opinion analysis of illegal financial activities is crucial for risk prevention and control, and accurate identification of illegal financial public opinion is key to ensuring the stability of the financial market. Existing technologies largely rely on keyword recognition methods to filter and analyze financial public opinion data. However, illegal financial activities frequently employ tactics such as rhetoric shifting, concept substitution, and scenario packaging to evade keyword recognition during dissemination, rendering keyword sets ineffective in different contexts and significantly reducing the accuracy and timeliness of public opinion analysis results. These traditional methods are insufficient to meet the needs for accurate identification, timely early warning, and effective control of illegal financial public opinion. Summary of the Invention

[0003] This application provides a method and system for analyzing public opinion on financial activities based on multimodal data analysis. It solves the technical problem that traditional public opinion monitoring is unable to cope with the evasion tactics of illegal financial activities such as rhetoric shifting, concept replacement, and scenario packaging, resulting in a significant lack of accuracy and timeliness in the analysis results.

[0004] The first aspect of this application provides a method for analyzing public opinion on financial activities based on multimodal data analysis. The method includes: collecting a multimodal public opinion dataset related to a target financial activity; performing structured parsing on the multimodal public opinion dataset to extract multimodal financial behavior structural features, wherein the multimodal financial behavior structural features include at least return description structural features, risk expression structural features, funding path structural features, and behavior guidance structural features; identifying invariant-financial behavior structural features and variable-financial behavior structural features of the multimodal financial behavior structural features using a pre-constructed template of illegal financial activities; performing invariant structure similarity analysis on the invariant-financial behavior structural features to obtain a first risk analysis result; performing variable migration risk analysis on the variable-financial behavior structural features to obtain a second risk analysis result; and obtaining a public opinion risk analysis result based on the first risk analysis result and the second risk analysis result.

[0005] A second aspect of this application provides a financial activity public opinion analysis system based on multimodal data analysis. The system includes: a multimodal public opinion dataset acquisition module for collecting multimodal public opinion datasets related to target financial activities; a financial behavior structure feature acquisition module for performing structured parsing on the multimodal public opinion datasets to extract multimodal financial behavior structure features, which at least include return description structure features, risk expression structure features, funding path structure features, and behavior guidance structure features; a financial behavior structure feature identification module for identifying the invariant-financial behavior structure features and variable-financial behavior structure features of the multimodal financial behavior structure features using a pre-constructed illegal financial activity template; a risk analysis result acquisition module for performing invariant structure similarity analysis on the invariant-financial behavior structure features to obtain a first risk analysis result, and performing variable migration risk analysis on the variable-financial behavior structure features to obtain a second risk analysis result; and a public opinion risk analysis result acquisition module for obtaining public opinion risk analysis results based on the first risk analysis result and the second risk analysis result.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application collects multimodal public opinion data related to target financial activities, and obtains the structural features of financial behavior through structured parsing, timestamp alignment and fusion, and core feature extraction. It then uses an illegal financial activity template constructed based on historical samples to identify invariant and variable features, and conducts structural similarity analysis and migration risk analysis on both types of features. The results of these two risk analyses are then combined for comprehensive judgment, thereby accurately identifying public opinion risks related to illegal financial activities. This makes the analysis results of public opinion risks related to financial activities more comprehensive, accurate, and reliable, effectively addressing the evasion behaviors of illegal financial activities and improving the accuracy and timeliness of financial public opinion risk analysis. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a flowchart illustrating the financial activity public opinion analysis method based on multimodal data analysis provided in this application embodiment.

[0009] Figure 2 This is a schematic diagram of the structure of the financial activity public opinion analysis system based on multimodal data analysis provided in the embodiments of this application.

[0010] Figure labeling: Multimodal public opinion dataset acquisition module 1, financial behavior structure feature acquisition module 2, financial behavior structure feature identification module 3, risk analysis result acquisition module 4, public opinion risk analysis result acquisition module 5. Detailed Implementation

[0011] This application provides a method and system for analyzing public opinion on financial activities based on multimodal data analysis. It solves the technical problem that traditional public opinion monitoring is unable to cope with the evasion tactics of illegal financial activities such as rhetoric shifting, concept replacement, and scenario packaging, resulting in a significant lack of accuracy and timeliness in the analysis results.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0014] Example 1, as Figure 1 As shown, a method for analyzing public opinion on financial activities based on multimodal data analysis is described, wherein the method includes: Collect multimodal public opinion datasets related to target financial activities.

[0015] Specifically, the scope of data collection platforms is first clearly defined, covering mainstream domestic comprehensive news portals, social media platforms, forums and communities, video and live streaming platforms, local news media, local forums and communities, local self-media accounts, financial industry vertical platforms, recruitment platforms, as well as overseas social media platforms, instant messaging tools, and professional financial social platforms. At the same time, collection and screening rules are set based on the core risk terms and risk behavior-related expressions in the sensitive term library for illegal financial scenarios to ensure that only data related to financial behavior is collected. The construction process of the aforementioned sensitive term library for illegal financial scenarios will be explained in the following content.

[0016] Web crawling technology was used to extract text content from various platforms, including news articles, forum posts, social media posts, comments, job postings, and financial product promotional materials. During the crawling process, the text content was filtered in real time, and relevant expressions in a sensitive word library for illegal financial scenarios were matched. Text data related to financial behaviors, such as descriptions of returns, risk promises, fund transfers, and behavioral guidance, were extracted. The filtered text data was deduplicated to remove duplicate content, and text fragments containing personal privacy information such as personal names, ID numbers, and contact information were filtered out. Only text data directly related to financial behaviors, i.e., text modal sentiment data, was retained.

[0017] Image content related to financial activities was collected from various monitoring platforms using web crawling technology and platform interface calls. This included financial product promotional posters, event promotion images, images of fake approvals, and diagrams of wealth management plans. During the collection process, image recognition technology was used to analyze the image content, extract text information and pattern elements from the images, and match them with relevant expressions in the sensitive word library for illegal financial scenarios and visual features corresponding to typical financial behaviors. Image data involving the promotion of financial activities and financial activities were filtered out. Content involving personal privacy, such as personal facial features and private life scenes, in the images was blurred or removed. Focusing on information related to financial behavior, the final image modal public opinion data was obtained.

[0018] Video content related to financial behavior is collected from various monitoring platforms through compliant methods such as platform interface calls and screen recording. This includes short videos promoting financial products, live replays, and promotional videos for events. After collection, the audio tracks and keyframe images of the videos are extracted. The audio tracks are converted into text using speech recognition technology. Combined with the image recognition results of the keyframe images, relevant expressions in the sensitive word library of illegal financial scenarios and visual features of financial behavior are matched to filter out video data involving introductions to financial behaviors, promises of returns, fundraising, and behavioral guidance. Personal privacy information such as facial features, private addresses, and contact information contained in the videos is obscured or deleted, and video segments related to financial behavior are retained, i.e., video modal public opinion data.

[0019] We collect voice content from various monitoring platforms using compliant audio capture tools, including financial-related live broadcasts, podcasts, voice messages, and telemarketing recordings. After collection, we use speech recognition technology to convert the voice data into text data, match it with relevant expressions in a sensitive word library for illegal financial scenarios, and filter out voice data related to descriptions of financial behavior, profit promises, and explanations of fund transfers. We also filter out voice segments containing personal privacy information, i.e., voice modal public opinion data, to ensure that the collected voice data is only related to financial behavior.

[0020] Finally, the financial behavior-related public opinion data filtered from each modality are aggregated and formatted for storage according to preset standardized storage fields. The standardized storage fields include collection timestamp, public opinion publishing entity, public opinion dissemination platform, and dissemination sequence code. During the storage process, the data of each modality are associated and marked to ensure that the text, image, video, and audio data corresponding to the same public opinion event can be linked and queried through collection timestamp and dissemination sequence code. At the same time, the data content is verified again to ensure that no personal privacy information is left, forming a clean multimodal public opinion dataset related to financial behavior.

[0021] The multimodal public opinion dataset is subjected to structured analysis to extract multimodal financial behavior structural features. These features include at least return description structural features, risk expression structural features, funding path structural features, and behavior guidance structural features.

[0022] Optionally, firstly, structured parsing is performed on each modality of the multimodal public opinion dataset to obtain the corresponding text behavior features, image behavior features, video behavior features, and voice behavior features one by one. Then, cross-modal feature alignment and fusion are completed based on the collection timestamps corresponding to each feature to generate fused behavior features. Finally, multimodal financial behavior structure features are obtained by structured extraction of the fused behavior features. This step will be explained in detail in the following content.

[0023] Identify the invariant-financial behavior structural features and variable-financial behavior structural features of the multimodal financial behavior structural features by using a pre-constructed template of illicit financial activities.

[0024] In one embodiment of this application, firstly, historical instances of illegal financial activities corresponding to the target financial activity are obtained. Then, template-based identification is performed on the multimodal historical financial behavior structural feature samples corresponding to the samples. During the template-based identification process, the instance stability index of each historical financial behavior structural feature needs to be calculated. Then, feature samples that are greater than or equal to a preset instance stability threshold are marked as invariants, and feature samples that are less than the threshold are marked as variables. This completes the construction of the illegal financial activity template. Finally, the invariant-financial behavior structural feature and variable-financial behavior structural feature identification is achieved based on the template. This step will be described in detail in the following content.

[0025] An invariant structure similarity analysis is performed on the invariant-financial behavior structural features to obtain a first risk analysis result. A variable migration risk analysis is then performed on the variable-financial behavior structural features to obtain a second risk analysis result.

[0026] Specifically, firstly, financial behavior structural feature samples with values ​​greater than or equal to a preset instance stability threshold are marked as invariants, resulting in invariant-financial behavior structural feature samples. Based on the logical relationships of these samples, an invariant financial behavior structural diagram is constructed. Then, based on this structural diagram, invariant structural similarity is calculated for the target invariant-financial behavior structural features, yielding an invariant structural similarity value. Finally, this similarity value is compared with a preset structural similarity threshold to obtain the first risk analysis result.

[0027] Next, financial behavior structure feature samples that are less than the preset instance stability threshold are marked as variables to obtain variable-financial behavior structure feature samples. Then, the variable-financial behavior structure feature samples are traversed. If the traversal returns a non-empty result, the second risk analysis result is output based on the instance stability difference of the variable-financial behavior structure feature samples. The steps to obtain the first and second risk analysis results will be explained in detail in the following content.

[0028] Based on the results of the first risk analysis and the second risk analysis, the results of the public opinion risk analysis are obtained.

[0029] In one embodiment of this application, the core output formats of the first risk analysis result and the second risk analysis result are first clarified. The first risk analysis result is a qualitative conclusion of high-risk correlation and low-risk correlation, and is accompanied by a specific invariant structure similarity value, which takes a value of 0-1, reflecting the degree of fit between the core invariant features of the financial activity to be analyzed and the template of illegal financial activities. The second risk analysis result is a qualitative conclusion of high migration risk and low migration risk, and the stability of synchronous correlation instances is poor, which takes a value of 0-1, reflecting the fluctuation of variable features and the possibility of mutation and escalation of illegal financial activities. Both types of results are stored in a standardized format, including conclusion labels, quantitative indicators and key feature descriptions.

[0030] Next, a risk fusion decision matrix is ​​constructed. Based on the risk transmission logic of illegal financial activities, combination judgment rules are set to clarify the overall public opinion risk level corresponding to different risk outcome combinations: If the first risk analysis result is high-risk correlation and the second risk analysis result is high migration risk, the overall public opinion risk level is judged to be extremely high risk, indicating that the activity under analysis not only fits the typical structure of illegal finance, but also has characteristic variations and upgrades, and the risk transmission is strong; if the first risk analysis result is high-risk correlation and the second risk analysis result is low migration risk, it is judged to be high risk, meaning that the activity meets the core characteristics of illegal finance and has not shown obvious variations, and the risk attributes are clear; if the first risk analysis result is low-risk correlation and the second risk analysis result is high migration risk, it is judged to be medium-high risk, indicating that although the activity does not match the typical illegal structure, the variable characteristics fluctuate drastically, which may be the beginning of a new type of illegal financial model; if the first risk analysis result is low-risk correlation and the second risk analysis result is low migration risk, it means that the activity does not fit the typical illegal characteristics, and the variable characteristics do not fluctuate abnormally, and it does not currently have the risk attributes of illegal finance.

[0031] Subsequently, a second calibration is performed using the illegal financial monitoring knowledge base to further improve the accuracy of risk assessment. The database of typical illegal financial cases is consulted to compare the risk characteristics of the current risk portfolio with those of historical cases. If the variable characteristics in medium-to-high-risk scenarios highly match the early characteristics of a new type of illegal financial case, the risk level is upgraded to high risk. The compliance of risk characteristics is verified using the database of illegal financial laws and regulations. If the invariant characteristics in high-risk scenarios have clear legal basis, the high-risk assessment is reinforced. The attributes of involved entities are investigated through the illegal financial entity list database. If the involved entity is on the blacklist, all risk levels are upgraded by one level; if it is on the whitelist and the risk level is medium-to-high or lower, it is downgraded by one level, eliminating the risk of misjudging legitimate entities.

[0032] Finally, the final public opinion risk analysis results are generated, which include four core parts: First, a clear risk level, including extremely high risk, high risk, medium-high risk, and low risk; second, the basis for risk judgment, detailing the qualitative conclusions, quantitative indicators, and key characteristics of the first and second risks, such as the core invariant characteristics corresponding to high risk and the volatility variable characteristics corresponding to high migration risk; third, risk warnings, combining typical cases and investigation strategies from the illegal financial monitoring knowledge base to explain the possible transmission paths and scope of impact of the risks, such as the extremely high risk warning requiring a focus on the offline promotion and fund transfers of the involved entities; and fourth, handling recommendations, referring to the operational mechanisms of daily and special monitoring, proposing targeted measures such as clue verification, entity investigation, and information reporting, such as the recommendation to conduct on-site verification in conjunction with the economic crime investigation department for high risks, and to include medium-high risk entities in the gray list for continuous tracking.

[0033] By clarifying the attributes of risk outcomes, constructing a scientific decision-making matrix, relying on a knowledge base for secondary calibration, and generating a complete analysis report, the systemic integration of the two types of risk outcomes is achieved, ensuring the accuracy, logic, and operability of public opinion risk analysis results, and providing comprehensive support for early warning and precise handling of illegal financial activities.

[0034] Furthermore, the method provided in this application embodiment includes: The multimodal public opinion dataset is stored in a formatted manner using standardized storage fields; wherein, the standardized storage fields include the collection timestamp, the subject of the public opinion release, the public opinion dissemination platform, and the dissemination sequence code.

[0035] Specifically, the first step is to preprocess the multimodal public opinion data obtained after screening, removing redundant information and incorrectly formatted content, and unifying the file format of each modality. Text data is stored in JSON format, image data is uniformly converted to PNG format, video data is converted to MP4 format, and audio data is standardized to WAV format, ensuring that different modalities of data have a unified processing foundation.

[0036] Next, the standardized storage field step is performed, and the specific steps are as follows: Step a: Extract the collection time information of each modality of public opinion data. Based on the server standard time at the time of data collection, format the timestamp according to the ISO 8601 international standard and use the representation method of "year-month-day T hour: minute: second + time zone". Convert the time records of different collection channels into this standard format to form a standardized collection timestamp field to ensure the consistency and traceability of time information.

[0037] Step b: Identify the publishing entity information of each modality of public opinion data. Extract the author account ID and nickname from text data, and extract the unique identifier and related name information of the uploading account from image, video and audio data. Standardize the extracted entity information by removing special characters, spaces and invalid symbols, and uniformly adopt the combination format of "account ID-entity name" to form the public opinion publishing entity field, so as to realize the unified identification of the same publishing entity in different modal data.

[0038] Step c: Record the data collection platforms for each modality of public opinion data, establish a standardized coding system for dissemination platforms, assign a unique digital code to each monitoring platform, and associate the platform name and subdivided channel information. Map the collected platform names to the corresponding standard codes to form the public opinion dissemination platform field, ensuring the standardized storage and rapid retrieval of dissemination source information.

[0039] Step d: Track the propagation path of public opinion data of each modality. Starting from the original release data, assign an initial code to the original release node of each public opinion event. Each subsequent forwarding or propagation behavior generates a unique derivative code. The code contains the original release identifier, propagation level and node sequence information. By linking the original data with the data of each propagation node through the code, a propagation sequence code field is formed to realize the complete record of the public opinion propagation trajectory.

[0040] Next, a mapping relationship is established between standardized storage fields and multimodal public opinion data. The collection timestamp, the entity publishing the public opinion, the platform for public opinion dissemination, the dissemination sequence code, and the corresponding multimodal data file path are bound to form structured metadata records. Then, a hierarchical storage architecture is adopted to format and store the data. High-frequency access to recent data is stored in the hot storage layer using SSD storage media; medium-term data is stored in the warm storage layer using HDD storage media; and long-term historical data is stored in the cold storage layer using archive storage media. Metadata records are stored using an Iceberg table structure, supporting efficient querying and related retrieval based on standardized fields.

[0041] Finally, after storage is completed, the data integrity is verified. The four standardized storage fields corresponding to each multimodal public opinion data are checked to see if they are complete and whether the field format conforms to the preset specifications. This ensures that there are no missing fields or format errors. At the same time, a data access audit mechanism is established to record data read and write operations and ensure the traceability and security of stored data.

[0042] Furthermore, the method provided in this application embodiment includes: The multimodal public opinion dataset includes at least text modal public opinion data, image modal public opinion data, video modal public opinion data, and voice modal public opinion data. The multimodal public opinion dataset is subjected to structured parsing to obtain text behavior features, image behavior features, video behavior features, and voice behavior features. These text behavior features, image behavior features, video behavior features, and voice behavior features are then aligned and fused according to their collection timestamps to obtain fused behavior features. The fused behavior features are then subjected to structured extraction to extract multimodal financial behavior structure features.

[0043] Optionally, a sensitive word database for illegal financial scenarios is first constructed. Based on publicly available illegal financial cases, regulatory rules, and industry risk reports, core risk terms and related expressions of risky behaviors are compiled. Core risk terms include expressions such as guaranteed principal, high returns, and risk-free rebates. Expressions of risky behaviors include concentrated fund transfers, referral rebates, and tiered profit sharing. The compiled sensitive words are classified according to semantic categories, and the sensitive word set is stored using a prefix tree structure. An efficient matching index is constructed to ensure that relevant sensitive information can be quickly located during subsequent parsing.

[0044] Next, according to the multi-modal public opinion data set collected in the foregoing steps, which at least covers financial-related public opinion data of four core modalities including text, image, video and voice, the structural analysis of these four types of public opinion data is as follows: When performing structural analysis on text-modal public opinion data, first use the accurate mode of the jieba word segmentation tool to segment the text data, construct a directed acyclic graph based on the prefix dictionary and find the maximum probability path through dynamic programming to split the continuous text into independent lexical units. Then, by loading the custom stop word list, filter out meaningless words such as "de" (的) and "shi" (是) and punctuation marks. When retrieving sensitive words using the prefix tree matching algorithm, first insert all the words in the illegal financial scenario sensitive word library into the prefix tree one by one in character order, and each node stores the corresponding character and a flag indicating whether it is the end of a word. After construction, use the segmented lexical units as input, and traverse character by character from the root node of the prefix tree. If the character path can reach a node marked as the end of a word, it is determined that a sensitive word is hit, and then filter out the text fragments containing sensitive words.

[0045] Then use the named entity recognition model based on BERT to extract key entities. Select bert-base-chinese as the pre-trained model, and use the annotation samples in the illegal financial monitoring knowledge base as training data. This sample contains annotation labels of target entities such as income amount, rate of return, investment term, etc. When training the model, set the batch size to 32, the learning rate to 2e-5, and the number of iterations to 30 rounds. Use the cross-entropy loss function to optimize the model parameters, input the filtered text fragments into the fine-tuned model, and output the corresponding key entity information. Finally, extract the association relationships between entities based on preset rules, including the corresponding relationship between income amount and promised term, the association relationship between the direction of capital flow and the subject, etc., and integrate the entity information and association relationships to form text behavior characteristics.

[0046] When performing structural analysis on image-modal public opinion data, use the Tesseract OCR technology to extract the text information in the image, complete the denoising and normalization processing by removing the incorrect characters corresponding to the blurred pixels and correcting the misrecognition results of similar-shaped characters, and then compare the normalized text with the illegal financial scenario sensitive word library word by word to filter out the images containing risk-related text. Select ResNet50 as the basic convolutional neural network architecture to construct a visual feature extraction model. This model sequentially includes a convolutional layer with a 7×7 convolutional kernel, a max pooling layer with a stride of 2, 4 groups of residual blocks, each group of residual blocks contains 3, 4, 6, 3 residual units respectively, a global average pooling layer and a fully connected layer. Each residual unit consists of two convolutional layers with 3×3 convolutional kernels, a batch normalization layer and a ReLU activation function, and realizes the residual mapping of input features and output features through shortcut connections to ensure the gradient transmission of the deep network.

[0047] During the model training phase, labeled image samples from the illegal financial monitoring knowledge base were used as training data. These samples covered various illegal financial-related images, including financial product promotional posters, images of fake approvals, and diagrams of wealth management schemes. Each sample was labeled with a corresponding visual feature category, such as displays of returns, illustrations of fund flows, and illegal symbols. The training samples were preprocessed by normalizing all image sizes to 224×224 pixels, converting them to RGB three-channel format, and standardizing pixel values ​​according to a mean of [0.485, 0.456, 0.406] and a variance of [0.229, 0.224, 0.225]. The model is initialized by loading pre-trained weights from the ImageNet dataset. The parameters of the first 80% of convolutional layers are frozen to reduce training computation, and only the parameters of the last 20% of convolutional and fully connected layers are trained. The batch size is set to 32, the initial learning rate to 1e-4, and the number of training iterations to 50 epochs. The Adam optimizer and cross-entropy loss function are used to minimize the model's prediction error. An early stopping strategy is introduced: training stops when the validation set loss does not decrease for five consecutive epochs to avoid overfitting. The model's input is a pre-processed 224×224 pixel RGB image, and the output is a 2048-dimensional visual feature vector. The vector elements correspond to the quantitative expression of core visual features in the image, such as the display style of profit figures, the diagram of fund flow, and typical illegal financial symbols. Finally, the sensitive text information obtained from OCR matching is associated and integrated with the model's output visual feature vector. Feature dimension alignment achieves mutual verification between text information and visual features, forming image behavior features containing risk-related text and corresponding visual representations.

[0048] When performing structured analysis on video modal sentiment data, a fixed time interval sampling method is adopted, extracting one keyframe from the video every second. The extracted keyframes are then subjected to the structured analysis process for image modal sentiment data, namely, extracting text information using Tesseract OCR technology and matching it with a sensitive word library for illegal financial scenarios, and extracting visual features using a ResNet50 convolutional neural network to obtain image-related features corresponding to each keyframe. Independent audio tracks are extracted from the video using an audio separation tool. These audio tracks are preprocessed to remove background noise, unify the sampling rate to 16kHz, and convert to mono WAV format. The open-source CMU Sphinx speech recognition tool is used to convert the preprocessed audio tracks. This tool uses built-in acoustic and language models to perform acoustic feature matching and semantic decoding of the audio signal, directly outputting the corresponding text data. The converted text data is then subjected to the structured analysis process for text modal sentiment data, namely, jieba word segmentation, stop word filtering, prefix tree sensitive word matching, fine-tuned BERT named entity recognition, and entity relationship extraction to obtain text-related features. Based on timestamps, the image features of keyframes within the same time segment are matched one-to-one with the text features converted from audio in chronological order. After removing duplicate and related information, video behavior features are formed.

[0049] When performing structured analysis on voice modal public opinion data, the same speech recognition technology described above is used to convert the speech data into text data. The converted text data is then processed through word segmentation and stop word removal. Similarly, a prefix tree matching algorithm is used to retrieve sensitive words from the text, filtering out text fragments containing risk-related information. Information such as profit descriptions, risk commitments, funding paths, and behavioral guidance is extracted from these text fragments to form voice behavior features.

[0050] Then, text behavior features, image behavior features, video behavior features, and speech behavior features obtained from each modality are collected. The acquisition timestamps corresponding to each feature are extracted synchronously and organized into independent timestamp sequences for each modality. A timestamp interpolation alignment algorithm is used for cross-modal feature association matching: first, a unified time granularity of 1 second is set as the alignment benchmark. The timestamp sequences of each modality are traversed. For modalities that do not have corresponding features at the benchmark time point, linear interpolation is used to estimate the feature value at the benchmark time point based on the feature values ​​corresponding to the two adjacent valid timestamps of that modality. All modal features are then uniformly mapped to the benchmark time axis. Subsequently, using the benchmark time point as the unit, different modal features with completely identical timestamps or a time difference ≤ 0.5 seconds are selected to complete the association matching of features with the same or similar time dimensions.

[0051] Then, a weighted fusion strategy was adopted for the aligned multimodal features. First, the confidence of each modality feature was calculated: the text modality confidence was the weighted average of the sensitive word matching accuracy and the BERT named entity recognition accuracy, with both accuracy rates having a weight of 0.5. The matching accuracy was calculated as the ratio of the number of correctly matched sensitive words to the total number of matches, and the recognition accuracy was calculated as the ratio of the number of correctly recognized entities to the total number of recognized entities. The image modality confidence was the weighted average of the OCR text matching accuracy and the ResNet visual feature recognition accuracy, with both accuracy rates having a weight of 0.5. The matching accuracy was the percentage of correctly matched OCR-extracted text to the sensitive word library, and the recognition accuracy was the percentage of samples in which the model correctly recognized the visual feature category. The video modality confidence was the average of the keyframe image feature confidence and the audio-to-text feature confidence. The speech modality confidence was the weighted average of the ASR speech conversion accuracy and the converted text recognition accuracy, with both accuracy rates having a weight of 0.5. The conversion accuracy was the ratio of the length of the correctly converted speech segment to the total audio length. The confidence scores of each modality are then normalized. The normalized values ​​are the fusion weights of the corresponding modal features. The aligned modal features are then weighted and summed according to these weights to obtain the fused behavioral features.

[0052] Finally, the characteristics of the integration behavior are extracted in a structured manner. According to fixed dimensions such as whether the return description structure is committed, the location of the commitment, and the strength of the commitment; whether the risk expression structure appears, the location of its appearance, and the degree of weakening; whether the funding path structure is closed, private, or circumvents regulation; and whether the behavior guidance structure exists, its urgency, and the order of guidance, the corresponding feature information is extracted from the integration behavior characteristics one by one to form a standardized multimodal financial behavior structure feature. This feature includes at least the return description structure feature, the risk expression structure feature, the funding path structure feature, and the behavior guidance structure feature.

[0053] By constructing a sensitive word database as a preliminary aid, and combining the above steps, efficient analysis and accurate feature extraction of multimodal public opinion data were achieved, improving the efficiency and targeting of multimodal financial behavior structural feature extraction.

[0054] Furthermore, the method provided in this application embodiment includes: Obtain historical instances of illegal financial activities samples of the target financial activity, perform template-based identification on the historical instances of illegal financial activities samples, and construct the illegal financial activity template; wherein, the template-based identification includes performing instance stability calculation on the multimodal historical financial behavior structural feature samples corresponding to the historical instances of illegal financial activities samples, and obtaining the instance stability index of each historical financial behavior structural feature; financial behavior structural feature samples that are greater than or equal to a preset instance stability threshold are marked as invariants, and financial behavior structural feature samples that are less than the preset instance stability threshold are marked as variables.

[0055] Specifically, the first step is to construct an illegal financial monitoring knowledge base. This knowledge base is the core source of historical examples of illegal financial activities and includes five sub-bases. The construction process of each sub-base is as follows: The illegal financial laws and regulations database is constructed by collecting administrative regulations, criminal law provisions, judicial interpretations, and policy documents from financial regulatory departments; the illegal financial typical case database collects typical cases of illegal finance and illegal fundraising, traces data clues, analyzes and summarizes pattern characteristics, and keeps it dynamically updated; the key monitoring media list database sorts out and classifies information from domestic mainstream media platforms, local media forums, industry vertical platforms, and overseas social media platforms; the illegal financial entity list database is constructed by dividing the database into a blacklist database marking known or highly suspected illegal financial information release entities, a whitelist database including licensed financial institutions and official information disclosure platforms, and a graylist database including clues to be verified and imported clues; and the illegal financial investigation strategy and sensitive word database is built around daily monitoring and special investigations, establishing a scenario-based sensitive word database and a strategy database including noise reduction, clustering, and tagging methods.

[0056] Next, based on the completed illegal financial monitoring knowledge base, historical examples of illegal financial activities were obtained as samples. A combination of keyword matching and manual screening was used, with illegal financial investigation strategies and sensitive words in the sensitive word database as the search criteria. Multimodal public opinion clues corresponding to the cases were extracted from the illegal financial typical case database, including text, image, video, and audio data. At the same time, the legality of the cases was verified with the help of the illegal financial laws and regulations database. The sample dimensions were supplemented by associating the involved entities through the illegal financial entity list database. After deduplication, a standardized sample of historical examples of illegal financial activities was formed.

[0057] Then, the historical instances of illegal financial activities are templated for identification. First, the multimodal historical financial behavior structural feature samples corresponding to each sample are extracted. Then, the instance stability index is calculated according to the preset method. Specifically, the instance stability index is obtained by analyzing the frequency of feature occurrence and the consistency of feature value distribution of each historical financial behavior structural feature and weighting these two indicators. This step will be explained in detail in the following content.

[0058] Next, a preset instance stability threshold is set. The process of setting the preset instance stability threshold is as follows: First, a sufficient number of historical instances of illegal financial activities are extracted from the typical cases of illegal financial activities in the illegal financial monitoring knowledge base. The number of samples is no less than 1,000, covering various illegal financial scenarios such as digital currency, pension financial management, and digital collectibles. The training set and the validation set are randomly divided in a 3:1 ratio. The training set is used to initially calculate the instance stability index of each financial behavior structural feature, including the weighted calculation results of feature occurrence frequency and change coefficient. The validation set is used to verify the threshold effect.

[0059] Next, based on the instance stability index distribution of the training set, a candidate threshold range of 0.5-0.9 was determined, with a step size of 0.05, resulting in 9 candidate values ​​to match the stability index range of 0-1. Then, using Python's sklearn library, each candidate threshold was applied sequentially to the validation set. For each threshold, the accuracy and recall of the features marked as invariants in the validation set were statistically analyzed: the proportion correctly identified as core features of illegal financial activities, and the proportion of all genuine core features of illegal financial activities successfully labeled. The F1 score was calculated using the formula F1 = 2 × (accuracy × recall) / (accuracy + recall). Finally, the relationship curve between the candidate threshold and the corresponding F1 score was plotted using the matplotlib library, and the threshold corresponding to the highest F1 score in the curve was selected as the base threshold.

[0060] Then, fine-tuning was performed based on industry experience in identifying features of illegal financial activities. If the base threshold was below 0.65, the instance stability index of core stable features such as "guaranteed principal" and "private transfers" in typical cases of illegal financial activities was referenced, which is usually not lower than 0.7. The threshold was then adjusted to the 0.65-0.7 range. If the base threshold was above 0.85, the possibility of feature variation in new types of illegal financial activities needed to be considered, and the threshold was adjusted to the 0.75-0.8 range. Finally, the stability of the threshold was verified through 5-fold cross-validation. The adjusted threshold was applied to 5 different training and validation sets. If the F1 score fluctuation of each set did not exceed 5%, the value was determined as the final preset instance stability threshold. Otherwise, the candidate threshold range step size was re-optimized, and the above process was repeated to ensure that the threshold could stably and accurately screen the core features of illegal financial activities under different data partitions.

[0061] Furthermore, financial behavior structural feature samples with values ​​greater than or equal to a preset instance stability threshold are labeled as invariant-financial behavior structural features, while those with values ​​less than the threshold are labeled as variable-financial behavior structural features. The two labeling results are then integrated to construct an illegal financial activity template. This template explicitly stores the core attributes of the invariant features and their fixed logical relationships, while also recording the common value ranges and fluctuation types of the variable features. During identification, the multimodal financial behavior structural features to be analyzed are first compared dimension-by-dimensionally with the invariant features in the template, calculating the matching degree of the core attributes of the features. If the matching degree is ≥0.7, it is determined to be an invariant-financial behavior structural feature. For features that do not meet the invariant matching condition, they are further compared with the value range and type of the variable features in the template. If the feature type belongs to the variable category recorded in the template and the value is within the allowable fluctuation range, it is determined to be a variable-financial behavior structural feature, thus clearly completing the identification of the two types of features.

[0062] By constructing a complete knowledge base for monitoring illegal financial activities and obtaining effective samples, and by combining feature stability calculation and threshold labeling to construct templates, we have achieved accurate identification of invariants and variables in the structural features of multimodal financial behavior, providing reliable template support for subsequent risk analysis.

[0063] Furthermore, the method provided in this application embodiment includes: The frequency of occurrence and the consistency of feature value distribution of each historical financial behavior structural feature in the multimodal historical financial behavior structural feature sample are analyzed; a weighted calculation is performed based on the frequency of occurrence and the consistency of feature value distribution to obtain the instance stability index.

[0064] Specifically, firstly, from the historical examples of illegal financial activities obtained from the illegal financial monitoring knowledge base, all multimodal historical financial behavior structural feature samples are extracted. Each independent financial behavior structural feature is used as a separate calculation unit, and the feature type is determined to be either numerical or categorical. Numerical features include quantifiable indicators such as the amount of profit, rate of return, and investment period, while categorical features include qualitative indicators such as the type of funding path, the method of guiding behavior, and the form of risk expression, forming a feature set to be calculated.

[0065] Next, the frequency of feature occurrence is calculated, with values ​​ranging from 0 to 1. A value closer to 1 indicates that the feature is more prevalent in the sample. To ensure consistency in feature value distribution, processing is performed based on the classification of numerical and categorical features; the specific calculation steps will be explained in detail later.

[0066] Next, weighting coefficients for feature frequency and feature value distribution consistency were determined. Based on industry experience in identifying illegal financial features, the weight for feature frequency was set to 0.4, and the weight for feature value distribution consistency was set to 0.6. This weighting allocation can be confirmed and adjusted through testing on a small validation set. An instance stability index was calculated using a weighted summation method. The specific formula is: Instance Stability Index = Feature Frequency × 0.4 + Feature Value Distribution Consistency × 0.6. This yields the instance stability index for each historical financial behavior structural feature. The index value ranges from 0 to 1; a higher value indicates stronger instance stability.

[0067] By clearly defining the feature calculation unit, calculating the frequency of feature occurrence and the consistency of value distribution separately, and then obtaining the instance stability index through weighted summation, the stability of each historical financial behavior structural feature instance is accurately quantified, providing a reliable basis for the labeling of invariants and variables in the subsequent construction of illegal financial activity templates.

[0068] Furthermore, the method provided in this application embodiment includes: The frequency of feature occurrence is obtained by calculating the ratio of the number of samples containing the feature to the total number of instance samples, and the consistency of feature value distribution is obtained by calculating the variation coefficient of the feature in different instance samples.

[0069] Specifically, the data of the multimodal historical financial behavior structural feature samples corresponding to the historical instances of illegal financial activities is first standardized. Using the Python pandas library, which is well known to those skilled in the art, all samples are organized into a structured data table, where each row corresponds to a historical instance of illegal financial activity and each column corresponds to a multimodal historical financial behavior structural feature sample. For numerical features, specific values ​​are directly filled in, and for categorical features, they are converted into quantifiable encoded values. At the same time, abnormal samples with a data missing rate of more than 50% are removed to ensure that the sample data used in the calculation has completeness and validity.

[0070] Next, the frequency of feature occurrence is calculated. Based on the normalized structured data table, the columns of each multimodal historical financial behavior structural feature sample are traversed and counted. The `sum` function of the pandas library is used to count the number of cells with valid feature values ​​in the column, i.e., the number of samples containing that feature. Then, the `len` function is used to obtain the total number of rows in the data table, i.e., the total number of instance samples. The frequency of occurrence of each feature is obtained by dividing the number of samples containing that feature by the total number of instance samples. The result ranges from 0 to 1, with a larger value indicating that the feature appears more frequently in the samples.

[0071] To ensure consistency in the distribution of feature values, corresponding statistical methods are used based on the feature type: For each numerical feature, the mean of all valid values ​​for that feature is first calculated using the `mean` function from the NumPy library, and then the standard deviation of those values ​​is calculated using the `std` function from the NumPy library. The coefficient of variation for that feature is obtained by dividing the calculated standard deviation by the mean. The coefficient of variation measures the degree of dispersion of the data; the smaller the value, the less the feature's value fluctuates across different samples, and the higher the consistency of the distribution. For the special case where the mean is 0, the coefficient of variation is directly set to 0, representing completely consistent values.

[0072] For categorical features, the Counter function in the collections library is used to count the occurrences of all values ​​of the feature. The mode with the highest frequency is found. The frequency of the mode is divided by the number of samples with values ​​for the feature to obtain the mode percentage. The higher the mode percentage, the more uniform the categorical values ​​are and the stronger the consistency. Finally, the reciprocal of the coefficient of variation or the mode percentage is normalized to the interval between 0 and 1 as an indicator of the consistency of feature value distribution.

[0073] After calculating the frequency of occurrence and coefficient of variation of all features, the results are stored in association according to the feature name, forming a structured result table containing feature name, frequency of occurrence, and coefficient of variation, providing basic data support for weighted calculation to obtain instance stability index.

[0074] By employing a data processing library and standardized statistical methods, and through steps such as data normalization, frequency ratio calculation, and coefficient of variation calculation, the system achieves accurate quantification of the frequency of feature occurrence and the consistency of value distribution, providing reliable and easily reproducible core data for the calculation of instance stability indicators.

[0075] Furthermore, the method provided in this application embodiment includes: Financial behavior structural feature samples with values ​​greater than or equal to a preset instance stability threshold are labeled as invariants to obtain invariant-financial behavior structural feature samples; an invariant financial behavior structural diagram based on the logical relationship of the invariant-financial behavior structural feature samples is constructed; invariant structural similarity is calculated on the invariant-financial behavior structural features based on the invariant financial behavior structural diagram to obtain invariant structural similarity values; the invariant structural similarity values ​​are compared with a preset structural similarity threshold to obtain the first risk analysis result.

[0076] In one embodiment, firstly, based on a preset instance stability threshold, samples of financial behavior structure features greater than or equal to the threshold are selected and marked as invariants to obtain invariant-financial behavior structure feature samples. The selection strictly follows a fixed-dimensional approach, extracting relevant information from the integrated behavioral features regarding the benefit description structure, risk expression structure, funding path structure, and behavioral guidance structure. The benefit description structure focuses on three dimensions: whether a commitment is made, the location of the commitment, and the strength of the commitment. The risk expression structure revolves around three dimensions: whether it occurs, the location of its occurrence, and the degree of weakening. The funding path structure covers three dimensions: whether it is closed, whether it is private, and whether it circumvents regulation. The behavioral guidance structure clarifies three dimensions: existence, urgency, and guidance sequence. These samples are all core features that have consistently appeared in historical instances of illegal financial activities. The core attributes of each feature are standardized and encoded according to unified rules, and the correlation information between features is recorded, organized into a structured data table to form a basic dataset for constructing a structure diagram.

[0077] Next, a graph modeling approach is employed, utilizing Python's networkx library to construct an invariant financial behavior structure graph. Each structure's subdivided dimensional features serve as the core nodes of the graph, with each node's attributes including feature name, structure type, subdivided dimensional label, and core value. The inherent logical relationships between features are used as directed edges in the graph. By combining the typical operational logic of illicit financial activities, causal relationships, process relationships, and other relationship types are established, clearly presenting the complete logical chain between the core features of illicit financial activities. This transforms discrete subdivided dimensional features into a visually compelling structure graph with strong logical connections, clearly revealing the inherent relationships between the invariant features of illicit financial activities.

[0078] Then, based on the constructed invariant financial behavior structure diagram, the invariant structure similarity is calculated in three steps. The first step calculates the structural node matching degree. The invariant financial behavior structure features to be analyzed are transformed into a set of nodes to be matched. Using the cosine similarity algorithm, the attributes of each node (such as feature name, type, and value range) are converted into numerical vectors. The vector similarity between each node to be matched and the corresponding node in the template structure diagram is calculated, and the average similarity value of all nodes is taken as the structural node matching degree, ranging from 0 to 1. A higher value indicates a more accurate matching of node attributes. The second step calculates the consistency of node connections. All directed edges in the template structure diagram are traversed, and the number of edges in the graph corresponding to the features to be analyzed that are completely consistent with the template edges in terms of association type and connection direction is counted. This number is divided by the total number of edges in the template structure diagram to obtain the node connection consistency index, also ranging from 0 to 1, reflecting the degree of logical fit between the two. The third step is to calculate the overall topological similarity. An improved Jaccard similarity algorithm is used, which divides the number of intersections of the edge sets of the two graphs by the number of unions of the edge sets, and then combines the weight correction of the node matching degree to obtain the overall topological similarity. This index focuses on reflecting the consistency of the overall structural layout of the two graphs.

[0079] Next, the structural node matching degree, node connection consistency, and overall topological similarity calculated in the above three steps are weighted and fused to determine the weight allocation of each indicator: structural node matching degree has a weight of 30%, node connection consistency has a weight of 30%, and overall topological similarity has a weight of 40%. The values ​​of each indicator are multiplied by their corresponding weights and then summed to obtain the final invariant structural similarity value, which ranges from 0 to 1. A higher value indicates a higher degree of fit between the feature to be analyzed and the template invariant structure.

[0080] Referring to the method for setting the preset instance stability threshold, a preset structural similarity threshold is determined. A subset of data is extracted from typical case samples in the illegal financial monitoring knowledge base as a validation set. Different candidate thresholds from 0.6 to 0.85 are applied sequentially to the validation set, and the accuracy, recall, and F1 score for risk identification are calculated at each threshold. The threshold corresponding to the highest F1 score is selected as the final preset structural similarity threshold. The calculated invariant structural similarity value is compared with this preset structural similarity threshold. If the invariant structural similarity value is greater than or equal to the preset threshold, the first risk analysis result is determined to be a high-risk association, indicating a high degree of fit between the core invariant structure of the financial activity to be analyzed and the template of illegal financial activities. If the invariant structural similarity value is less than the preset threshold, the first risk analysis result is determined to be a low-risk association, indicating insufficient matching of the core invariant structure.

[0081] By standardizing sample processing, graph structure modeling, multi-dimensional similarity calculation and weighted fusion, combined with threshold comparison, accurate similarity analysis of invariant-financial behavior structural features was achieved, providing scientific and reproducible technical support for the preliminary risk assessment of illegal financial activities.

[0082] Furthermore, the method provided in this application embodiment includes: The financial behavior structure feature samples that are less than the preset instance stability threshold are labeled as variables to obtain variable-financial behavior structure feature samples; if the variable-financial behavior structure feature samples are not empty after traversing them, the second risk analysis result is output based on the instance stability average difference of the variable-financial behavior structure feature samples.

[0083] Optionally, firstly, using a preset instance stability threshold as the screening criterion, features with instance stability indices lower than this threshold in all historical financial behavior structure feature samples are labeled as variables, obtaining variable-financial behavior structure feature samples. The screening process uses Python's pandas library to load the feature sample dataset, and conditional statements are used to filter feature records with index values ​​lower than the threshold. The screening results are then organized into a standardized variable feature sample set to ensure that the sample data format is consistent and can be directly used for subsequent analysis.

[0084] Next, Python's loop iteration function is used to iterate through the variable-financial behavior structure feature sample set, checking one by one whether there is a valid feature record in the sample set. If at least one valid variable feature data is detected in the sample set during the iteration, it is determined that the iteration returns non-empty, and the calculation of the instance stability average is entered; if no valid feature record is detected, it is processed according to the subsequent preset rules. Here, we focus on the scenario where the iteration returns non-empty.

[0085] Next, the mean deviation of instance stability for the variable-financial behavioral structural feature samples is calculated: First, the mean function of the NumPy library is used to calculate the average instance stability index of all variable features in the sample set. Then, the absolute difference between the instance stability index of each variable feature and this average is calculated by iteratively calculating the average. Finally, all absolute differences are summed and divided by the total number of variable features to obtain the mean deviation of instance stability. The larger the mean deviation value, the more volatile the stability of the variable feature and the higher the migration risk; conversely, the smaller the mean deviation value, the lower the migration risk.

[0086] A risk assessment threshold for the average instance stability deviation is set, using a method consistent with the preset instance stability threshold: A subset of data is extracted from typical case samples in the illegal financial monitoring knowledge base as a validation set. Candidate thresholds ranging from 0.1 to 0.3 are applied sequentially to the validation set, and the precision, recall, and F1 score for migration risk identification are calculated for each candidate threshold. The threshold corresponding to the highest F1 score is selected as the final risk assessment threshold. The calculated average instance stability deviation is compared with this risk assessment threshold. If the deviation is greater than or equal to the risk assessment threshold, the second risk analysis result is output as high migration risk, indicating drastic fluctuations in variable characteristics, which may be new characteristics of iterative escalation of illegal financial activities. If the deviation is less than the risk assessment threshold, the second risk analysis result is output as low migration risk, indicating smooth fluctuations in variable characteristics and no obvious signs of risk migration.

[0087] By standardizing variable sample screening, traversing verification, statistically calculating the mean difference and comparing thresholds, we have achieved accurate quantitative analysis of the risk of variable-financial behavior structural feature migration, providing comprehensive variable feature risk support for the overall public opinion risk analysis results.

[0088] Furthermore, the method provided in this application embodiment includes: If the traversal of the variable-financial behavior structure feature sample returns empty, the stability of the updated calculation instance based on the variable-financial behavior structure feature is poor; based on the stability of the updated instance, the second risk analysis result is output.

[0089] In one embodiment, Python's loop traversal function is first used to traverse and check the variable-financial behavior structure feature sample set. If no valid variable feature record is detected, it is determined that the traversal returns empty, and at this time the variable-financial behavior structure feature update process is started.

[0090] The variable feature update uses the illegal financial monitoring knowledge base as the core data source and adopts an incremental supplementation method to obtain new samples: Newly included illegal financial cases are extracted from the typical illegal financial case database. Using the previously described multimodal data structuring analysis method, the text, image, video, and audio-based public opinion clues corresponding to the cases are analyzed to extract the structural features of financial behavior. Features whose stability does not reach a preset threshold are selected using illegal financial investigation strategies and a sensitive word database as new variable feature samples. Simultaneously, the entities involved in the new cases are linked to the illegal financial entity list database to supplement the historical variable features related to these entities. All new samples are then organized in a unified format and added to the original variable-financial behavior structural feature sample set to form the updated variable feature sample set.

[0091] Next, the mean deviation of instance stability is calculated for the updated variable-financial behavior structure feature sample set: the mean function of the NumPy library is used to calculate the average value of the instance stability index of all variable features in the sample set. By iteratively calculating the absolute difference between the instance stability index of each variable feature and this average value, the sum of all absolute differences is divided by the total number of updated variable features to obtain the updated mean deviation of instance stability. This value directly reflects the degree of fluctuation of the updated variable features.

[0092] The risk assessment threshold for the instance stability mean difference determined in the previous steps is then used without resetting it to ensure consistency of the assessment criteria. The updated instance stability mean difference is compared with this risk assessment threshold. If the mean difference is greater than or equal to the risk assessment threshold, the second risk analysis result is output as high migration risk, indicating that the newly added variable features have brought significant fluctuations and may be new variant features of illegal financial activities. If the mean difference is less than the risk assessment threshold, the second risk analysis result is output as low migration risk, indicating that the updated variable features still maintain gentle fluctuations and no obvious signs of risk migration have appeared.

[0093] By updating variable features on empty sample sets based on the illegal financial monitoring knowledge base, and combining standardized mean difference calculation and threshold comparison, accurate analysis of variable migration risk is achieved in scenarios where the traversal returns empty, ensuring the comprehensiveness and continuity of variable feature risk assessment.

[0094] In summary, the financial activity public opinion analysis method based on multimodal data analysis provided in this application has the following technical effects: This application collects a multimodal public opinion dataset of target financial activities, extracts structural features of various financial behaviors through structured analysis, identifies invariants and variables based on illegal financial activity templates, conducts structural similarity and migration risk analysis, integrates the results of the two types of risks to adjust and optimize, accurately obtains public opinion risk analysis results, improves the accuracy and reliability of identifying clues to illegal financial activities, and achieves the technical effect of effectively cracking the evasion behaviors of illegal financial activities and improving the accuracy and timeliness of financial public opinion risk analysis.

[0095] Example 2, as Figure 2 As shown, based on the same inventive concept as in Embodiment 1 above, this application provides a financial activity public opinion analysis system based on multimodal data analysis, the system comprising: Multimodal public opinion dataset acquisition module 1, which is used to collect multimodal public opinion datasets related to the target financial activities.

[0096] The financial behavior structure feature acquisition module 2 is used to perform structured analysis on the multimodal public opinion dataset and extract multimodal financial behavior structure features. The multimodal financial behavior structure features include at least return description structure features, risk expression structure features, funding path structure features, and behavior guidance structure features.

[0097] The financial behavior structure feature identification module 3 is used to identify the invariant-financial behavior structure features and variable-financial behavior structure features of the multimodal financial behavior structure features through a pre-constructed template of illegal financial activities.

[0098] The risk analysis result acquisition module 4 is used to perform invariant structure similarity analysis on the invariant-financial behavior structural features to obtain a first risk analysis result, and to perform variable migration risk analysis on the variable-financial behavior structural features to obtain a second risk analysis result.

[0099] The public opinion risk analysis result acquisition module 5 is used to acquire public opinion risk analysis results based on the first risk analysis result and the second risk analysis result.

[0100] Furthermore, the financial behavior structure feature acquisition module 2 is used to perform the following steps: The multimodal public opinion dataset includes at least text modal public opinion data, image modal public opinion data, video modal public opinion data, and voice modal public opinion data. The multimodal public opinion dataset is subjected to structured parsing to obtain text behavior features, image behavior features, video behavior features, and voice behavior features. These text behavior features, image behavior features, video behavior features, and voice behavior features are then aligned and fused according to their collection timestamps to obtain fused behavior features. The fused behavior features are then subjected to structured extraction to extract multimodal financial behavior structure features.

[0101] Furthermore, the multimodal public opinion dataset acquisition module 1 is used to perform the following steps: The multimodal public opinion dataset is stored in a formatted manner using standardized storage fields; wherein, the standardized storage fields include the collection timestamp, the subject of the public opinion release, the public opinion dissemination platform, and the dissemination sequence code.

[0102] Furthermore, the financial behavior structure feature recognition module 3 is used to perform the following steps: Obtain historical instances of illegal financial activities samples of the target financial activity, perform template-based identification on the historical instances of illegal financial activities samples, and construct the illegal financial activity template; wherein, the template-based identification includes performing instance stability calculation on the multimodal historical financial behavior structural feature samples corresponding to the historical instances of illegal financial activities samples, and obtaining the instance stability index of each historical financial behavior structural feature; financial behavior structural feature samples that are greater than or equal to a preset instance stability threshold are marked as invariants, and financial behavior structural feature samples that are less than the preset instance stability threshold are marked as variables.

[0103] Furthermore, the financial behavior structure feature recognition module 3 is used to perform the following steps: The frequency of occurrence and the consistency of feature value distribution of each historical financial behavior structural feature in the multimodal historical financial behavior structural feature sample are analyzed; a weighted calculation is performed based on the frequency of occurrence and the consistency of feature value distribution to obtain the instance stability index.

[0104] Furthermore, the financial behavior structure feature recognition module 3 is used to perform the following steps: The frequency of feature occurrence is obtained by calculating the ratio of the number of samples containing the feature to the total number of instance samples, and the consistency of feature value distribution is obtained by calculating the variation coefficient of the feature in different instance samples.

[0105] Furthermore, the risk analysis result acquisition module 4 is used to perform the following steps: Financial behavior structural feature samples with values ​​greater than or equal to a preset instance stability threshold are labeled as invariants to obtain invariant-financial behavior structural feature samples; an invariant financial behavior structural diagram based on the logical relationship of the invariant-financial behavior structural feature samples is constructed; invariant structural similarity is calculated on the invariant-financial behavior structural features based on the invariant financial behavior structural diagram to obtain invariant structural similarity values; the invariant structural similarity values ​​are compared with a preset structural similarity threshold to obtain the first risk analysis result.

[0106] Furthermore, the risk analysis result acquisition module 4 is used to perform the following steps: The financial behavior structure feature samples that are less than the preset instance stability threshold are labeled as variables to obtain variable-financial behavior structure feature samples; if the variable-financial behavior structure feature samples are not empty after traversing them, the second risk analysis result is output based on the instance stability average difference of the variable-financial behavior structure feature samples.

[0107] Furthermore, the risk analysis result acquisition module 4 is used to perform the following steps: If the traversal of the variable-financial behavior structure feature sample returns empty, the stability of the updated calculation instance based on the variable-financial behavior structure feature is poor; based on the stability of the updated instance, the second risk analysis result is output.

[0108] The financial activity public opinion analysis system based on multimodal data analysis provided in this invention can execute the financial activity public opinion analysis method based on multimodal data analysis provided in any embodiment of this invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0109] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for analyzing public opinion on financial activities based on multimodal data analysis, characterized in that: The method includes: Collect multimodal public opinion datasets related to target financial activities; The multimodal public opinion dataset is subjected to structured analysis to extract multimodal financial behavior structural features, which include at least return description structural features, risk expression structural features, funding path structural features, and behavior guidance structural features. Identify the invariant-financial behavior structural features and variable-financial behavior structural features of the multimodal financial behavior structural features by using a pre-constructed template of illegal financial activities; An invariant structure similarity analysis is performed on the invariant-financial behavior structural features to obtain a first risk analysis result; a variable migration risk analysis is performed on the variable-financial behavior structural features to obtain a second risk analysis result. Based on the results of the first risk analysis and the second risk analysis, the results of the public opinion risk analysis are obtained.

2. The method as described in claim 1, characterized in that, The multimodal public opinion dataset is subjected to structured analysis to extract multimodal financial behavior structural features. The methods include: The multimodal public opinion dataset includes at least text modal public opinion data, image modal public opinion data, video modal public opinion data, and voice modal public opinion data; The multimodal public opinion dataset is subjected to structured parsing to obtain text behavior features, image behavior features, video behavior features, and voice behavior features; The text behavior features, image behavior features, video behavior features, and voice behavior features are aligned and fused according to the acquisition timestamp to obtain fused behavior features; The fusion behavior features are structurally extracted to extract multimodal financial behavior structural features.

3. The method as described in claim 1, characterized in that, The multimodal public opinion dataset is stored in a formatted manner using standardized storage fields; The standardized storage fields include the collection timestamp, the subject of the public opinion release, the public opinion dissemination platform, and the dissemination sequence code.

4. The method as described in claim 1, characterized in that, The method for identifying invariant-financial behavior structural features and variable-financial behavior structural features of the multimodal financial behavior structure features by using a pre-constructed template of illicit financial activities includes: Obtain historical instances of illegal financial activities samples of the target financial activity, perform template-based identification on the historical instances of illegal financial activities samples, and construct the illegal financial activity template; The templated identification includes performing instance stability calculations on the multimodal historical financial behavior structural feature samples corresponding to the historical instance illegal financial activity samples, and obtaining the instance stability index of each historical financial behavior structural feature. Financial behavior structural feature samples that are greater than or equal to a preset instance stability threshold are labeled as invariants, while financial behavior structural feature samples that are less than the preset instance stability threshold are labeled as variables.

5. The method as described in claim 4, characterized in that, Instance stability calculations are performed on the multimodal historical financial behavior structural feature samples corresponding to the historical instances of illegal financial activities, including: Analyze the consistency of feature occurrence frequency and feature value distribution for each historical financial behavior structural feature in the multimodal historical financial behavior structural feature sample; The instance stability index is obtained by weighting the frequency of occurrence of the features and the consistency of the distribution of the feature values.

6. The method as described in claim 5, characterized in that, The frequency of feature occurrence is obtained by calculating the ratio of the number of samples containing the feature to the total number of instance samples, and the consistency of feature value distribution is obtained by calculating the variation coefficient of the feature in different instance samples.

7. The method as described in claim 4, characterized in that, The method includes performing invariant structure similarity analysis on the invariant-financial behavior structural features to obtain the first risk analysis result. Financial behavior structural feature samples that are greater than or equal to the preset instance stability threshold are labeled with invariants to obtain invariant-financial behavior structural feature samples. Construct an invariant financial behavior structure diagram based on the logical relationship between the invariant-financial behavior structure feature samples; Based on the invariant financial behavior structure diagram, the invariant structure similarity of the invariant-financial behavior structure features is calculated to obtain the invariant structure similarity value; The invariant structural similarity value is compared with the preset structural similarity threshold to obtain the first risk analysis result.

8. The method as described in claim 4, characterized in that, Variable migration risk analysis is performed on the aforementioned variable-financial behavior structural characteristics to obtain a second risk analysis result. The method includes: The financial behavior structure feature samples that are less than the preset instance stability threshold are labeled as variables to obtain variable-financial behavior structure feature samples. Based on the variable-financial behavior structural features, if the traversal of the variable-financial behavior structural feature samples returns non-empty, the second risk analysis result is output based on the average stability difference of the instances of the variable-financial behavior structural feature samples.

9. The method as described in claim 8, characterized in that, If traversing the variable-financial behavior structure feature sample returns empty, the stability of the update calculation instance based on the variable-financial behavior structure feature is poor. Based on the poor stability of the updated instances, the second risk analysis result is output.

10. A financial activity public opinion analysis system based on multimodal data analysis, characterized in that: The system is used to implement the financial activity public opinion analysis method based on multimodal data analysis as described in any one of claims 1-9, the system comprising: The multimodal public opinion dataset acquisition module is used to collect multimodal public opinion datasets related to target financial activities; The financial behavior structure feature acquisition module is used to perform structured parsing on the multimodal public opinion dataset and extract multimodal financial behavior structure features. The multimodal financial behavior structure features include at least return description structure features, risk expression structure features, funding path structure features, and behavior guidance structure features. The financial behavior structure feature recognition module is used to identify the invariant-financial behavior structure features and variable-financial behavior structure features of the multimodal financial behavior structure features through a pre-constructed template of illegal financial activities. The risk analysis result acquisition module is used to perform invariant structure similarity analysis on the invariant-financial behavior structural features to obtain a first risk analysis result, and to perform variable migration risk analysis on the variable-financial behavior structural features to obtain a second risk analysis result. The public opinion risk analysis result acquisition module is used to acquire public opinion risk analysis results based on the first risk analysis result and the second risk analysis result.

Images