Specific APP and subject identification method based on deep multi-dimensional data fusion

Abstract

The invention relates to the technical field of data processing, in particular to a specific APP and subject recognition method based on deep multi-dimensional data fusion, which is characterized in that efficient fusion of features among different modals is carried out on acquired feature data of the APP by using a multi-dimensional feature interleaving algorithm, and the similarity between the feature of each modal and other modals is combined, so that the recognition accuracy of the specific APP is improved. The complex relation between the modals can be accurately expressed, and the recognition accuracy is improved; by introducing a dynamic adjustment function and a Gaussian attenuation function, similarity calculation of features between modals can be adjusted more accurately, and complex changes and mutual influences between the modals are captured, so that the adaptability and precision of recognition are improved; according to the method, in combination with preliminary identification of implicit information contained in a specific APP protocol, through analysis of a depth map structure propagation algorithm on a protocol text, potential subject information related to the specificity can be identified, key indication information such as possible financial fraud and fund chain problems can be extracted from the protocol, and the risk of the APP is further verified.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a specific APP and subject identification method based on deep multi-dimensional data fusion. Background Art

[0002] With the development of the digital economy and the popularity of smart phones, the development of financial apps has increased significantly, but some of these financial apps have malicious behaviors and the subject is highly concealed, so there is an urgent need to conduct subject mining and association technology research for specific financial apps. Traditional app identification methods usually rely on manual review or rule engines, which are difficult to process large amounts of app data and have weak identification capabilities for specific financial apps that do not meet traditional characteristics.

[0003] At the same time, the above-mentioned existing technologies still have certain drawbacks. During the specific App recognition process, the processing accuracy of multimodal data is low, resulting in low recognition accuracy. Summary of the invention

[0004] Technical purpose: In order to overcome the deficiencies in the prior art, the present invention provides a specific APP and subject identification method based on deep multidimensional data fusion to solve the technical problem of low recognition accuracy caused by low processing accuracy of multimodal data during specific APP identification.

[0005] Technical solution: To achieve the above purpose, the present invention discloses a specific APP and subject identification method based on deep multi-dimensional data fusion, which includes the following steps:

[0006] S1. Preliminary identification of specific APPs: The feature data of images, texts and keywords of the APP interface are integrated using a multi-dimensional feature interweaving algorithm to obtain comprehensive data. A multimodal recognition model is constructed based on the comprehensive data to preliminarily identify specific APPs.

[0007] S2. Identify the specific APP subject: Combined with the implicit information contained in the protocol in the text library of the preliminary identification of the specific APP, the deep graph structure propagation algorithm is used to conduct in-depth mining and analysis of the preliminary identification of the specific APP to identify the specific APP subject.

[0008] Furthermore, the step S1 comprises:

[0009] S11, image acquisition and preprocessing, as follows:

[0010] S111, intercepting the APP interface and extracting data information of images, texts and keywords from the intercepted interface;

[0011] S112. Preprocess the acquired data information to obtain the preprocessed data of images, text, and keywords.

[0012] S12. Data fusion: Use the multi-dimensional feature interleaving algorithm to fuse the preprocessed data of images, text, and keywords to obtain comprehensive data. Input the comprehensive data into the trained multi-modal recognition model for preliminary recognition of a specific APP, and obtain the preliminary recognized specific APP, as follows:

[0013] S121. Construct multi-modal feature data: Extract the feature data from images, text, and keywords to construct multi-modal feature data;

[0014] S122. Solve the similarity between modalities: For each pair of modalities X and modality Y, the similarity between any two features and is F XY (i, j); where X, Y ∈ {img, txt, kw};

[0015] S123. Interleaving processing of multi-modal feature data: Based on the similarity, perform interleaving processing on the multi-modal feature data to obtain the interleaved feature data S XYZ (i, j, k);

[0016] S124. High-dimensional mapping: After the interleaving processing operation, further map the multi-dimensional interleaved features to a new high-dimensional space through non-linear mapping to obtain comprehensive data;

[0017] S125. Model training: Based on the comprehensive data, perform model training. After obtaining the multi-modal recognition model, perform preliminary recognition on the specific APP to obtain the preliminary recognized specific APP.

[0018] Further, the step S121 includes: respectively using a convolutional neural network to extract image features from the preprocessed data of images, text, and keywords to obtain the feature data img of the image; using a long short-term memory network to perform text feature extraction on the text data to obtain the feature data txt of the text; extracting the feature data in the keywords through the TF-IDF method to obtain the feature data kw of the keywords; img, txt, and kw are multi-modal feature data.

[0019] Further, in the step S122, the calculation formula of F XY (i, j) is:

[0020]

[0021] Where, and are the i-th and j-th feature representations in modalities X and Y; is the inner product of the $i$-th feature in modality $X$ and the $j$-th feature in modality $Y$; $\sigma$ XY is the width parameter for controlling the Gaussian decay function; is the Gaussian decay function; $\alpha$ XY $(i, j)$ is the weighting factor; is the dynamic adjustment function; $\beta$ XY $(t)$ is the global adjustment term, which changes with time $t$.

[0022] Furthermore, in the step S123, $S$ XYZ $(i, j, k)$ is calculated as follows:

[0023]

[0024] where $S$ XYZ $(i, j, k)$ is the interleaving result of features among modalities $X$, $Y$, and $Z$, where modalities $X$, $Y$, $Z\in\{img, txt, kw\}$; and are the $i$-th, $j$-th, and $k$-th feature representations in modalities $X$, $Y$, and $Z$; $F$ XY $(i, j)$, $F$ XZ $(i, k)$, $F$ YZ $(j, k)$ represent the similarity between the $i$-th and $j$-th features in modalities $X$ and $Y$, the similarity between the $i$-th and $k$-th features in modalities $X$ and $Z$, and the similarity between the $j$-th and $k$-th features in modalities $Y$ and $Z$ respectively; $\odot$ is the interleaving operation.

[0025] Furthermore, the high-dimensional mapping calculation formula in the step S124 is as follows:

[0026]

[0027] where is the deep learning network; $F$ fusion $(i, j, k)$ is the feature data after high-dimensional mapping, i.e., the comprehensive data.

[0028] Furthermore, in the step S125, the expert experience method is used to select a neural network based on the actual application scenario to construct a multi-modal recognition model, and transfer learning and data augmentation techniques are used for model training to obtain a multi-modal recognition model. The comprehensive data is input into the multi-modal recognition model to obtain a preliminary recognition of a specific APP.

[0029] Furthermore, the step S2 includes:

[0030] S21. Extract the implicit information in the protocol text, specifically as follows:

[0031] S211. Build a text library: Based on the preliminary identification of a specific APP, crawl and summarize through web crawler technology to obtain a text library of the preliminarily identified specific APP. The text library includes protocol texts such as privacy agreements, user agreements, and service terms;

[0032] S212. Extract implicit information: Extract implicit information in the protocol text through context modeling, including implicit operating entity information and indicative information related to the specific, including: hints of financial fraud, promised high returns, potential vulnerability of the capital chain, investment companies, registered capital, and shareholders;

[0033] S22. Conduct in-depth mining and analysis of the preliminarily identified specific APP through the deep graph structure propagation algorithm. The deep graph structure propagation algorithm realizes the identification of the specific APP entity by fusing the implicit information in the protocol text through a hierarchical information propagation mechanism, as follows:

[0034] S221. Preprocess implicit information: Preprocess the implicit information in the protocol text to generate initial features. Each implicit information is transformed into a feature vector after preprocessing The implicit information includes keywords, syntactic structures, and sentiment analysis;

[0035] S222. Establish a similarity graph: Use the initial features to calculate the similarity between implicit information to construct the adjacency matrix A of the similarity graph. The adjacency matrix A is used to describe the association strength between implicit information in the protocol text. Use cosine similarity as the measure of similarity represents the implicit information and the implicit information The similarity between them constitutes any element in the adjacency matrix A.

[0036] S223. After establishing the similarity graph, enter the information propagation stage. The implicit information will be propagated according to the similarity graph. The propagation mechanism is based on the association strength between nodes, that is, the elements in the adjacency matrix A. Nodes represent any implicit information in the protocol text; Introduce a hierarchical information propagation mechanism to continuously update the state of each node through an iterative method.

[0037] S224. Perform weighted fusion on the node states of different layers to obtain the final feature data of any node and the final feature data of any node constitutes H final H final represents the final feature data;

[0038] S225. Based on the final feature data H final, identify specific APP entities through community discovery algorithm or shortest path algorithm.

[0039] Furthermore, in step S221, the TF-IDF algorithm is used to extract the weight of the keywords, the sentiment analysis score is obtained through the sentiment classification model, and the syntactic features are analyzed through dependency syntax to obtain the corresponding structural information. Formed by the following steps:

[0040]

[0041] Among them, TF-IDF(y s ) indicates implicit information The weighted sum of the term frequency and inverse document frequency of each word in , is the result of sentiment analysis, reflecting the sentiment tendency of the text. Represents the structural information extracted through syntactic analysis, which is the encoded representation of the syntax tree.

[0042] Furthermore, the node status calculation process in step 223 includes:

[0043] node The state at the lth layer of implicit information propagation is The node status update formula is:

[0044]

[0045] in, Is a node The state vector at layer l+1 includes the influence of the node itself and its neighbors; Is a node The state vector at the lth layer; W1 is the weight matrix of the node state update at the lth layer, which represents the weight coefficient of the node state information; λ1 is a hyperparameter used to control the influence of the node's own information on the state update; Is a node The historical accumulated information at the lth layer represents the past information and propagation process of the node; ∈ (l) is the noise term of the lth layer, which is used to simulate the uncertainty or random factors in the information propagation process; f(·) is the nonlinear activation function, which is used to perform nonlinear transformation on the node state and determines the nonlinear mapping of the node information; n is the total number of nodes.

[0046] The beneficial effects of the technical solution of the present invention are:

[0047] 1. Through the multi-dimensional feature interweaving algorithm, image, text, and keyword data are efficiently fused and processed. Feature interweaving operations are used to efficiently fuse features between different modalities. By combining the features of each modality with the similarity to other modalities, the complex relationships between modalities can be accurately expressed, further improving the accuracy of recognition. By introducing a dynamic adjustment function and a Gaussian decay function, the similarity calculation of features between modalities can be adjusted more precisely, capturing the complex changes and mutual influences between modalities, thereby improving the adaptability and accuracy of recognition.

[0048] 2. After initially identifying a specific APP, by analyzing the protocol text using the deep graph structure propagation algorithm, potential subject information related to the specific one can be identified, and key indication information such as possible financial fraud and fund chain problems can be extracted from the protocol to further verify the risk of the APP. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for identifying a specific APP and a subject based on deep multi-dimensional data fusion according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The principles and features of the present invention will be described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0051] A method for identifying a specific APP and a subject based on deep multi-dimensional data fusion provided by an embodiment of the present invention includes the following steps:

[0052] S1. The acquired images, texts, and keywords are fused and processed using the multi-dimensional feature interweaving algorithm to obtain comprehensive data. A multi-modal recognition model is constructed based on the comprehensive data to initially identify a specific APP.

[0053] S11. Image acquisition and preprocessing.

[0054] S111. The APP interface is intercepted using an existing image capture tool such as an automated script, and the intercepted interface is extracted using existing technologies to obtain data information of the image, text, and keywords of the intercepted interface.

[0055] S112. The acquired data information is preprocessed such as denoising, cleaning, and standardization to obtain preprocessed data of the image, text, and keywords. The technical means used in the above preprocessing process are well-known to those skilled in the art and will not be elaborated here.

[0056] S12. Further, the preprocessed image, text, and keyword data are fused using a multi-dimensional feature interweaving algorithm to obtain comprehensive data. The multi-dimensional feature interweaving algorithm ensures the deep fusion of multi-modal data by introducing a deep multi-dimensional analysis of modal features, comprehensively considering the non-linear relationships between modalities, the importance of global features, the similarity of local features, and multi-dimensional feature interweaving. The specific implementation process is as follows:

[0057] S121. First, the preprocessed image, text, and keyword data are respectively used to extract image features using an existing convolutional neural network to obtain image feature data, use an existing long short-term memory network to extract text features from the text data to obtain text feature data, and extract feature data in the keywords through the TF-IDF (Term Frequency-Inverse Document Frequency) method to obtain keyword feature data;

[0058] S122. Further, the feature data of the image, text, and keywords are used as multi-modal feature data. To improve the calculation accuracy of the similarity between modalities, a deep multi-dimensional analysis of modal features is introduced. For any two features and in each pair of modalities X and Y (where X, Y ∈ {img, txt, kw}, and img, txt, and kw are the feature data of the image, text, and keywords respectively), the similarity F XY (i, j) is calculated as follows:

[0059]

[0060] where, and are the i-th and j-th feature representations in modalities X and Y; is the inner product of the i-th feature in modality X and the j-th feature in modality Y, representing the linear similarity between these two features and quantifying the correlation between these two features in the common feature space; σ XY is the width parameter that controls the Gaussian decay function and affects the distance calculation between modal features, which is set according to the specific scenario; is the Gaussian decay function, which is used to measure the non-linear similarity between two features. When the Euclidean distance between the two features is larger, the similarity is smaller; α XY (i, j) is the weighting factor, which is used to adjust the similarity contribution between the i-th and j-th features in modalities X and Y, and can dynamically adjust the weight of each pair of features to handle the importance of different features, and is calculated according to the local importance of the features through certain strategies (such as weighting based on the attention mechanism); is a dynamic adjustment function that temporally adjusts the similarity based on time t and feature indices i and j, used to capture the relationship of feature changes over time between modalities, obtained using time series analysis methods; β XY (t) is a global adjustment term that changes with time t and is used to dynamically adjust the similarity calculation between the features of modality X and modality Y, which can ensure fine-tuning of the similarity of different features within a specific time period, obtained through experimental methods.

[0061] S123. Based on the above similarity, perform interleaving processing on multi-dimensional feature data (multi-modal feature data) to obtain the interleaved feature data S XYZ (i, j, k), and the interleaving formula is as follows:

[0062]

[0063] Among them, S XYZ (i, j, k) is the interleaving result of features between three different modalities (modality X, Y, Z ∈ {img, txt, kw}), representing the preliminary fusion features of multi-modalities, combining the similarity of each modality feature with other modalities and reflecting the complex relationships between modalities. Among them, and are the i-th, j-th, and k-th feature representations in modality X, modality Y, and modality Z; F XY (i, j), F XZ (i, k), F YZ (j, k) respectively represent the similarity between the i-th and j-th features in modality X and modality Y, the similarity between the i-th and k-th features in modality X and modality Z, and the similarity between the j-th and k-th features in modality Y and modality Z; ⊙ is an interleaving operation, that is, an element-wise multiplication operation (Hadamard Product), used to multiply the elements of two feature vectors at the element level, for effectively combining the features of different modalities and capturing the mutual relationships between modalities.

[0064] S124. Further, after the multi-dimensional interleaving operation, map the multi-dimensionally interleaved features to a new high-dimensional space through a non-linear mapping. This mapping uses existing deep learning networks such as multi-layer perceptrons or self-attention networks, and its purpose is to achieve more precise feature fusion by capturing the high-order relationships between modalities. The formula for the high-dimensional mapping is:

[0065]

[0066] Among them, is a deep learning network such as a multi-layer perceptron or a self-attention network; F fusion(i, j, k) is the feature data after high-dimensional mapping, which contains depth information from multiple modalities and integrates the feature interleaving information from different dimensions, that is, the comprehensive data.

[0067] S125. Further, based on the comprehensive data, use the expert experience method to select an existing neural network such as a fully connected neural network based on the actual application scenario to construct a multi-modal recognition model, and use existing transfer learning and data augmentation techniques to train the model to obtain a trained multi-modal recognition model. Further, input the comprehensive data into the trained multi-modal recognition model for preliminary recognition of a specific APP to obtain a preliminary recognition of the specific APP.

[0068] S2. After preliminary recognition, combine the implicit information contained in the relevant protocols in the text library of the preliminary recognition of the specific APP to conduct in-depth mining and analysis of the preliminary recognition of the specific APP through the deep graph structure propagation algorithm to further identify the specific APP entity.

[0069] S21. Extract the implicit information in the protocol text as follows:

[0070] S211. After preliminary recognition, based on the preliminary recognition of the specific APP, use existing web crawler technology to crawl and summarize to obtain a text library of the preliminary recognition of the specific APP (including protocol texts such as privacy policies, user agreements, and terms of service).

[0071] S212. After the text library is constructed, extract the implicit information in the protocol text through existing context modeling (such as BERT or GPT), including implicit operating entity information and indicative information related to the specific, such as: hints of financial fraud, promised high returns, potential vulnerability of the capital chain, investment companies, registered capital, shareholders, etc.;

[0072] S22. Further, conduct in-depth mining and analysis of the preliminary recognition of the specific APP through the deep graph structure propagation algorithm to further identify the specific APP entity; the deep graph structure propagation algorithm realizes the recognition of the specific APP entity through a hierarchical information propagation mechanism, combines the implicit information in the protocol text, and uses the propagation and deep feature fusion of the graph structure. The specific implementation process is as follows:

[0073] D221. First, preprocess the implicit information in the protocol text to generate initial features, and each implicit information is transformed into a feature vector after preprocessing It is to convert the implicit information of each text (such as keywords, syntactic structures, sentiment analysis, etc.) into structured data for subsequent calculations. The specific operations are as follows. The TF-IDF algorithm is used to extract the weights of keywords. The sentiment analysis score can be obtained through a sentiment classification model, and the syntactic features can obtain the corresponding structural information through dependency syntactic analysis. Therefore, the initial features are formed through the following steps:

[0074]

[0075] Among them, TF-IDF(y s ) represents the weighting of the term frequency and inverse document frequency of each word in the implicit information , is the result of sentiment analysis, reflecting the sentiment tendency of the text, and

[0076] represents the structural information extracted through syntactic analysis, usually the encoded representation of a syntactic tree. Through the above process, a high-dimensional feature vector containing the implicit features of the implicit information is provided for each implicit information.

[0077] S222. Further, the initial features are used to calculate the similarity between implicit information to construct the adjacency matrix A of the similarity graph. This matrix is used to describe the association strength between implicit information in the protocol text, and cosine similarity is selected as the measure of similarity represents the similarity between implicit information and implicit information , forming any element in the adjacency matrix.

[0078] S223. After establishing the similarity graph, enter the information propagation stage. In this stage, the implicit information will be propagated according to the similarity graph, and the propagation mechanism is based on the connection strength between nodes (any implicit information in the protocol text), that is, the elements in the adjacency matrix A. Further, a hierarchical information propagation mechanism is introduced to continuously update the state of each node through an iterative method.

[0079] At each layer l of the implicit information propagation, the state of each node is updated based on the information of its neighbor nodes. Assume that the state of node at the l-th layer is Then the state update formula of the node is:

[0080]

[0081] Among them, is node The state vector at the (l + 1)-th layer represents the hidden information (feature vector) of the node, including the influence of the node itself and its neighbors; is the node The state vector at the l-th layer; W1 is the weight matrix for updating the node state at the l-th layer, representing the weighted coefficient of the node state information, aiming to optimize the efficiency and accuracy of information propagation between nodes, obtained through the experimental method; λ1 is a hyperparameter used to control the influence degree of the node's own information on state update, preventing over-reliance on neighbor nodes, determined through the expert experience method; is the node The historical cumulative information at the l-th layer represents the node's past information and propagation process; ∈ (l) is the noise term at the l-th layer, used to simulate the uncertainty or random factors in the information propagation process, determined according to the expert experience method; f(·) is a non-linear activation function used for non-linear transformation of the node state, such as ReLU, Sigmoid, Tanh, etc., determining the non-linear mapping of the node information; n is the total number of nodes.

[0082] The above propagation process continues until the L-th layer preset according to the expert experience method, that is, the maximum number of layers; the output of each layer is accumulated and used for the propagation of the next layer, capable of capturing deeper correlations and patterns between nodes layer by layer. Finally, the state of the node at the L-th layer contains information from all layers.

[0083] S224. To further combine the information of different layers, the node states of different layers are weighted and fused to obtain the final feature data of any node and from the final feature data of any node constitutes the final feature data H final , and the weights in the above weighted fusion are determined according to the expert experience method.

[0084] S225. Finally, based on the final feature data H final , the main body of a specific APP is further identified through existing algorithms such as community discovery algorithms and shortest path algorithms.

[0085] In summary, the method for identifying a specific APP and its main body based on deep multi-dimensional data fusion is completed.

[0086] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A specific APP and subject identification method based on deep multi-dimensional data fusion, characterized in that: The following steps are involved: S1. Preliminary identification of specific APPs: The acquired feature data of images, texts and keywords of the APP interface are fused and processed using a multi-dimensional feature interweaving algorithm to obtain comprehensive data. A multimodal recognition model is constructed based on the comprehensive data to preliminarily identify specific APPs. S2. Identify the specific APP subject: Combined with the implicit information contained in the protocol in the text library of the preliminary identification of the specific APP, the deep graph structure propagation algorithm is used to conduct in-depth mining and analysis of the preliminary identification of the specific APP to identify the specific APP subject.

2. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 1 is characterized in that: The step S1 comprises: S11, image acquisition and preprocessing, as follows: S111, intercepting the APP interface and extracting data information of images, texts and keywords from the intercepted interface; S112, preprocessing the acquired data information to obtain preprocessed image, text and keyword data; S12, data fusion: The pre-processed image, text and keyword data are fused using a multi-dimensional feature interweaving algorithm to obtain comprehensive data, which is then input into a trained multimodal recognition model to perform preliminary recognition of a specific APP, and obtain preliminary recognition of the specific APP, as follows: S121, constructing multimodal feature data: extracting feature data from images, texts, and keywords, and constructing multimodal feature data; S122. Solve the similarity between modes: For each pair of mode X and mode Y, any two features and The similarity between them is F XY (i, j); where X, Y∈{img, txt, kw}; S123, interleaving processing of multimodal feature data: based on similarity, interleaving processing is performed on the multimodal feature data to obtain interleaved feature data S XYZ (i, j, k); S124, high-dimensional mapping: after the interleaving operation, the multi-dimensional interleaved features are further mapped to a new high-dimensional space through nonlinear mapping to obtain comprehensive data; S125. Model training: Perform model training based on comprehensive data, and after obtaining a multimodal recognition model, perform preliminary recognition on the specific APP to obtain preliminary recognition of the specific APP.

3. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: The step S121 includes: using a convolutional neural network to extract image features from the preprocessed image, text and keyword data to obtain image feature data img; using a long short-term memory network to extract text features from text data to obtain text feature data txt; extracting feature data from keywords through a TF-IDF method to obtain keyword feature data kw; img, txt, kw are multimodal feature data.

4. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: In step S122, F XY The calculation formula for (i, j) is: in, and are the i-th and j-th feature representations in modality X and modality Y; is the inner product of the i-th feature in mode X and the j-th feature in mode Y; σ XY To control the width parameter of the Gaussian attenuation function; is the Gaussian decay function; α XY (i, j) is the weighting factor; is a dynamic adjustment function; β XY (t) is a global adjustment term that changes with time t.

5. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: In step S123, XYZ The calculation formula for (i, j, k) is as follows: Among them, s XYZ (i, j, k) is the interleaving result of features between modality X, modality Y and modality Z, modality X, Y, Z ∈ {img, txt, kw}; and are the i-th, j-th, and k-th feature representations in modality X, modality Y, and modality Z; F XY (i, j), F XZ (i, k), F YZ (j, k) represents the similarity between the i-th and j-th features in modality X and modality Y, the similarity between the i-th and k-th features in modality X and modality Z, and the similarity between the j-th and k-th features in modality Y and modality Z, respectively; ⊙ is the interleaving operation.

6. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: The high-dimensional mapping calculation formula in step S124 is as follows: in, is a deep learning network; F fusion (i, j, k) is the feature data after high-dimensional mapping, that is, comprehensive data.

7. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: The step S125 uses expert experience to select a neural network based on the actual application scenario to build a multimodal recognition model, and uses transfer learning and data enhancement technology to train the model to obtain a multimodal recognition model, and inputs the comprehensive data into the multimodal recognition model to obtain a preliminary recognition of the specific APP.

8. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 2 is characterized in that: The step S2 comprises: S21. Extract the implicit information in the agreement text, as follows: S211. Build a text library: Based on the preliminary identification of a specific APP, crawl and summarize through web crawler technology to obtain a text library of the preliminary identified specific APP, which includes the agreement texts of the privacy agreement, user agreement, and terms of service; S212. Extract implicit information: Extract implicit information from the agreement text through context modeling, including implicit operating entity information and specific indicative information, including: hints of financial fraud, promised high returns, potential fragility of the capital chain, investment companies, registered capital and shareholders; S22. Perform in-depth mining and analysis on the initially identified specific APP through the deep graph structure propagation algorithm. The deep graph structure propagation algorithm integrates the implicit information in the protocol text through a hierarchical information propagation mechanism to achieve the identification of the specific APP subject, as follows: S221, implicit information preprocessing: preprocess the implicit information in the protocol text to generate initial features. Each implicit information y i After preprocessing, it is converted into a feature vector v i ,The implicit information includes keywords, syntactic structure and sentiment analysis; S222. Establish similarity graph: Use the initial features to calculate the similarity between implicit information to construct the adjacency matrix A of the similarity graph. The adjacency matrix A is used to describe the correlation strength between implicit information in the protocol text. Cosine similarity is used as the measure of similarity A. ij , A ij Indicates implicit information y i and implicit information y j The similarity between them constitutes any element in the adjacency matrix A; S223. After the similarity graph is established, the information propagation phase is entered. The implicit information will be propagated according to the similarity graph. The propagation mechanism is based on the association strength between nodes, that is, the elements in the adjacency matrix A. The nodes represent any implicit information in the protocol text. A hierarchical information propagation mechanism is introduced to continuously update the status of each node in an iterative manner. S224: Perform weighted fusion on the node states of different layers to obtain the final feature data of any node And the final feature data of any node Composition H final , H final Represents the final feature data; S225, based on the final feature data H final , identify specific APP entities through community discovery algorithm or shortest path algorithm.

9. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 8 is characterized in that: In step S221, the TF-IDF algorithm is used to extract the weights of keywords, the sentiment analysis score is obtained through the sentiment classification model, the syntactic features are analyzed through dependency syntax to obtain the corresponding structural information, and the initial feature v i Formed by the following steps: Among them, TF-IDF(y s ) represents the implicit information y i The weighted sum of the term frequency and inverse document frequency of each word in , is the result of sentiment analysis, reflecting the sentiment tendency of the text. Represents the structural information extracted through syntactic analysis, which is the encoded representation of the syntax tree.

10. The specific APP and subject identification method based on deep multi-dimensional data fusion according to claim 8 is characterized in that: The node status calculation process in step 223 includes: node The state at the lth layer of implicit information propagation is The node status update formula is: in, Is a node The state vector at layer l+1 includes the influence of the node itself and its neighbors; Is a node The state vector at the lth layer; W1 is the weight matrix of the node state update at the lth layer, which represents the weight coefficient of the node state information; λ1 is a hyperparameter used to control the influence of the node's own information on the state update; Is a node The historical accumulated information at the lth layer represents the past information and propagation process of the node; ∈ (l) is the noise term of the lth layer, which is used to simulate the uncertainty or random factors in the information propagation process; f(·) is the nonlinear activation function, which is used to perform nonlinear transformation on the node state and determines the nonlinear mapping of the node information; n is the total number of nodes.

Images