Cross-media intelligent identification method and system based on social characteristics of population data

Abstract

The embodiment of the invention provides a cross-media intelligent identification method and system based on social features of population data, and relates to the technical field of computers. The method comprises the following steps: acquiring personnel data sent by a plurality of data sources, wherein the personnel data is used for representing related data of activity personnel participating in a target social activity; the personnel data are aligned after being analyzed, and personnel characteristic data are obtained; performing semantic alignment on the personnel feature data, constructing a knowledge graph for the active personnel, and constructing a region data graph based on the knowledge graph, the knowledge graph being used for indicating a semantic relationship between the personnel feature data, and the region data graph being used for indicating correlation between region association features; the region association features are used for representing personnel feature data related to the region attributes; and determining an activity analysis result based on the knowledge graph and the region data graph, the activity analysis result being used for evaluating the risk degree of the target social activity. The accuracy of activity analysis is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cross-media intelligent recognition method and system based on social characteristics of population data. Background Art

[0002] People are the subject and object of all social activities. The data generated by seemingly unrelated social activities are actually directly or indirectly related to people. With the development of information technology, different social feature models are stored and classified among various media software and systems. Some social activities are often associated with multiple social features.

[0003] At present, the personnel activity analysis method based on the social characteristics of population data combined with cross-media software has the following defects: First, the information access point of cross-media software is the same source or similar source information access point, and the received information is single, so the social characteristics of the population data analyzed are single, which makes it difficult to fully describe the state of social population activities; second, only using historical activity data to predict personnel activity risks and behaviors, but the same activity may have different risks in different regions (such as Shandong and Sichuan), and the analysis results are too one-sided. Therefore, the accuracy of existing social activity analysis methods is low. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, the present invention provides a cross-media intelligent recognition method and system based on the social characteristics of population data, so as to solve the problem of low accuracy of the existing social activity analysis methods.

[0005] In a first aspect, the present application provides a cross-media intelligent recognition method based on the social characteristics of population data, comprising: Acquire personnel data sent by multiple data sources, where the personnel data is used to characterize relevant data of personnel participating in a target social activity; Parsing and aligning the personnel data to obtain personnel feature data; Semantically aligning the personnel feature data, constructing a knowledge graph for the activity personnel, and constructing a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to the preset regional attributes; Based on the knowledge graph and the regional data graph, an activity analysis result is determined, and the activity analysis result is used to evaluate the risk of the target social activity.

[0006] In a possible implementation, parsing and aligning the personnel data to obtain personnel feature data includes: When the personnel data is represented in JSON format, the keys identified in the personnel data are matched with the preset key set in sequence, and the matched keys and corresponding key values ​​are superimposed to obtain the first data to be processed, wherein the key is used to identify the data related to the specified feature in the personnel data, and the key value is used to indicate the personnel data associated with the key; When the personnel data is represented as communication data of a specified protocol, all time windows are traversed, the keys identified in the personnel data obtained within the time window are matched with the key set, and the matched keys and corresponding key values ​​are aggregated to obtain second data to be processed, wherein the time window is determined by a preset time window length; The first data to be processed and the second data to be processed are aligned to obtain personnel feature data.

[0007] In a possible implementation, aligning the first data to be processed and the second data to be processed to obtain personnel feature data includes: Using a preset variational autoencoder to perform feature extraction on the first to-be-processed data and the second to-be-processed data to obtain a feature vector; Reconstructing the feature vector using a preset decoder and determining a reconstruction loss; Calculate the KL divergence based on the prior distribution data determined by the feature vector and the posterior distribution data obtained by the variational autoencoder; According to the reconstruction loss and the KL divergence, the variational autoencoder is modified until the reconstruction loss and the KL divergence converge; Based on the first data to be processed and the second data to be processed, determine the transmission probability and cost function between the first posterior distribution data and the second posterior distribution data, and calculate the transmission cost value according to the transmission probability, the cost function and the transmission matrix, wherein the first posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the first data to be processed, the second posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the second data to be processed, and the transmission matrix is ​​organized by the first data to be processed and the second data to be processed; A transmission matrix that minimizes the transmission cost value is screened out, and the first data to be processed and the second data to be processed are aligned based on the screened transmission matrix to obtain personnel feature data.

[0008] In a possible implementation, aligning the first data to be processed and the second data to be processed to obtain personnel feature data further includes: The reconstruction loss is determined by the following formula (1), including: (1) in, is the reconstruction loss, represents one of the first data to be processed and the second data to be processed, is the feature vector, represents the reconstructed data obtained after reconstruction by the decoder, represents a specified trainable parameter in the decoder; The KL divergence is determined by the following formula (2), including: (2) in, represents the KL divergence, represents the prior distribution data, represents the posterior distribution data; The cost function is determined by the following formula (3), including: (3) in, is the i-th first posterior distribution data, is the jth second posterior distribution data, Represents the cost function of the i-th first posterior distribution data and the j-th second posterior distribution data; The transmission cost is determined by the following formula (4), including: (4) in, represents the transfer cost value, is the transmission matrix, represents the transmission probability from the i-th first posterior distribution data to the j-th second posterior distribution data.

[0009] In a possible implementation, the semantically aligning the personnel feature data to construct a knowledge graph for the activity personnel includes: Based on the keys and key values ​​corresponding to the personnel feature data, calculating the feature similarity between any two keys; Determine the corresponding key as an associated key when the feature similarity exceeds a preset similarity threshold, and obtain a semantic relationship indicating the degree of semantic association between the keys; A connection is established between the associated keys to form a knowledge graph, wherein a node in the knowledge graph represents a key and its key value, and an edge in the knowledge graph represents a semantic relationship between two connected nodes.

[0010] In a possible implementation, constructing a regional data graph based on the knowledge graph includes: Mapping the nodes and edges in the knowledge graph to a vector space to obtain corresponding node embedding vectors and edge embedding vectors; Calculating the regional correlation between the node embedding vector and the regional attribute, and selecting the node embedding vector corresponding to the regional correlation exceeding a preset correlation threshold as the regional correlation feature; The region association feature is used as a node in the region data graph, and the edge in the region data graph is generated according to the edge embedding vector corresponding to the screened node embedding vector to form the region data graph.

[0011] In a possible implementation, determining the activity analysis result based on the knowledge graph and the regional data graph includes: Using a preset graph convolutional network, feature extraction is performed on the knowledge graph and the regional data graph respectively to generate a first feature matrix and a second feature matrix, wherein the first feature matrix is ​​used to indicate nodes in the knowledge graph, and the second feature matrix is ​​used to indicate nodes in the regional data graph; Using a preset fusion self-attention mechanism, the first feature matrix and the second feature matrix are associated to obtain an associated value; When the association value is greater than a preset association threshold, the corresponding nodes in the knowledge graph and the regional data graph are connected, and the association value is used as the weight of the edge connecting the nodes to form a target data graph, which is used to indicate the prediction of the risk of the target social activity; The target data graph is analyzed using the graph convolutional network to obtain an activity analysis result of the target social activity.

[0012] In a possible implementation, determining the activity analysis result based on the knowledge graph and the regional data graph further includes: The first characteristic matrix is ​​determined by the following formula (5), including: (5) in, Indicates the first The first characteristic matrix of the layer, represents the activation function of the graph convolutional network, represents the adjacency matrix of the knowledge graph, represents the degree matrix of the adjacency matrix of the knowledge graph, Indicates the first The first characteristic matrix of the layer, Indicates the first layer weight matrix; The second characteristic matrix is ​​determined by the following formula (6), including: (6) in, Indicates the area data in the figure The second characteristic matrix of the layer, represents the adjacency matrix of the region data graph, a degree matrix representing the adjacency matrix of the region data graph, Indicates the area data in the figure The second characteristic matrix of the layer, Indicates the area data in the figure layer weight matrix; The correlation value is determined by the following formula (7), including: (7) in, represents the associated value, Represents the fusion self-attention mechanism function, represents the attention vector, represents the transposed matrix of the attention vector, represents a weight matrix associated with the regional data graph, Represents the weight matrix associated with the knowledge graph.

[0013] In a possible implementation, aligning the first data to be processed and the second data to be processed to obtain personnel feature data further includes: Based on the transfer matrix, determining an entropy regularization term; Determining an optimization factor based on the entropy regularization term and a preset regularization parameter, wherein the regularization parameter is used to control the weight of the entropy regularization term; In the process of calculating the transmission cost value, the optimization factor is introduced to screen out a transmission matrix that minimizes the transmission cost value.

[0014] In a second aspect, the present application provides a cross-media intelligent recognition system based on social characteristics of population data, including: A data acquisition module, used to acquire personnel data sent by multiple data sources, wherein the personnel data is used to characterize relevant data of personnel participating in a target social activity; A data analysis module, used to parse and align the personnel data to obtain personnel feature data; A feature analysis module, used to semantically align the personnel feature data, construct a knowledge graph for the activity personnel, and construct a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, and the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to the preset regional attributes; The activity analysis module is used to determine the activity analysis results based on the knowledge graph and the regional data graph, and the activity analysis results are used to evaluate the risk of the target social activity.

[0015] The cross-media intelligent recognition method and system based on the social characteristics of population data provided by the present application obtain multi-source data of participants in target social activities, improve data diversity, and perform analysis and alignment to ensure the consistency of data distribution, thereby achieving a comprehensive characterization of the social population activity status, and then semantically aligning the personnel characteristic data to construct corresponding knowledge graphs and regional data graphs to determine the activity analysis results. Taking into account the impact of the characteristics of the participants and the regional characteristics on the activity analysis, the problem of low accuracy of existing social activity analysis methods is solved, and the accuracy of activity analysis and risk assessment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a process flow of a cross-media intelligent recognition method based on social characteristics of population data provided in an embodiment of the present application; Figure 2 A schematic diagram of a data alignment process provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a cross-media intelligent recognition system based on social characteristics of population data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Figure 1A flowchart of a cross-media intelligent recognition method based on social characteristics of population data provided in an embodiment of the present application is provided, and the method includes steps S101 to S104.

[0019] S101. Acquire personnel data sent by multiple data sources, where the personnel data is used to characterize relevant data of personnel participating in a target social activity.

[0020] In this application, the data source can be a multimedia platform such as an archive system, a household registration system, and various social software. Different data sources correspond to different data features. This application can obtain personnel data of various data types of active personnel from different data sources. Personnel data is relevant data describing active personnel. For example, the archive system stores personnel data such as personal academic records, work experience, and unit certification, the household registration system records personal household registration information, and social software analyzes personal preferences and other personnel data through user behaviors such as browsing records, search records, and interaction records.

[0021] S102: Analyze and align the personnel data to obtain personnel feature data.

[0022] It should be noted that since the data sources in this application are diverse (such as WebSocket, JSON files, etc.), the acquisition methods are different, and the data formats are complex (structured, semi-structured, unstructured), this application parses and aligns the personnel data obtained from multiple data sources to extract personnel feature data and ensure data consistency, thereby improving the reliability and accuracy of the data during the analysis process.

[0023] In this application, personnel characteristic data is extracted from personnel data, and the personnel characteristic data includes social characteristics of population data. The social characteristics of population data are represented by characteristic information related to individuals, such as age, gender, date of birth, occupation, income, education file, work experience, unit certificate, report record, violation record, regulation record, fingerprint information, face information, household registration information, place of residence, personal preferences, etc.

[0024] S103. Perform semantic alignment on the personnel feature data, construct a knowledge graph for the active personnel, and construct a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, and the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to preset regional attributes.

[0025] In this application, semantic alignment refers to establishing and discovering corresponding relationships between data in different modalities or from different sources to achieve effective integration of information, eliminate semantic differences between data from different sources, and ensure that data from different sources or modalities are semantically consistent. Then, based on the semantic relationship between personnel feature data, a knowledge graph is constructed to integrate data from different sources into the knowledge graph. It should be noted that this application extracts regional association features related to regional attributes in the knowledge graph, and constructs a regional data graph based on the regional association features. The regional data graph is used to identify the corresponding relationship between regional association features. For example, regional association features may include household registration information, place of residence, unit certification, work experience, occupation, income, personal preferences, etc.

[0026] S104. Determine an activity analysis result based on the knowledge graph and the regional data graph, wherein the activity analysis result is used to evaluate the risk of the target social activity.

[0027] In this application, the target social activities are evaluated and analyzed based on the semantic relationship and correspondence between the personnel feature data, and the correspondence between the regional related features, such as the activity risk assessment, taking into account the characteristics of the activity personnel themselves and the characteristics of the region where they are located. That is, different regions may have different views on the same social activity due to differences in cultural background, laws and regulations, climatic conditions, etc., so the activity analysis of the social activity in different regions (such as the activity risk) is different. Therefore, this application uses knowledge graphs and regional data graphs to analyze activities, taking into account the social characteristics of the activity personnel and the impact of regional characteristics on the target social activities, which can effectively improve the accuracy of activity analysis.

[0028] The cross-media intelligent recognition method based on the social characteristics of population data provided in this embodiment obtains multi-source data of participants in the target social activities, improves the diversity of the data, and performs analysis and alignment to ensure the consistency of data distribution, thereby comprehensively describing the state of social population activities, and then semantically aligns the personnel characteristic data to construct corresponding knowledge graphs and regional data graphs to determine the activity analysis results. Taking into account the influence of the characteristics of the participants and the regional characteristics on the activity analysis, the problem of low accuracy of the existing social activity analysis methods is solved, and the accuracy of activity analysis and risk assessment is improved.

[0029] In some embodiments, step S102 includes: When the personnel data is represented in JSON format, the keys identified in the personnel data are matched with the preset key set in sequence, and the matched keys and corresponding key values ​​are superimposed to obtain the first data to be processed, wherein the key is used to identify the data related to the specified feature in the personnel data, and the key value is used to indicate the personnel data associated with the key; When the personnel data is represented as communication data of a specified protocol, all time windows are traversed, the keys identified in the personnel data obtained within the time window are matched with the key set, and the matched keys and corresponding key values ​​are aggregated to obtain second data to be processed, wherein the time window is determined by a preset time window length; The first data to be processed and the second data to be processed are aligned to obtain personnel feature data.

[0030] In this embodiment, due to different ways of acquiring personnel data, personnel data may exist in multiple data forms (such as WebSocket, JSON files, etc.). The corresponding parsing method is determined according to the data form of the personnel data, and data alignment and feature extraction are performed on the parsed personnel data to obtain personnel feature data.

[0031] It should be noted that for personnel data in JSON format, one or more keys are identified in the personnel data, and the key is used to identify the field or field combination of the data record, that is, the key is used to identify the personnel data related to personal characteristics, and the key value is used to indicate the data content associated with the key. For example, the personnel data is "age 30, graduated from Peking University", key 1 "age", key 1 key value is "30", key 2 "education", key 2 key value is "Peking University". Then in this embodiment, each key in the personnel data is traversed, the key is matched with the key set, and the matched keys and their key values ​​are superimposed to obtain the first data to be processed. Among them, the key set contains at least one key related to the specified feature (such as personal characteristics), so as to quickly filter out the keys related to the personnel feature data.

[0032] Specifically, recursively parse the personnel data in JSON format and expand it by key-value pairs, as shown in the following formula: in, Represents personnel data in JSON format. represents the kth key, represents the key value of the kth key, Indicates that from personnel data The information extracted or the parsed standardized data structure can be understood as the first data to be processed. represents a parsing function, the parsing process is to match and filter the keys in the personnel data according to the key set, and use the filtered keys and their key values ​​as the parsing output results. Then, traverse each key, and superimpose the parsing output results to obtain the first data to be processed.

[0033] In this embodiment, the personnel data is represented as communication data of a specified protocol, and the specified protocol may be WebSocket. The real-time stream data is processed by time windowing, that is, according to the length of the time window, the personnel data in each time window is obtained, the key identified in the personnel data in the time window is matched with the key set, and it is determined whether it has a key in the key set, if it exists, it is retained, otherwise it is discarded, each time window is traversed, and the matched keys are aggregated to obtain the second data to be processed.

[0034] Specifically, the WebSocket data is parsed using the following formula: in, represents the data after time window aggregation (i.e. the second data to be processed), represents the personnel data obtained in the tth time window, is the time window length, Represents an aggregation operation.

[0035] Furthermore, since the personnel data comes from different data sources, there may be problems such as naming differences, different time scales, and different data formats. By aligning the first data to be processed and the second data to be processed obtained after parsing, these data are matched and adjusted to achieve data consistency and ensure the reliability and accuracy of the data during the analysis process.

[0036] Therefore, this embodiment performs corresponding parsing operations through the data form of personnel data, aligns the parsed data, achieves consistency of multi-source data, improves the consistency of personnel feature data, and further improves the accuracy of subsequent analysis processes.

[0037] In some embodiments, Figure 2 This is a flow chart of data alignment provided in an embodiment of the present application. This embodiment provides a data alignment method, that is, step S102 also includes steps S201 to S206.

[0038] S201. Use a preset variational autoencoder to perform feature extraction on the first data to be processed and the second data to be processed to obtain a feature vector.

[0039] In this embodiment, since different data sources have different ways of expressing the same thing, directly aligning the data through comparison will result in poor alignment effect. In this embodiment, a variational autoencoder is used to perform feature mapping on the multi-source data after parsing. After the features of all the data are mapped to the same feature space, the optimal transmission method is used to perform data alignment and distribution standardization.

[0040] Specifically, the variational autoencoder is used to extract the potential representation of the input data, as shown in the following formula: in, is the feature vector, represents the processing operation of the variational autoencoder, represents the first data to be processed, In this embodiment, the feature vector is used to characterize the potential deep features. Representation data Through the feature vector obtained by the variational autoencoder, the features of all data are mapped to the same feature space .

[0041] S202: Reconstruct data of the feature vector using a preset decoder, and determine a reconstruction loss.

[0042] Specifically, after the mapping in step S201 is completed, in order to ensure the accuracy of the mapping, the mapping result needs to be evaluated using the reconstruction loss and the KL divergence. First, the feature vector is input to the decoder, and the decoder reconstructs an output similar to the original input (i.e., the first data to be processed and the second data to be processed). The similarity between the output of the decoder and the actual input data (i.e., the first data to be processed and the second data to be processed) is measured in this embodiment by the reconstruction loss (such as the mean square error).

[0043] More specifically, the reconstruction loss is determined by the following formula (1), including: (1) in, is the reconstruction loss, represents one of the first data to be processed and the second data to be processed, is the feature vector, represents the reconstructed data obtained after reconstruction by the decoder, Represents the trainable parameters specified in the decoder.

[0044] S203. Calculate the KL divergence based on the prior distribution data determined by the feature vector and the posterior distribution data obtained by the variational autoencoder.

[0045] In this embodiment, the KL divergence is used to measure the difference between the prior distribution of the latent space (which is assumed to be a standard normal distribution in this embodiment) and the posterior distribution output by the variational autoencoder.

[0046] In a specific embodiment, the KL divergence is determined by the following formula (2), including: (2) in, represents the KL divergence, represents the prior distribution data, Represents the posterior distribution data.

[0047] S204. According to the reconstruction loss and the KL divergence, the variational autoencoder is modified until the reconstruction loss and the KL divergence converge.

[0048] S205. Based on the first data to be processed and the second data to be processed, determine the transmission probability and cost function between the first posterior distribution data and the second posterior distribution data, and calculate the transmission cost value based on the transmission probability, the cost function and the transmission matrix, wherein the first posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the first data to be processed, the second posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the second data to be processed, and the transmission matrix is ​​organized by the first data to be processed and the second data to be processed.

[0049] S206: Filter out a transmission matrix that minimizes the transmission cost value, and align the first data to be processed and the second data to be processed based on the filtered transmission matrix to obtain personnel feature data.

[0050] In this embodiment, the first posterior distribution data is a representation of any first data to be processed. The probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of is expressed as The second posterior distribution data is used to characterize the given second data to be processed The probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of is expressed as The transmission matrix consists of the first data to be processed With the second data to be processed The transmission matrix which minimizes the transmission cost is obtained, thereby optimizing the transmission matrix.

[0051] In a specific embodiment, the cost function is determined by the following formula (3), including: (3) in, is the i-th first posterior distribution data, is the jth second posterior distribution data, Represents the cost function of the i-th first posterior distribution data and the j-th second posterior distribution data.

[0052] The transmission cost is determined by the following formula (4), including: (4) in, represents the transfer cost value, is the transmission matrix, Represents data from the first posterior distribution To the second posterior distribution data transmission probability.

[0053] Based on the above embodiments, in some embodiments, step S205 includes: Based on the transfer matrix, determining an entropy regularization term; Determining an optimization factor based on the entropy regularization term and a preset regularization parameter, wherein the regularization parameter is used to control the weight of the entropy regularization term; In the process of calculating the transmission cost value, the optimization factor is introduced to screen out a transmission matrix that minimizes the transmission cost value.

[0054] In this embodiment, by introducing an optimization factor in the calculation process of the transmission cost value, the calculation efficiency is improved, and the first to-be-processed data and the second to-be-processed data are aligned by optimizing the transmission matrix, thereby improving the consistency of data distribution.

[0055] Specifically, the optimization factor is introduced through the following formula: in, is the regularization parameter, represents the entropy regularization term.

[0056] Therefore, this embodiment further improves the calculation efficiency and the efficiency of transmission cost calculation by introducing an optimization factor composed of an entropy regularization term and a regularization parameter.

[0057] Furthermore, this embodiment optimizes the transmission cost value and extracts the transmission matrix that minimizes the transmission cost value, thereby realizing the alignment of the first data to be processed and the second data to be processed. Specifically, this embodiment calculates the transmission cost value through the transmission probability, the cost function and the transmission matrix, and takes into account the difference between the first data to be processed and the second data to be processed, which can effectively improve the accuracy of the calculation.

[0058] In some embodiments, step S103 includes: Based on the keys and key values ​​corresponding to the personnel feature data, calculating the feature similarity between any two keys; Determine the corresponding key as an associated key when the feature similarity exceeds a preset similarity threshold, and obtain a semantic relationship indicating the degree of semantic association between the keys; A connection is established between the associated keys to form a knowledge graph, wherein a node in the knowledge graph represents a key and its key value, and an edge in the knowledge graph represents a semantic relationship between two connected nodes.

[0059] In this embodiment, semantic representation is performed based on the embedding model, and the associated keys are matched by calculating the feature similarity between the keys. Optionally, the feature similarity is calculated by the following formula: in, represents the embedding vector of the a-th key, represents the embedding vector of the b-th key, Indicates the feature similarity between the a-th key and the b-th key.

[0060] Then, the association key is determined based on the feature similarity between the keys, achieving fine-grained semantic feature alignment. For example, "age" and "date of birth" from different data sources are determined as association keys, and "income" in financial data and "occupation" in social data are determined as association keys.

[0061] Furthermore, a knowledge graph is constructed based on the associated keys and the semantic relationship between the keys. The semantic relationship can represent the association between the keys (i.e., personnel feature data) or the degree of association between the keys. For example, taking the population data type as an example, a knowledge graph is constructed, the key "age" (key 1) value is 19, and the key "date of birth" (key 2) value is 1992-11-1. The nodes in the knowledge graph represent the keys, and the edges represent the semantic relationship between the keys. The relationship between "age" and "date of birth" is "derivable", and the relationship between "income" and "occupation" is "strongly associated".

[0062] Therefore, this embodiment constructs a knowledge graph through the relationship between keys, extracts, integrates and stores the entities, attributes and relationships in the personnel feature data, and intuitively displays the associations between the personnel features of the activity personnel.

[0063] Based on the above embodiment, in some embodiments, step S103 further includes: Mapping the nodes and edges in the knowledge graph to a vector space to obtain corresponding node embedding vectors and edge embedding vectors; Calculating the regional correlation between the node embedding vector and the regional attribute, and selecting the node embedding vector corresponding to the regional correlation exceeding a preset correlation threshold as the regional correlation feature; The region association feature is used as a node in the region data graph, and the edge in the region data graph is generated according to the edge embedding vector corresponding to the screened node embedding vector to form the region data graph.

[0064] In this embodiment, the nodes (i.e., keys, personnel feature data) and edges of the knowledge graph are embedded into the vector space to obtain node embedding vectors and edge embedding vectors. Then, it is judged whether the key has regional attributes to extract regional association features from the personnel feature data and construct a regional data graph. Specifically, the regional association between the node embedding vector and the regional attribute is calculated, and the node embedding vectors (i.e., keys, personnel feature data) with higher association are screened out as regional association features. Then, the screened node embedding vectors and the corresponding edge embedding vectors are extracted, and the node embedding vectors are used as nodes in the regional data graph. The edges in the regional data graph are generated according to the corresponding edge embedding vectors to form a regional data graph. Optionally, the regional association between the node embedding vector and the regional attribute is calculated by template matching to quickly determine the regional association and quickly screen out regional association features.

[0065] Therefore, this embodiment extracts regional association features related to regional attributes in the knowledge graph, constructs a regional data graph based on the regional association features, and takes into account the impact of regional characteristics of activity personnel on target social activities, thereby effectively improving the accuracy of activity analysis.

[0066] In some embodiments, step S104 includes: Using a preset graph convolutional network, feature extraction is performed on the knowledge graph and the regional data graph respectively to generate a first feature matrix and a second feature matrix, wherein the first feature matrix is ​​used to indicate nodes in the knowledge graph, and the second feature matrix is ​​used to indicate nodes in the regional data graph; Using a preset fusion self-attention mechanism, the first feature matrix and the second feature matrix are associated to obtain an associated value; When the association value is greater than a preset association threshold, the corresponding nodes in the knowledge graph and the regional data graph are connected, and the association value is used as the weight of the edge connecting the nodes to form a target data graph, which is used to indicate the prediction of the risk of the target social activity; The target data graph is analyzed using the graph convolutional network to obtain an activity analysis result of the target social activity.

[0067] In this embodiment, a graph convolutional network is used to extract features from the knowledge graph and the regional data graph to obtain a feature matrix of each node. In a specific embodiment, the first feature matrix is ​​determined by the following formula (5), including: (5) in, Indicates the first The first characteristic matrix of the layer, represents the activation function of the graph convolutional network, represents the adjacency matrix of the knowledge graph, represents the degree matrix of the adjacency matrix of the knowledge graph, Indicates the first The first characteristic matrix of the layer, Indicates the first Layer weight matrix.

[0068] In another specific embodiment, the second characteristic matrix is ​​determined by the following formula (6), including: (6) in, Indicates the area data in the figure The second characteristic matrix of the layer, represents the adjacency matrix of the region data graph, a degree matrix representing the adjacency matrix of the region data graph, Indicates the area data in the figure The second characteristic matrix of the layer, Indicates the area data in the figure Layer weight matrix.

[0069] Then, the first feature matrix and the second feature matrix are associated by using the fusion self-attention mechanism. Specifically, the association value is determined by the following formula (7), including: (7) in, represents the associated value, Represents the fusion self-attention mechanism function, represents the attention vector, represents the transposed matrix of the attention vector, represents a weight matrix associated with the regional data graph, Represents the weight matrix associated with the knowledge graph.

[0070] It should be noted that the association value can be expressed as a potential edge weight. When the association value is greater than the association threshold, the corresponding nodes in the knowledge graph and the regional data graph are connected, and the association value is used as the weight of the edge connecting the nodes. At this time, the knowledge graph and the regional data graph are merged into a new graph structure, namely the target data graph. Therefore, the target social activities are predicted and evaluated based on the target data graph.

[0071] Therefore, this embodiment integrates the knowledge graph and the regional data graph into a target data graph, and performs activity analysis on the target social activity based on the target data graph. Taking into account the association between the nodes in the knowledge graph and the regional data graph, combined with the characteristics of the participants themselves (social characteristics) and the associated regional characteristics, the accuracy of the activity analysis is improved.

[0072] Figure 3 A schematic diagram of the structure of a cross-media intelligent recognition system based on social characteristics of population data provided in an embodiment of the present application, the cross-media intelligent recognition system based on social characteristics of population data 300 includes: A data acquisition module 301 is used to acquire personnel data sent by multiple data sources, where the personnel data is used to characterize relevant data of personnel participating in a target social activity; A data analysis module 302 is used to parse and align the personnel data to obtain personnel feature data; A feature analysis module 303 is used to semantically align the personnel feature data, construct a knowledge graph for the activity personnel, and construct a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, and the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to the preset regional attributes; The activity analysis module 304 is used to determine the activity analysis result based on the knowledge graph and the regional data graph, and the activity analysis result is used to evaluate the risk of the target social activity.

[0073] In some embodiments, the data parsing module 302 includes: A first parsing unit is configured to, when the personnel data is represented in JSON format, sequentially match the keys identified in the personnel data with a preset key set, and superimpose the matched keys and corresponding key values ​​to obtain first data to be processed, wherein the keys are used to identify data related to a specified feature in the personnel data, and the key values ​​are used to indicate the personnel data associated with the keys; a second parsing unit, configured to, when the personnel data is represented as communication data of a specified protocol, traverse all time windows, match the keys identified in the personnel data acquired within the time window with the key set, aggregate the matched keys and corresponding key values, and obtain second data to be processed, wherein the time window is determined by a preset time window length; A data alignment unit is used to align the first data to be processed and the second data to be processed to obtain personnel feature data.

[0074] In some embodiments, the data alignment unit includes: an encoding unit, configured to extract features from the first data to be processed and the second data to be processed using a preset variational autoencoder to obtain a feature vector; A reconstruction unit, used to reconstruct data of the feature vector using a preset decoder and determine a reconstruction loss; A divergence calculation unit, configured to calculate a KL divergence based on the prior distribution data determined by the feature vector and the posterior distribution data obtained by the variational autoencoder; A correction unit, configured to correct the variational autoencoder according to the reconstruction loss and the KL divergence until the reconstruction loss and the KL divergence converge; a transmission cost calculation unit, configured to determine a transmission probability and a cost function between first posterior distribution data and second posterior distribution data based on the first data to be processed and the second data to be processed, and to calculate a transmission cost value according to the transmission probability, the cost function and a transmission matrix, wherein the first posterior distribution data is used to characterize the probability distribution of a feature vector obtained by the modified variational autoencoder under the condition of given any of the first data to be processed, the second posterior distribution data is used to characterize the probability distribution of a feature vector obtained by the modified variational autoencoder under the condition of given any of the second data to be processed, and the transmission matrix is ​​organized by the first data to be processed and the second data to be processed; The alignment unit is used to select a transmission matrix that minimizes the transmission cost value, and align the first data to be processed and the second data to be processed based on the selected transmission matrix to obtain personnel feature data.

[0075] In some embodiments, the data parsing module 302 further includes: The reconstruction loss calculation unit is used to determine the reconstruction loss through the following formula (1), including: (1) in, is the reconstruction loss, represents one of the first data to be processed and the second data to be processed, is the feature vector, represents the reconstructed data obtained after reconstruction by the decoder, represents a specified trainable parameter in the decoder; The KL divergence calculation unit is used to determine the KL divergence according to the following formula (2), including: (2) in, represents the KL divergence, represents the prior distribution data, represents the posterior distribution data; The cost function calculation unit is used to determine the cost function through the following formula (3), including: (3) in, is the i-th first posterior distribution data, is the jth second posterior distribution data, Represents the cost function of the i-th first posterior distribution data and the j-th second posterior distribution data; The transmission cost value calculation unit is used to determine the transmission cost value by the following formula (4), including: (4) in, represents the transfer cost value, is the transmission matrix, represents the transmission probability from the i-th first posterior distribution data to the j-th second posterior distribution data.

[0076] In some embodiments, the feature analysis module 303 includes: A feature similarity calculation unit, used to calculate the feature similarity between any two keys based on the keys and key values ​​corresponding to the personnel feature data; An associated key extraction unit, used to determine the corresponding key as an associated key when the feature similarity exceeds a preset similarity threshold, and obtain a semantic relationship indicating the degree of semantic association between the keys; The knowledge graph construction unit is used to establish connections between the associated keys to form a knowledge graph, wherein a node in the knowledge graph represents a key and its key value, and an edge in the knowledge graph represents a semantic relationship between two connected nodes.

[0077] In some embodiments, the feature analysis module 303 further includes: An embedding unit, used to map the nodes and edges in the knowledge graph to a vector space to obtain corresponding node embedding vectors and edge embedding vectors; A region association feature extraction unit, used to calculate the region association degree between the node embedding vector and the region attribute, and filter out the node embedding vector corresponding to the region association degree exceeding a preset association threshold as the region association feature; A regional data graph construction unit is used to use the regional association feature as a node in the regional data graph, and generate an edge in the regional data graph according to an edge embedding vector corresponding to the screened node embedding vector to form the regional data graph.

[0078] In some embodiments, the activity analysis module 304 includes: A node feature extraction unit, used to use a preset graph convolutional network to perform feature extraction on the knowledge graph and the regional data graph, respectively, to generate a first feature matrix and a second feature matrix, wherein the first feature matrix is ​​used to indicate nodes in the knowledge graph, and the second feature matrix is ​​used to indicate nodes in the regional data graph; an associating unit, configured to associate the first feature matrix and the second feature matrix using a preset fusion self-attention mechanism to obtain an associated value; A target data graph construction unit, used to establish a connection between the corresponding nodes in the knowledge graph and the regional data graph when the association value is greater than a preset association threshold, and use the association value as the weight of the edge connecting the nodes to form a target data graph, wherein the target data graph is used to indicate a prediction of the risk of the target social activity; An activity analysis result generating unit is used to analyze the target data graph using the graph convolutional network to obtain an activity analysis result of the target social activity.

[0079] In some embodiments, the activity analysis module 304 includes: The first characteristic matrix calculation unit is used to determine the first characteristic matrix by the following formula (5), including: (5) in, Indicates the first The first characteristic matrix of the layer, represents the activation function of the graph convolutional network, represents the adjacency matrix of the knowledge graph, represents the degree matrix of the adjacency matrix of the knowledge graph, Indicates the first The first characteristic matrix of the layer, Indicates the first layer weight matrix; The second characteristic matrix calculation unit is used to determine the second characteristic matrix by the following formula (6), including: (6) in, Indicates the area data in the figure The second characteristic matrix of the layer, represents the adjacency matrix of the region data graph, a degree matrix representing the adjacency matrix of the region data graph, Indicates the area data in the figure The second characteristic matrix of the layer, Indicates the area data in the figure layer weight matrix; The correlation value calculation unit is used to determine the correlation value by the following formula (7), including: (7) in, represents the associated value, Represents the fusion self-attention mechanism function, represents the attention vector, represents the transposed matrix of the attention vector, represents a weight matrix associated with the regional data graph, Represents the weight matrix associated with the knowledge graph.

[0080] In a possible implementation, the data alignment unit further includes: An entropy regularization term calculation unit, configured to determine an entropy regularization term based on the transmission matrix; An optimization factor construction unit, used to determine an optimization factor based on the entropy regularization term and a preset regularization parameter, wherein the regularization parameter is used to control the weight of the entropy regularization term; The calculation unit is used to introduce the optimization factor in the calculation process of the transmission cost value, so as to screen out the transmission matrix that minimizes the transmission cost value.

[0081] The system of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the system of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the system, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0082] In the description of the embodiments of the present invention, the terms "first", "second", "third", and "fourth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cross-media intelligent recognition method based on the social characteristics of population data, characterized in that: include: Acquire personnel data sent by multiple data sources, where the personnel data is used to characterize relevant data of personnel participating in a target social activity; Parsing and aligning the personnel data to obtain personnel feature data; Semantically aligning the personnel feature data, constructing a knowledge graph for the activity personnel, and constructing a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to the preset regional attributes; Based on the knowledge graph and the regional data graph, an activity analysis result is determined, and the activity analysis result is used to evaluate the risk of the target social activity.

2. The cross-media intelligent recognition method based on social characteristics of population data according to claim 1 is characterized in that: The parsing and alignment of the personnel data to obtain personnel feature data includes: When the personnel data is represented in JSON format, the keys identified in the personnel data are matched with the preset key set in sequence, and the matched keys and corresponding key values ​​are superimposed to obtain the first data to be processed, wherein the key is used to identify the data related to the specified feature in the personnel data, and the key value is used to indicate the personnel data associated with the key; When the personnel data is represented as communication data of a specified protocol, all time windows are traversed, the keys identified in the personnel data obtained within the time window are matched with the key set, and the matched keys and corresponding key values ​​are aggregated to obtain second data to be processed, wherein the time window is determined by a preset time window length; The first data to be processed and the second data to be processed are aligned to obtain personnel feature data.

3. The cross-media intelligent recognition method based on social characteristics of population data according to claim 2 is characterized in that: The step of aligning the first data to be processed and the second data to be processed to obtain personnel characteristic data includes: Using a preset variational autoencoder to perform feature extraction on the first to-be-processed data and the second to-be-processed data to obtain a feature vector; Reconstructing the feature vector using a preset decoder and determining a reconstruction loss; Calculate the KL divergence based on the prior distribution data determined by the feature vector and the posterior distribution data obtained by the variational autoencoder; According to the reconstruction loss and the KL divergence, the variational autoencoder is modified until the reconstruction loss and the KL divergence converge; Based on the first data to be processed and the second data to be processed, determine the transmission probability and cost function between the first posterior distribution data and the second posterior distribution data, and calculate the transmission cost value according to the transmission probability, the cost function and the transmission matrix, wherein the first posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the first data to be processed, the second posterior distribution data is used to characterize the probability distribution of the feature vector obtained by the modified variational autoencoder under the condition of given any of the second data to be processed, and the transmission matrix is ​​organized by the first data to be processed and the second data to be processed; A transmission matrix that minimizes the transmission cost value is screened out, and the first data to be processed and the second data to be processed are aligned based on the screened transmission matrix to obtain personnel feature data.

4. The cross-media intelligent recognition method based on social characteristics of population data according to claim 3 is characterized in that: The step of aligning the first data to be processed with the second data to be processed to obtain personnel feature data further includes: The reconstruction loss is determined by the following formula (1), including: （1） in, is the reconstruction loss, represents one of the first data to be processed and the second data to be processed, is the feature vector, represents the reconstructed data obtained after reconstruction by the decoder, represents a specified trainable parameter in the decoder; The KL divergence is determined by the following formula (2), including: （2） in, represents the KL divergence, represents the prior distribution data, represents the posterior distribution data; The cost function is determined by the following formula (3), including: （3） in, is the i-th first posterior distribution data, is the jth second posterior distribution data, Represents the cost function of the i-th first posterior distribution data and the j-th second posterior distribution data; The transmission cost is determined by the following formula (4), including: （4） in, represents the transfer cost value, is the transmission matrix, represents the transmission probability from the i-th first posterior distribution data to the j-th second posterior distribution data.

5. The cross-media intelligent recognition method based on social characteristics of population data according to claim 4 is characterized in that: The semantic alignment of the personnel feature data to construct a knowledge graph for the activity personnel includes: Based on the keys and key values ​​corresponding to the personnel feature data, calculating the feature similarity between any two keys; Determine the corresponding key as an associated key when the feature similarity exceeds a preset similarity threshold, and obtain a semantic relationship indicating the degree of semantic association between the keys; A connection is established between the associated keys to form a knowledge graph, wherein a node in the knowledge graph represents a key and its key value, and an edge in the knowledge graph represents a semantic relationship between two connected nodes.

6. The cross-media intelligent recognition method based on social characteristics of population data according to claim 5 is characterized in that: The step of constructing a regional data graph based on the knowledge graph includes: Mapping the nodes and edges in the knowledge graph to a vector space to obtain corresponding node embedding vectors and edge embedding vectors; Calculating the regional correlation between the node embedding vector and the regional attribute, and selecting the node embedding vector corresponding to the regional correlation exceeding a preset correlation threshold as the regional correlation feature; The region association feature is used as a node in the region data graph, and the edge in the region data graph is generated according to the edge embedding vector corresponding to the screened node embedding vector to form the region data graph.

7. The cross-media intelligent recognition method based on social characteristics of population data according to claim 6 is characterized in that: The determining of the activity analysis result based on the knowledge graph and the regional data graph includes: Using a preset graph convolutional network, feature extraction is performed on the knowledge graph and the regional data graph respectively to generate a first feature matrix and a second feature matrix, wherein the first feature matrix is ​​used to indicate nodes in the knowledge graph, and the second feature matrix is ​​used to indicate nodes in the regional data graph; Using a preset fusion self-attention mechanism, the first feature matrix and the second feature matrix are associated to obtain an associated value; When the association value is greater than a preset association threshold, the corresponding nodes in the knowledge graph and the regional data graph are connected, and the association value is used as the weight of the edge connecting the nodes to form a target data graph, which is used to indicate the prediction of the risk of the target social activity; The target data graph is analyzed using the graph convolutional network to obtain an activity analysis result of the target social activity.

8. The cross-media intelligent recognition method based on social characteristics of population data according to claim 7 is characterized in that: The determining of the activity analysis result based on the knowledge graph and the regional data graph further includes: The first characteristic matrix is ​​determined by the following formula (5), including: （5） in, Indicates the first The first characteristic matrix of the layer, represents the activation function of the graph convolutional network, represents the adjacency matrix of the knowledge graph, represents the degree matrix of the adjacency matrix of the knowledge graph, Indicates the first The first characteristic matrix of the layer, Indicates the first layer weight matrix; The second characteristic matrix is ​​determined by the following formula (6), including: （6） in, Indicates the area data in the figure The second characteristic matrix of the layer, represents the adjacency matrix of the region data graph, a degree matrix representing the adjacency matrix of the region data graph, Indicates the area data in the figure The second characteristic matrix of the layer, Indicates the area data in the figure layer weight matrix; The correlation value is determined by the following formula (7), including: （7） in, represents the associated value, Represents the fusion self-attention mechanism function, represents the attention vector, represents the transposed matrix of the attention vector, represents a weight matrix associated with the regional data graph, Represents the weight matrix associated with the knowledge graph.

9. The cross-media intelligent recognition method based on social characteristics of population data according to claim 8, characterized in that: The step of aligning the first data to be processed with the second data to be processed to obtain personnel feature data further includes: Based on the transfer matrix, determining an entropy regularization term; Determining an optimization factor based on the entropy regularization term and a preset regularization parameter, wherein the regularization parameter is used to control the weight of the entropy regularization term; In the process of calculating the transmission cost value, the optimization factor is introduced to screen out a transmission matrix that minimizes the transmission cost value.

10. A cross-media intelligent recognition system based on the social characteristics of population data, characterized in that: include: A data acquisition module, used to acquire personnel data sent by multiple data sources, wherein the personnel data is used to characterize relevant data of personnel participating in a target social activity; A data analysis module, used to parse and align the personnel data to obtain personnel feature data; A feature analysis module, used to semantically align the personnel feature data, construct a knowledge graph for the activity personnel, and construct a regional data graph based on the knowledge graph, wherein the knowledge graph is used to indicate the semantic relationship between the personnel feature data, the regional data graph is used to indicate the correlation between regional association features, and the regional association features are used to characterize the personnel feature data related to the preset regional attributes; The activity analysis module is used to determine the activity analysis results based on the knowledge graph and the regional data graph, and the activity analysis results are used to evaluate the risk of the target social activity.

Images