An intelligent SMS voice conversion method based on machine learning

Abstract

The present invention relates to the field of machine learning technology, and in particular to an intelligent SMS voice conversion method based on machine learning. The method comprises the following steps: obtaining SMS text data, SMS voice data and user mobile route data, and performing feature extraction to obtain SMS text features, SMS voice features and user mobile route features. Utilizing these features to perform graph fusion, and constructing SMS text and voice fusion graph feature data. Performing graph convolution calculation based on the fusion graph feature data, obtaining text and voice graph convolution data, and obtaining text and voice fully connected layer data through graph pooling and full connection calculation. Performing privacy processing on user mobile route data, and generating user mobile location privacy data. Combining the user mobile location privacy data with the text and voice fully connected layer data, constructing and iteratively training an intelligent SMS voice conversion model, and realizing accurate SMS voice conversion operations. The present invention improves the accuracy of conversion and protects user privacy.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to an intelligent text message voice conversion method based on machine learning. Background Art

[0002] In the current field of communication and information processing, intelligent SMS voice conversion technology has become an important research direction. With the popularization of mobile Internet and smart devices, users' demand for convenient and efficient communication methods is increasing, and intelligent SMS voice conversion technology has emerged. However, traditional SMS and voice conversion technologies often face some problems in practical applications. Traditional text-to-speech (TTS) and speech recognition technologies are prone to errors when dealing with complex contexts and diverse accents, resulting in low conversion accuracy. The generalization ability of the model is limited, and it is difficult to handle variations in different users and environments. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes an intelligent SMS voice conversion method based on machine learning to solve at least one of the above technical problems.

[0004] The present application provides an intelligent SMS voice conversion method based on machine learning, comprising the following steps:

[0005] Step S1: Acquire SMS text data, SMS voice data and user movement route data;

[0006] Step S2: performing feature extraction based on the SMS text data, SMS voice data and user movement route data to obtain SMS text feature data, SMS voice feature data and user movement route feature data, and performing graph fusion based on the SMS text feature data, SMS voice feature data and user movement route feature data to obtain SMS text and voice fusion graph feature data;

[0007] Step S3: performing graph convolution calculation based on the SMS text-speech fusion graph feature data to obtain text-speech graph convolution data, performing graph pooling processing based on the text-speech graph convolution data to obtain text-speech graph pooling data, performing full connection calculation based on the text-language graph pooling data to obtain text-speech fully connected layer data;

[0008] Step S4: Perform user location privacy processing based on the user movement route data to obtain the user movement location privacy data, and build and iteratively train a model based on the user movement location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS speech conversion model to perform intelligent SMS speech conversion operations.

[0009] In the present invention, by obtaining multiple data sources, including text, voice and user movement routes, it is ensured that the model has rich data input, and the generalization ability and accuracy of the model are improved. Acquiring data related to user movement behavior can better understand the user's context, thereby improving the accuracy and relevance of SMS voice conversion. Through feature extraction, the multi-dimensional features of text, voice and user movement routes are obtained to ensure the diversity and richness of input data. Using graph fusion technology, the features of different data sources are fused to construct SMS text and voice fusion graph feature data, thereby capturing the complex relationship between data and improving the representation ability of the model. Through graph fusion technology, the user's movement route information can be better utilized to enhance the contextual understanding ability of voice conversion. Through graph convolution calculation, the local and global features of graph fusion data can be better extracted to improve the ability of feature expression. Graph pooling processing can effectively reduce feature dimensions, retain important information, reduce computational complexity, and avoid overfitting. Through full connection calculation, the extracted features are further fused and abstracted to obtain more accurate text and voice full connection layer data, providing high-quality input for subsequent model construction. Through privacy processing, user mobile data is protected to ensure that user privacy is not leaked, thereby increasing the security of the model and user trust. Based on the user's mobile route and privacy data, more customized and personalized voice conversion services can be provided to improve user experience. Through iterative training of the model, the performance of the intelligent SMS voice conversion model can be continuously optimized to improve the accuracy and efficiency of the conversion.

[0010] Preferably, step S1 specifically includes:

[0011] Step S11: collecting primary text data from a short message service provider to obtain primary short message text data;

[0012] Step S12: collecting primary voice data from a public voice database to obtain primary SMS voice data; or, performing voice conversion based on the primary SMS text data to obtain primary SMS voice data;

[0013] Step S13: collecting user movement route data according to the primary SMS text data to obtain user movement route data;

[0014] Step S14: Associating the primary SMS text data and the primary SMS voice data according to the user movement route data to obtain SMS voice associated data and SMS text associated data;

[0015] Step S15: performing deduplication processing on the SMS voice associated data and the SMS text associated data to obtain SMS voice data and SMS text data.

[0016] The present invention integrates information from different data sources, including SMS text, voice and user movement route data, providing a comprehensive data foundation for subsequent processing. By generating related data, the user's behavior and needs in specific situations can be better understood, and the utilization value of the data can be improved. Deduplication processing can ensure the uniqueness of the data and avoid interference of duplicate information on model training and result analysis. From primary data to related data and then to deduplication processing, the entire step ensures the accuracy and reliability of the data and improves the effect of subsequent analysis and application. Obtaining user movement route data in an authorized manner ensures the security of user privacy information and enhances the user's trust in the service.

[0017] Preferably, step S2 specifically includes:

[0018] Extract features based on the SMS text data, SMS voice data, and user movement route data to obtain SMS text feature data, SMS voice feature data, and user movement route feature data;

[0019] Constructing a correlation graph based on the SMS text feature data and the user's mobile route feature data to obtain SMS text route correlation feature graph data;

[0020] Constructing a correlation graph based on the SMS voice feature data and the user's mobile route feature data to obtain SMS voice route correlation feature graph data;

[0021] Graph fusion is performed based on the SMS text route associated feature graph data and the SMS voice route associated feature graph data to obtain SMS text and voice fusion graph feature data.

[0022] In the present invention, SMS text, voice and user mobile route data are fused to obtain more comprehensive and rich feature data, which helps to improve the expressiveness and prediction accuracy of the model. By constructing an associated feature graph, the associated information between different data sources can be better captured, and the model's ability to understand and analyze data is improved. Through graph fusion technology, the features of different data sources are fused, which can better capture the complex relationship between data and improve the representation and generalization capabilities of the model. Fusion of features from different data sources can improve the diversity and expressiveness of features, which helps to more accurately describe the characteristics of data.

[0023] Preferably, step S3 specifically includes:

[0024] Perform adjacency matrix normalization processing on the SMS text and voice fusion graph feature data to obtain graph feature normalized data;

[0025] Calculate the information entropy of the graph feature normalized data to obtain the graph feature information entropy data;

[0026] When it is determined that the graph feature information entropy data is greater than or equal to the preset graph feature information entropy threshold data, the graph feature information entropy data and the graph feature normalization data are feature spliced ​​to obtain graph feature adjustment data;

[0027] When it is determined that the graph feature information entropy data is less than the preset graph feature information entropy threshold data, the graph feature information entropy data is determined as the graph feature adjustment data;

[0028] According to the graph feature information entropy data and the preset graph convolution activation function data, the activation function parameters are mapped to obtain the graph convolution activation function data;

[0029] According to the graph feature adjustment data, the graph convolution calculation is performed through the graph convolution activation function to obtain the text speech graph convolution data;

[0030] Perform graph pooling processing based on graph feature information entropy data and text-speech graph convolution data to obtain text-speech graph pooling data;

[0031] Perform full connection calculation based on the text language graph pooling data to obtain the text speech full connection layer data.

[0032] In the present invention, through information entropy calculation and feature splicing, the graph feature data can be dynamically adjusted to improve the feature expression ability, which is helpful to improve the performance and generalization ability of the model. According to the information entropy data, the activation function parameters are mapped, and the activation function can be dynamically adjusted according to the characteristics of the data to improve the adaptability and effect of the model. Through graph convolution calculation, the graph structure information can be better utilized to extract the relationship between the data, and the model's ability to understand the data is enhanced. Graph pooling processing can reduce the data dimension, retain important information, reduce the calculation complexity, and avoid overfitting, which is helpful to improve the generalization ability of the model. Through full connection calculation, the features obtained by graph convolution calculation can be further fused and abstracted to obtain a higher level of feature representation, which is helpful to improve the performance and effect of the model. The present invention introduces an information entropy threshold mechanism to perform adaptive feature adjustment according to the graph feature information entropy data. For data with information entropy higher than the threshold, feature splicing is performed to ensure the retention of important information; for data with information entropy lower than the threshold, simple feature adjustment is performed to reduce unnecessary calculation overhead, so that the model is more flexible and efficient when processing different data.

[0033] Preferably, step S4 is specifically:

[0034] Performing privacy assessment based on user movement route data to obtain privacy assessment data;

[0035] Performing privacy labeling on the user's mobile route data according to the privacy assessment data to obtain privacy labeling data;

[0036] Performing privacy masking on high-privacy location data in the privacy-labeled data to obtain privacy masked data;

[0037] Extract privacy features from the privacy mask data to obtain privacy data of the user's moving location;

[0038] The model is constructed and iteratively trained based on the user's mobile location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS speech conversion model for performing intelligent SMS speech conversion operations.

[0039] The present invention can effectively protect the user's privacy information and avoid the leakage of privacy information through privacy assessment, labeling and mask generation. Feature extraction and mask processing of private information can enhance the security of data and reduce the risk of sensitive information leakage. Using private data to guide model training can improve the model's sensitivity to user privacy and enhance the model's effect in privacy protection. Through private data analysis, users' behaviors and preferences can be better understood, and customized services and suggestions can be provided to users.

[0040] Preferably, the SMS voice data includes first SMS voice data and second SMS voice data, and the SMS text data includes first SMS text data and second SMS text data, and the deduplication process is specifically as follows:

[0041] Performing high correlation deduplication processing on the SMS voice-related data and the SMS text-related data to obtain first SMS voice data and first SMS text data respectively;

[0042] Calculate the information entropy of the SMS voice-related data and the SMS text-related data to obtain SMS voice-related information entropy data and SMS text-related information entropy data respectively;

[0043] Information entropy deduplication processing is performed on the SMS voice-related data and the SMS text-related data according to the SMS voice-related information entropy data and the SMS text-related information entropy data to obtain second SMS voice data and second SMS text data.

[0044] In the present invention, duplicate data can be removed through deduplication processing, the quality and accuracy of data can be improved, and the complexity of data processing and analysis can be reduced. Deduplication processing can avoid waste of resources, ensure the effective use of resources, and improve the value and utilization efficiency of data. Deduplication processing can reduce errors caused by data duplication, improve the accuracy of data processing and analysis, and enhance the reliability and stability of the model. Deduplication processing can reduce the processing and calculation of duplicate data, save computing resources, and improve computing efficiency and performance. Deduplication processing can be performed based on information entropy data to more intelligently remove redundant data. For data with higher information entropy, it is retained as the second SMS voice data and the second SMS text data to ensure that important information is retained. Compared with traditional simple deduplication methods, information entropy deduplication processing can more effectively remove useless or duplicate information, retain high-value data, and improve the performance and accuracy of the model.

[0045] Preferably, the SMS text and speech fusion graph feature data includes first SMS text and speech fusion graph feature data and second SMS text and speech fusion graph feature data, and the graph fusion is specifically:

[0046] According to the SMS text route associated feature graph data and the SMS voice route associated feature graph data, feature matrix concatenation and adjacency matrix addition are performed to obtain first SMS text and voice fusion graph feature data;

[0047] The self-attention weight is calculated based on the SMS text route associated feature map data and the SMS voice route associated feature map data to obtain the text route feature self-attention weight data and the voice route feature self-attention weight data respectively;

[0048] The text route feature self-attention weight data and the voice route feature self-attention weight data are used to perform feature graph weighted calculation on the SMS text route associated feature graph data and the SMS voice route associated feature graph data and add the adjacency matrix to obtain the second SMS text and voice fusion graph feature data.

[0049] In the present invention, through graph fusion processing, the feature information of SMS text and voice can be effectively fused, the dimension and diversity of data can be improved, and the model's ability to express data can be enhanced. Through self-attention weight calculation and feature graph weighted calculation, features can be weighted according to their importance, the quality and effectiveness of the fused features can be improved, and the model's attention and discrimination to data can be enhanced. The fused feature data can be used for model training, improving the model's understanding and processing capabilities for SMS voice conversion tasks, and enhancing the model's generalization and adaptability.

[0050] Preferably, the graph feature information entropy data includes first graph feature information entropy data and second graph feature information entropy data, and the information entropy calculation is specifically as follows:

[0051] Extract the changed nodes from the graph feature normalized data to obtain the changed node data;

[0052] Calculate information entropy based on the changed node data to obtain feature information entropy data of the first graph;

[0053] Perform Gaussian clustering calculation on the graph feature normalized data to obtain graph feature clustering data;

[0054] Perform feature extraction on the graph feature clustering data to obtain graph feature clustering feature data;

[0055] The information entropy of the graph feature clustering feature data is calculated to obtain second graph feature information entropy data.

[0056] In the present invention, by extracting the changed nodes and calculating the information entropy, the change and distribution of the nodes in the graph can be analyzed, which is helpful to understand the evolution and development trend of the graph. By Gaussian clustering calculation and feature extraction, the nodes in the graph can be clustered and grouped, which is convenient for analyzing and understanding the structure and features of the graph. Feature extraction can abstract and refine the features of the nodes in the graph to obtain more representative and distinguishing features, which helps to improve the expression and generalization capabilities of the model. Information entropy calculation can measure the complexity and diversity of the graph features, help to understand the features and structure of the graph, and provide an important reference for subsequent analysis and application. By extracting the changed nodes from the normalized graph feature data, the present invention can identify the nodes with significant changes in the graph, which often contain key information. The extraction of changed nodes can use methods such as difference analysis and mutation detection to improve the precision and accuracy of feature analysis. By calculating the information entropy of the changed node data, the first graph feature information entropy data is obtained, which can measure the uncertainty and information content of the changed nodes. Nodes with high information entropy usually contain more useful information. In this way, important feature data can be effectively identified and retained, and the accuracy of the model can be improved. The information entropy of the graph feature clustering feature data is calculated to obtain the second graph feature information entropy data, which can measure the uncertainty and diversity of the data at the clustering level. Multi-level information entropy analysis enables the model to more comprehensively capture and utilize useful information in the data, improving the quality of feature representation and the prediction performance of the model.

[0057] Preferably, the privacy assessment is specifically:

[0058] Extracting user behavior features based on user movement route data to obtain user movement route behavior feature data, wherein the user movement route behavior feature data includes geographic location feature data, time feature data, and user behavior feature data, and the user behavior feature data includes visit location frequency data and stay time data;

[0059] Perform feature distribution fitting based on user movement route behavior feature data to obtain a user privacy evaluation model;

[0060] Calculate the privacy weight according to the user privacy assessment model to obtain the privacy weight data;

[0061] Risk calculation is performed based on the privacy weight data and the user privacy assessment data to obtain the privacy assessment data.

[0062] The present invention can perform personalized assessment of user privacy through user behavior feature extraction and privacy assessment model, and more accurately reflect the user's privacy risk. Through privacy weight calculation, the impact of different behavior features on user privacy can be analyzed, providing a reference for the formulation of privacy protection strategies. Through risk calculation, the user's privacy risk can be predicted, potential privacy leakage risks can be discovered early, and the effect of privacy protection can be improved.

[0063] Preferably, the privacy assessment data includes node privacy assessment data and route privacy assessment data, and the risk calculation is specifically as follows:

[0064] Calculate the local node risk based on the privacy weight data, user privacy assessment data, and user movement route behavior feature data to obtain node privacy assessment data;

[0065] Route node risks are calculated based on privacy weight data, user privacy assessment data, and user mobile route behavior feature data to obtain route privacy assessment data.

[0066] In the present invention, the privacy assessment data of each node is calculated based on the privacy weight data, the user privacy assessment data and the user's mobile route behavior characteristic data, which helps determine the privacy risk level of each node and the degree of impact of the node on the user's privacy. According to the privacy weight data, the user privacy assessment data and the user's mobile route behavior characteristic data, the privacy assessment data of the entire route is calculated to help determine the privacy risk level of the entire route and the overall impact of the route on the user's privacy. Through the node privacy assessment data and the route privacy assessment data, the privacy risk of the user during the movement is evaluated in multiple dimensions, which helps to better understand the user's privacy status. Through the node privacy assessment data and the route privacy assessment data, it can help to accurately locate the possible locations and routes of user privacy leakage, which helps to formulate targeted privacy protection strategies. Through the privacy assessment data of different nodes and routes, personalized privacy protection solutions can be provided for different users to improve the effect of user privacy protection.

[0067] The beneficial effects of the present invention are as follows: by integrating SMS text data, SMS voice data and user mobile route data, and extracting features using multi-source data, it helps to improve the performance and accuracy of the model. By adopting a graph fusion method, SMS text features, SMS voice features and user mobile route features are integrated to better capture the correlation and complexity between data and improve the expressiveness of features. The introduction of a graph convolutional neural network to calculate the fused features can effectively process unstructured data and improve the ability to understand and process data. By performing privacy processing on user mobile route data, the user's privacy information is protected and the acceptability and compliance of the model in practical applications are improved. By adopting a model iterative training method, the model parameters are continuously optimized, the performance and generalization ability of the model are improved, and the intelligent SMS voice conversion model is made more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0069] Figure 1 A flowchart of a method for intelligent SMS voice conversion based on machine learning according to an embodiment of the present invention is shown;

[0070] Figure 2 A flowchart showing a method for collecting basic SMS voice data according to an embodiment of the present invention is provided;

[0071] Figure 3 A flowchart of the steps of a method for generating a text-speech fusion graph feature of a short message according to an embodiment is shown;

[0072] Figure 4 A flowchart of the steps of a method for deep processing of text and speech fusion graph features of a short message according to an embodiment is shown;

[0073] Figure 5 A flowchart of a method for constructing an intelligent SMS voice conversion model according to an embodiment is shown. DETAILED DESCRIPTION

[0074] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0076] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0077] See also Figures 1 to 5 The present application provides an intelligent SMS voice conversion method based on machine learning, comprising the following steps:

[0078] Step S1: Acquire SMS text data, SMS voice data and user movement route data;

[0079] Step S2: performing feature extraction based on the SMS text data, SMS voice data and user movement route data to obtain SMS text feature data, SMS voice feature data and user movement route feature data, and performing graph fusion based on the SMS text feature data, SMS voice feature data and user movement route feature data to obtain SMS text and voice fusion graph feature data;

[0080] Step S3: performing graph convolution calculation based on the SMS text-speech fusion graph feature data to obtain text-speech graph convolution data, performing graph pooling processing based on the text-speech graph convolution data to obtain text-speech graph pooling data, performing full connection calculation based on the text-language graph pooling data to obtain text-speech fully connected layer data;

[0081] Step S4: Perform user location privacy processing based on the user movement route data to obtain the user movement location privacy data, and build and iteratively train a model based on the user movement location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS speech conversion model to perform intelligent SMS speech conversion operations.

[0082] In one embodiment of the present invention, primary text and voice data are obtained from a short message service provider and a voice database, voice conversion is performed based on the text data to obtain primary voice data, and user movement route data is collected.

[0083] Extract features from text, voice and mobile route data to obtain their respective feature data. Construct a text and voice association graph to obtain text route association feature graph data and voice route association feature graph data. Fuse the text and voice association feature graphs to obtain text and voice fusion graph feature data.

[0084] Use TF-IDF or Word2Vec to extract features from SMS text data. Use MFCC to extract voice features. Extract the user's geographic location features, time features, and behavioral features, such as visit frequency and dwell time. Build a text and voice association graph based on the extracted feature data to obtain text route association feature graph data and voice route association feature graph data respectively. Use a graph neural network (GNN) to fuse the text and voice feature graphs to obtain text and voice fusion graph feature data.

[0085] Perform adjacency matrix normalization on the text-speech fusion graph feature data to obtain graph feature normalized data. Calculate the information entropy of the graph feature normalized data, perform graph feature information entropy processing, and obtain graph feature information entropy data. Adjust the convolution layer according to the information entropy data, such as increasing the feature dimension or adjusting the activation function parameters. Perform graph convolution calculation using the information entropy data to obtain text-speech graph convolution data. Perform graph pooling processing on the text-speech graph convolution data to obtain text-speech graph pooling data.

[0086] Perform privacy assessment on user mobile route data, extract user behavior features, perform feature distribution fitting, and establish a user privacy assessment model. Calculate privacy weights based on the privacy assessment model to obtain privacy weight data. Perform privacy labeling of user locations based on the privacy assessment data to obtain privacy labeling data. Perform privacy masking on the privacy labeling data to obtain privacy masking data. Perform privacy feature extraction on the privacy masking data to obtain user mobile location privacy data. Combine the user mobile location privacy data with the text-to-speech fully connected layer data to build an intelligent SMS voice conversion model.

[0087] Construct an intelligent SMS voice conversion model, which is a deep neural network model. Label the user's mobile location privacy data and the text-to-voice fully connected layer data to obtain a primary intelligent SMS voice conversion model. Obtain historical SMS text data, historical SMS voice data, and historical user mobile route data; divide data according to historical SMS text data, historical SMS voice data, and historical user mobile route data to obtain model training data and model test data; iteratively train the model training data and the primary intelligent SMS voice conversion model through a cross loss function (graph convolution, graph pooling, and fully connected calculation in the present invention) to obtain a secondary intelligent SMS voice conversion model; optimize according to the model test data and the secondary intelligent SMS voice conversion model to obtain an intelligent SMS voice conversion model.

[0088] Extract geographic location, time features, and user behavior features (visit frequency, stay time, etc.) from user mobile route data. Perform statistical analysis on the extracted user behavior features and fit the feature distribution model (such as Gaussian distribution). importnumpy as np; from scipy.stats import norm; # Assume route_df is a DataFrame containing user mobile route data location_features = route_df[['latitude','longitude']].values; time_features = route_df['timestamp'].values; behavior_features= route_df[['visit_frequency', 'stay_duration']].values; # Fit feature distribution (taking visit frequency as an example) visit_frequency= behavior_features[:, 0]; mu, std = norm.fit(visit_frequency).

[0089] Calculate the privacy weight of each feature. def calculate_privacy_weight(value, mu, std):return norm.pdf(value, mu, std)# Calculate the privacy weights of visit frequency and stay timeprivacy_weights_vf= calculate_privacy_weight(visit_frequency, mu_vf, std_vf);privacy_weights_sd = calculate_privacy_weight(stay_duration, mu_sd, std_sd);# Comprehensive privacy weightprivacy_weights= 0.5 * privacy_weights_vf + 0.5 * privacy_weights_sd#Add privacy weights to DataFrameroute_df['privacy_weight'] = privacy_weights.

[0090] The model is iteratively trained to achieve accurate SMS voice conversion operations. In the present invention, by obtaining multiple data sources, including text, voice and user mobile routes, the model is ensured to have rich data input, thereby improving the generalization ability and accuracy of the model. By obtaining data related to user mobile behavior, the user's context can be better understood, thereby improving the accuracy and relevance of SMS voice conversion. Through feature extraction, multi-dimensional features of text, voice and user mobile routes are obtained to ensure the diversity and richness of input data. Using graph fusion technology, the features of different data sources are fused to construct SMS text voice fusion graph feature data, thereby capturing the complex relationship between data and improving the representation ability of the model. Through graph fusion technology, the user's mobile route information can be better utilized to enhance the contextual understanding ability of voice conversion. Through graph convolution calculation, the local and global features of graph fusion data can be better extracted to improve the ability of feature expression. Graph pooling processing can effectively reduce feature dimensions, retain important information, reduce computational complexity, and avoid overfitting. Through full connection calculation, the extracted features are further fused and abstracted to obtain more accurate text voice full connection layer data, providing high-quality input for subsequent model construction. Through privacy processing, user mobile data is protected to ensure that user privacy is not leaked, thereby increasing the security of the model and user trust. Based on the user's mobile route and privacy data, more customized and personalized voice conversion services can be provided to improve user experience. Through iterative training of the model, the performance of the intelligent SMS voice conversion model can be continuously optimized to improve the accuracy and efficiency of the conversion.

[0091] Preferably, step S1 specifically includes:

[0092] Step S11: collecting primary text data from a short message service provider to obtain primary short message text data;

[0093] Step S12: collecting primary voice data from a public voice database to obtain primary SMS voice data; or, performing voice conversion based on the primary SMS text data to obtain primary SMS voice data;

[0094] Step S13: collecting user movement route data according to the primary SMS text data to obtain user movement route data;

[0095] Step S14: Associating the primary SMS text data and the primary SMS voice data according to the user movement route data to obtain SMS voice associated data and SMS text associated data;

[0096] Step S15: performing deduplication processing on the SMS voice associated data and the SMS text associated data to obtain SMS voice data and SMS text data.

[0097] In one embodiment of the present invention, an API or crawler tool provided by a short message service provider is used to obtain short message data from the short message service provider. It can be considered to use Python's requests library to make API requests, or to use BeautifulSoup or Scrapy to crawl web pages.

[0098] Get primary voice data from public voice databases using corresponding APIs or crawler tools. Perform voice conversion on primary SMS text data using speech synthesis technology such as Google Text-to-Speech API, Microsoft Text-to-Speech API, etc.

[0099] The user's real-time location information is obtained through the user's mobile phone positioning function and saved as the user's movement route data. According to the user's movement route data, the primary SMS text data and the primary SMS voice data are associated with the user's movement route to form SMS voice associated data.

[0100] De-duplicate SMS voice-related data and SMS text-related data to ensure data uniqueness. De-duplicate data based on unique identifiers or use content-based de-duplication methods.

[0101] The present invention integrates information from different data sources, including SMS text, voice and user movement route data, providing a comprehensive data foundation for subsequent processing. By generating related data, the user's behavior and needs in specific situations can be better understood, and the utilization value of the data can be improved. Deduplication processing can ensure the uniqueness of the data and avoid interference of duplicate information on model training and result analysis. From primary data to related data and then to deduplication processing, the entire step ensures the accuracy and reliability of the data and improves the effect of subsequent analysis and application. Obtaining user movement route data in an authorized manner ensures the security of user privacy information and enhances the user's trust in the service.

[0102] Preferably, step S2 specifically includes:

[0103] Step S21: extracting features based on the SMS text data, SMS voice data and user movement route data to obtain SMS text feature data, SMS voice feature data and user movement route feature data;

[0104] Step S22: constructing a correlation graph according to the SMS text feature data and the user movement route feature data to obtain SMS text route correlation feature graph data;

[0105] Step S23: constructing a correlation graph based on the SMS voice feature data and the user movement route feature data to obtain SMS voice route correlation feature graph data;

[0106] Step S24: performing graph fusion according to the SMS text route associated feature graph data and the SMS voice route associated feature graph data to obtain SMS text and voice fusion graph feature data.

[0107] In one embodiment of the present invention, text feature extraction for SMS text data may use natural language processing (NLP) technology, such as Bag of Words, TF-IDF (Term Frequency-Inverse Document Frequency), etc. Voice feature extraction for SMS voice data may use voice processing technology, such as MFCC (MelFrequency Cepstral Coefficients), etc. Feature extraction for user movement route data may extract geographic location features, time features, user behavior features, etc.

[0108] The SMS text route association feature graph is constructed based on the SMS text feature data and the user mobile route feature data, with the text features and geographic location features as nodes and similarity as edge weights. The SMS voice route association feature graph is constructed based on the SMS voice feature data and the user mobile route feature data, and the construction method is similar to the text route association feature graph.

[0109] The SMS text route associated feature graph data and the SMS voice route associated feature graph data are fused, and the two graphs are weighted fused or spliced ​​using methods such as the Graph Neural Network.

[0110] In the present invention, SMS text, voice and user mobile route data are fused to obtain more comprehensive and rich feature data, which helps to improve the expressiveness and prediction accuracy of the model. By constructing an associated feature graph, the associated information between different data sources can be better captured, and the model's ability to understand and analyze data is improved. Through graph fusion technology, the features of different data sources are fused, which can better capture the complex relationship between data and improve the representation and generalization capabilities of the model. Fusion of features from different data sources can improve the diversity and expressiveness of features, which helps to more accurately describe the characteristics of data.

[0111] Preferably, step S3 specifically includes:

[0112] Step S31: performing adjacency matrix normalization processing on the SMS text and voice fusion graph feature data to obtain graph feature normalized data;

[0113] Step S32: performing information entropy calculation on the graph feature normalized data to obtain graph feature information entropy data;

[0114] Step S33: when it is determined that the graph feature information entropy data is greater than or equal to the preset graph feature information entropy threshold data, the graph feature information entropy data and the graph feature normalization data are feature spliced ​​to obtain graph feature adjustment data;

[0115] Step S34: when it is determined that the graph feature information entropy data is less than the preset graph feature information entropy threshold data, the graph feature information entropy data is determined as the graph feature adjustment data;

[0116] Step S35: mapping activation function parameters according to the graph feature information entropy data and the preset graph convolution activation function data to obtain the graph convolution activation function data;

[0117] Step S36: performing graph convolution calculation through a graph convolution activation function according to the graph feature adjustment data to obtain text-speech graph convolution data;

[0118] Step S37: performing graph pooling processing according to the graph feature information entropy data and the text-speech graph convolution data to obtain text-speech graph pooling data;

[0119] Step S38: Perform full connection calculation based on the text language graph pooling data to obtain text speech full connection layer data.

[0120] In one embodiment provided by the present invention, adjacency matrix normalization processing is performed on graph feature normalized data, using common graph neural network methods, such as symmetric normalization or left normalization in GCN (Graph Convolutional Network).

[0121] The information entropy of the normalized adjacency matrix is ​​calculated. Using the definition formula of entropy, each element in the adjacency matrix is ​​regarded as the value of a random variable and its entropy value is calculated.

[0122] According to the information entropy calculation results and the preset information entropy threshold, determine whether the graph features need to be adjusted. If the information entropy is greater than or equal to the threshold, feature splicing is performed; if the information entropy is less than the threshold, the information entropy data is directly used as feature adjustment data.

[0123] According to the graph feature information entropy data and the preset graph convolution activation function data, the activation function parameters are mapped. For example, the output of the convolution layer is , whose size is . Calculate the entropy matrix S, whose university is . Adjust the parameters of the activation function, such as the slope parameter of ReLU , ,in is a mapping function (such as linear mapping, nonlinear mapping (such as Sigmoid function, Tanh function, etc.)), which converts the entropy value Mapping to activation function parameters . Use the adjusted activation function: Adjust the data according to the obtained graph features, perform graph convolution calculations through graph convolution activation functions, such as GCN, GAT (Graph Attention Network), etc., or perform graph convolution operations through the product of the adjacency matrix and the node feature matrix, and use the topological structure of the graph to extract features. Node feature matrix X: represents the features of each node in the graph. Assuming there are N nodes in the graph and each node has F features, then . Adjacency matrix A: represents the connection relationship between nodes in the graph. , if there is an edge between node i and node j, then is 1, otherwise it is 0. Degree matrix D: a diagonal matrix representing the degree of each node. Represents the degree of node i. The specific process of graph convolution: multiply the node feature matrix by the weight matrix. The weight matrix is ​​a trainable parameter in the graph convolution layer, which is used to map node features to a new feature space. Multiply the normalized adjacency matrix by the feature matrix, and use the topological structure of the graph to aggregate features. Pass the result through an activation function (such as ReLU) to introduce nonlinear capabilities.

[0124] According to the graph feature information entropy data and the text speech graph convolution data, graph pooling processing is performed. The specific method can be simple average pooling or maximum pooling, etc. Select the pooling strategy (such as average pooling, maximum pooling) according to the entropy value. defchoose_pooling_strategy(entropy_value):if entropy_value<0.5:return 'avg' # Average poolingelif entropy_value<1.0:return 'max' # Maximum poolingelse:return 'adaptive'#Adaptive pooling. Adaptive average pooling divides the input feature map into multiple regions and performs average pooling on each region. Each element of the output feature map is the average value of all elements in the corresponding region. Specifically, weighted calculation is performed according to the graph feature information entropy data to obtain the region quantity data; according to the region quantity data and the text speech graph convolution data, graph pooling processing is performed to obtain the text speech graph pooling data.

[0125] Perform full connection calculation based on the text-speech graph pooling data to obtain the final text-speech fully connected layer data.

[0126] In the present invention, through information entropy calculation and feature splicing, the graph feature data can be dynamically adjusted to improve the feature expression ability, which is helpful to improve the performance and generalization ability of the model. According to the information entropy data, the activation function parameters are mapped, and the activation function can be dynamically adjusted according to the characteristics of the data to improve the adaptability and effect of the model. Through graph convolution calculation, the graph structure information can be better utilized to extract the relationship between the data, and the model's ability to understand the data is enhanced. Graph pooling processing can reduce the data dimension, retain important information, reduce the calculation complexity, and avoid overfitting, which is helpful to improve the generalization ability of the model. Through full connection calculation, the features obtained by graph convolution calculation can be further fused and abstracted to obtain a higher level of feature representation, which is helpful to improve the performance and effect of the model. The present invention introduces an information entropy threshold mechanism to perform adaptive feature adjustment according to the graph feature information entropy data. For data with information entropy higher than the threshold, feature splicing is performed to ensure the retention of important information; for data with information entropy lower than the threshold, simple feature adjustment is performed to reduce unnecessary calculation overhead, so that the model is more flexible and efficient when processing different data.

[0127] Preferably, step S4 is specifically:

[0128] Step S41: performing privacy evaluation according to the user's movement route data to obtain privacy evaluation data;

[0129] Step S42: performing privacy marking on the user's movement route data according to the privacy assessment data to obtain privacy marked data;

[0130] Step S43: performing privacy masking on the high-privacy location data in the privacy-labeled data to obtain privacy masked data;

[0131] Step S44: extracting privacy features from the privacy mask data to obtain privacy data of the user's moving location;

[0132] Step S45: construct a model and iteratively train it based on the user's mobile location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS-to-speech conversion model to perform an intelligent SMS-to-speech conversion operation.

[0133] The present invention provides an embodiment in which, according to the user's mobile route data, a machine learning method or a statistical method is used to analyze the user's mobile behavior, so as to evaluate the user's privacy, including considering factors such as geographic location, time, and the frequency and residence time of the user's behavior. Collect the user's mobile route data, and the data should include information such as timestamp, geographic location (longitude and latitude), access frequency and residence time. Extract features from the user's mobile route data, including geographic location features, time features and user behavior features (access frequency, residence time, etc.). Perform statistical analysis and distribution fitting on the user behavior features, using common statistical distributions such as Gaussian distribution. According to the fitted feature distribution model, calculate the privacy weight of each feature, which reflects the degree of influence of the feature on the user's privacy. Calculate the privacy score of the user at different locations, and obtain the total privacy score by combining the privacy weights of each feature. Use machine learning methods (such as random forest, logistic regression, etc.) to build a privacy evaluation model, and use training data to train the model. Evaluate and optimize the model, use evaluation indicators such as mean square error (MSE) to evaluate, and adjust the model parameters to improve performance.

[0134] According to the privacy evaluation data, the user's mobile route data is labeled as private. According to the evaluation results, a threshold is set to mark high-privacy locations as private locations.

[0135] Perform privacy masking on the highly private location data in the private labeling data, such as data desensitization or fuzzy processing of geographic location.

[0136] The privacy mask data is subjected to feature extraction to obtain the privacy data of the user's mobile location, including features extracted from aspects such as geographic location and time.

[0137] Based on the user's mobile location privacy data and the text-to-speech fully connected layer data, an intelligent SMS speech conversion model is constructed, such as a supervised learning model, such as a neural network model, which learns based on the input features to predict the text-to-speech conversion results.

[0138] The present invention can effectively protect the user's privacy information and avoid the leakage of privacy information through privacy assessment, labeling and mask generation. Feature extraction and mask processing of private information can enhance the security of data and reduce the risk of sensitive information leakage. Using private data to guide model training can improve the model's sensitivity to user privacy and enhance the model's effect in privacy protection. Through private data analysis, users' behaviors and preferences can be better understood, and customized services and suggestions can be provided to users.

[0139] Preferably, the SMS voice data includes first SMS voice data and second SMS voice data, and the SMS text data includes first SMS text data and second SMS text data, and the deduplication process is specifically as follows:

[0140] Performing high correlation deduplication processing on the SMS voice-related data and the SMS text-related data to obtain first SMS voice data and first SMS text data respectively;

[0141] Calculate the information entropy of the SMS voice-related data and the SMS text-related data to obtain SMS voice-related information entropy data and SMS text-related information entropy data respectively;

[0142] Information entropy deduplication processing is performed on the SMS voice-related data and the SMS text-related data according to the SMS voice-related information entropy data and the SMS text-related information entropy data to obtain second SMS voice data and second SMS text data.

[0143] The present invention provides an embodiment in which a data mining method, such as an Apriori algorithm, can be used to perform association rule mining on SMS voice-related data and SMS text-related data to find highly correlated data pairs, and then perform deduplication processing according to the association rules, retaining a copy of the data as the first SMS voice data and the first SMS text data.

[0144] The information entropy is calculated for SMS voice-related data and SMS text-related data respectively. The information entropy is realized by the calculation formula of Shannon entropy, that is, the probability distribution of the elements in each data set is calculated, and the information entropy is obtained according to the calculation results.

[0145] According to the calculated SMS voice-related information entropy data and SMS text-related information entropy data, information entropy deduplication processing is performed on the SMS voice-related data and the SMS text-related data. An information entropy threshold is set. When the information entropy of the data is higher than the threshold, the data is considered to be of high importance and is retained as the second SMS voice data and the second SMS text data.

[0146] In the present invention, duplicate data can be removed through deduplication processing, the quality and accuracy of data can be improved, and the complexity of data processing and analysis can be reduced. Deduplication processing can avoid waste of resources, ensure the effective use of resources, and improve the value and utilization efficiency of data. Deduplication processing can reduce errors caused by data duplication, improve the accuracy of data processing and analysis, and enhance the reliability and stability of the model. Deduplication processing can reduce the processing and calculation of duplicate data, save computing resources, and improve computing efficiency and performance. Deduplication processing can be performed based on information entropy data to more intelligently remove redundant data. For data with higher information entropy, it is retained as the second SMS voice data and the second SMS text data to ensure that important information is retained. Compared with traditional simple deduplication methods, information entropy deduplication processing can more effectively remove useless or duplicate information, retain high-value data, and improve the performance and accuracy of the model.

[0147] Preferably, the SMS text and speech fusion graph feature data includes first SMS text and speech fusion graph feature data and second SMS text and speech fusion graph feature data, and the graph fusion is specifically:

[0148] According to the SMS text route associated feature graph data and the SMS voice route associated feature graph data, feature matrix concatenation and adjacency matrix addition are performed to obtain first SMS text and voice fusion graph feature data;

[0149] The self-attention weight is calculated based on the SMS text route associated feature map data and the SMS voice route associated feature map data to obtain the text route feature self-attention weight data and the voice route feature self-attention weight data respectively;

[0150] The text route feature self-attention weight data and the voice route feature self-attention weight data are used to perform feature graph weighted calculation on the SMS text route associated feature graph data and the SMS voice route associated feature graph data and add the adjacency matrix to obtain the second SMS text and voice fusion graph feature data.

[0151] In one embodiment of the present invention, the feature matrices of the SMS text route associated feature graph data and the SMS voice route associated feature graph data are concatenated to obtain a new feature matrix. At the same time, the adjacency matrices of the two associated feature graphs are added to obtain the first SMS text and voice fusion graph feature data.

[0152] The self-attention weights are calculated for the text route features and the voice route features respectively. The self-attention mechanism in the graph neural network (such as GAT, GraphSAGE, etc.) can be used to calculate the weight of each node.

[0153] The calculated text route feature self-attention weight and voice route feature self-attention weight are used to perform weighted calculations on the SMS text route associated feature graph data and the SMS voice route associated feature graph data, respectively. Then, the adjacency matrices of the two weighted feature graphs are added together to obtain the second SMS text and voice fusion graph feature data.

[0154] In the present invention, through graph fusion processing, the feature information of SMS text and voice can be effectively fused, the dimension and diversity of data can be improved, and the model's ability to express data can be enhanced. Through self-attention weight calculation and feature graph weighted calculation, features can be weighted according to their importance, the quality and effectiveness of the fused features can be improved, and the model's attention and discrimination to data can be enhanced. The fused feature data can be used for model training, improving the model's understanding and processing capabilities for SMS voice conversion tasks, and enhancing the model's generalization and adaptability.

[0155] Preferably, the graph feature information entropy data includes first graph feature information entropy data and second graph feature information entropy data, and the information entropy calculation is specifically as follows:

[0156] Extract the changed nodes from the graph feature normalized data to obtain the changed node data;

[0157] Calculate information entropy based on the changed node data to obtain feature information entropy data of the first graph;

[0158] Perform Gaussian clustering calculation on the graph feature normalized data to obtain graph feature clustering data;

[0159] Perform feature extraction on the graph feature clustering data to obtain graph feature clustering feature data;

[0160] The information entropy of the graph feature clustering feature data is calculated to obtain second graph feature information entropy data.

[0161] In one embodiment of the present invention, node change detection is performed on graph feature normalized data. This can be achieved by comparing node features in adjacent time periods. For example, the difference or change rate of node features in adjacent time periods is calculated, and then which nodes have changed are determined based on a set threshold to obtain changed node data.

[0162] The information entropy of the changed node data is calculated. Information entropy can measure the uncertainty or diversity of node features. For each changed node, the information entropy of its features can be calculated as the first graph feature information entropy data.

[0163] Gaussian clustering is performed on the normalized data of graph features. Gaussian Mixture Model (GMM) is one of the commonly used clustering methods. It can perform soft clustering on data and obtain the probability that each node belongs to each cluster.

[0164] Perform feature extraction on graph feature clustering data. The clustering results, such as the probability of each node belonging to each cluster, can be used as the feature representation of the node.

[0165] The information entropy of the graph feature clustering feature data is calculated. The information entropy of each node feature is calculated as the second graph feature information entropy data.

[0166] In the present invention, by extracting the changed nodes and calculating the information entropy, the change and distribution of the nodes in the graph can be analyzed, which is helpful to understand the evolution and development trend of the graph. By Gaussian clustering calculation and feature extraction, the nodes in the graph can be clustered and grouped, which is convenient for analyzing and understanding the structure and features of the graph. Feature extraction can abstract and refine the features of the nodes in the graph to obtain more representative and distinguishing features, which helps to improve the expression and generalization capabilities of the model. Information entropy calculation can measure the complexity and diversity of the graph features, help to understand the features and structure of the graph, and provide an important reference for subsequent analysis and application. By extracting the changed nodes from the normalized graph feature data, the present invention can identify the nodes with significant changes in the graph, which often contain key information. The extraction of changed nodes can use methods such as difference analysis and mutation detection to improve the precision and accuracy of feature analysis. By calculating the information entropy of the changed node data, the first graph feature information entropy data is obtained, which can measure the uncertainty and information content of the changed nodes. Nodes with high information entropy usually contain more useful information. In this way, important feature data can be effectively identified and retained, and the accuracy of the model can be improved. The information entropy of the graph feature clustering feature data is calculated to obtain the second graph feature information entropy data, which can measure the uncertainty and diversity of the data at the clustering level. Multi-level information entropy analysis enables the model to more comprehensively capture and utilize useful information in the data, improving the quality of feature representation and the prediction performance of the model.

[0167] Preferably, the privacy assessment is specifically:

[0168] Extracting user behavior features based on user movement route data to obtain user movement route behavior feature data, wherein the user movement route behavior feature data includes geographic location feature data, time feature data, and user behavior feature data, and the user behavior feature data includes visit location frequency data and stay time data;

[0169] Perform feature distribution fitting based on user movement route behavior feature data to obtain a user privacy evaluation model;

[0170] Calculate the privacy weight according to the user privacy assessment model to obtain the privacy weight data;

[0171] Risk calculation is performed based on the privacy weight data and the user privacy assessment data to obtain the privacy assessment data.

[0172] The present invention provides an embodiment in which user behavior features are extracted from user movement route data, including geographic location features, time features, and user behavior features. Geographic location features may include places frequently visited by users, time features may include time periods of user activities, and user behavior features may include the time and frequency of users staying at different places.

[0173] Use fitting models (such as Gaussian distribution, exponential distribution, etc.) to fit the distribution of user behavior characteristics, establish a probability model for user behavior, fit the probability distribution for each feature data, select the distribution type, and use methods such as maximum likelihood estimation (MLE) to estimate the distribution parameters. Fit Gaussian distribution to behavioral characteristics such as visit frequency and dwell time. Fit exponential distribution to behavioral characteristics such as visit frequency and dwell time. Establish a probability model for user behavior based on the fitted distribution parameters. Probability models can be used to evaluate the privacy of user behavior. Combine the probability densities of Gaussian distribution and exponential distribution to evaluate the privacy of user behavior. Weighted average these probability densities according to specific needs.

[0174] Based on the distribution model of user behavior characteristics, establish a user privacy assessment model, such as a machine learning model, such as a classifier or regression model based on feature data, to predict the user's privacy level. Generate user privacy labels based on the distribution of visit frequency and residence time. These labels are binary classification (high privacy and low privacy) or multi-classification (low, medium, high privacy). Combine the extracted features and the generated privacy labels into a feature dataset for training the machine learning model. Divide the feature dataset into a training set and a test set. Select a machine learning algorithm (such as random forest, logistic regression, etc.) to train the privacy assessment model. Use the random forest model, from sklearn.ensemble importRandomForestClassifier# Build a random forest model model=RandomForestClassifier(n_estimators=100,random_state=42); model.fit(X_train, y_train). Each node selects a feature and its threshold to divide the data to maximize the information gain or the reduction of Gini impurity. Node selection is performed by information gain. Recursive tree construction is to continuously split nodes until the stopping condition (such as maximum depth, minimum number of samples, etc.) is met. Post-pruning reduces overfitting by removing unimportant branches. Pruning is based on cross-validation or other statistical methods. Build multiple decision trees and improve model performance through random sampling and ensemble methods (such as averaging and voting). Train the random forest model with the test set and make predictions.

[0175] Use the logistic regression model, from sklearn.linear_model import LogisticRegression#Build a logistic regression model model = LogisticRegression(random_state=42); model.fit(X_train, y_train). Use the test data to evaluate the model performance and calculate indicators such as accuracy, precision, and recall.

[0176] Based on the user's behavioral characteristics and privacy assessment model, the weight of each feature's impact on the user's privacy is calculated and determined through the model's parameters or feature importance.

[0177] Privacy weight calculation: Calculate the privacy weight of each feature based on the distribution fitting results of the feature. The weight can be calculated based on statistics such as the entropy and variance of the distribution.

[0178] Combine privacy weights and user behavior characteristics to calculate the user's privacy risk. For example, a weighted sum or a risk assessment model can be used to determine the user's privacy level or risk score. Risk calculation formula: Based on probability distribution and privacy weights, a privacy risk calculation formula is constructed. The formula can be a weighted sum or other comprehensive methods (such as Bayesian inference method). The Bayesian inference method uses Bayes' theorem to calculate the posterior probability through the prior probability and likelihood function to evaluate privacy risk.

[0179] ;

[0180] It is the probability that a user is at a certain privacy risk level H after given user behavior data D, which means the probability of evaluating the user's privacy risk level after considering the user's behavior characteristics.

[0181] P(D|H) is the probability of data D appearing when assumption H holds. The probability of observing user behavior data D under the assumption that the user is at a certain privacy risk level H indicates the possibility of user behavior characteristics appearing under a certain privacy risk level.

[0182] P(H) is the initial probability that hypothesis H is true. The initial probability that a user is at a certain privacy risk level H (the probability before any behavior data is observed) is set based on historical data or domain expert knowledge.

[0183] P(D) is the total probability of the occurrence of data D. The overall probability of the occurrence of user behavior data D is obtained by integrating or summing all possible privacy risk levels H.

[0184] The present invention can perform personalized assessment of user privacy through user behavior feature extraction and privacy assessment model, and more accurately reflect the user's privacy risk. Through privacy weight calculation, the impact of different behavior features on user privacy can be analyzed, providing a reference for the formulation of privacy protection strategies. Through risk calculation, the user's privacy risk can be predicted, potential privacy leakage risks can be discovered early, and the effect of privacy protection can be improved.

[0185] Preferably, the privacy assessment data includes node privacy assessment data and route privacy assessment data, and the risk calculation is specifically as follows:

[0186] Calculate the local node risk based on the privacy weight data, user privacy assessment data, and user movement route behavior feature data to obtain node privacy assessment data;

[0187] Route node risks are calculated based on privacy weight data, user privacy assessment data, and user mobile route behavior feature data to obtain route privacy assessment data.

[0188] In one embodiment of the present invention, the user's mobile route data is collected from the user's mobile device or other sources, including geographic location, time, behavior, etc. The privacy weight data can be obtained by analyzing a large amount of historical data and training a machine learning model.

[0189] Extract visit frequency, dwell time and other features from user movement route data. Perform weighted calculation on the extracted features according to privacy weight data. Calculate the privacy evaluation data of each node using the formula: , Evaluate data for node privacy, is the characteristic order term, is the feature quantity data, For the feature privacy weight items, For the Features

[0190] The privacy assessment data of all nodes are combined to obtain the privacy assessment data of the entire route. To calculate the privacy risk of the entire route, use the formula: , For route privacy assessment data, is the route node sequence item, is the number of route nodes, For the The privacy evaluation data of each node.

[0191] In the present invention, the privacy assessment data of each node is calculated based on the privacy weight data, the user privacy assessment data and the user's mobile route behavior characteristic data, which helps determine the privacy risk level of each node and the degree of impact of the node on the user's privacy. According to the privacy weight data, the user privacy assessment data and the user's mobile route behavior characteristic data, the privacy assessment data of the entire route is calculated to help determine the privacy risk level of the entire route and the overall impact of the route on the user's privacy. Through the node privacy assessment data and the route privacy assessment data, the privacy risk of the user during the movement is evaluated in multiple dimensions, which helps to better understand the user's privacy status. Through the node privacy assessment data and the route privacy assessment data, it can help to accurately locate the possible locations and routes of user privacy leakage, which helps to formulate targeted privacy protection strategies. Through the privacy assessment data of different nodes and routes, personalized privacy protection solutions can be provided for different users to improve the effect of user privacy protection.

[0192] The beneficial effects of the present invention are as follows: by integrating SMS text data, SMS voice data and user mobile route data, and extracting features using multi-source data, it helps to improve the performance and accuracy of the model. By adopting a graph fusion method, SMS text features, SMS voice features and user mobile route features are integrated to better capture the correlation and complexity between data and improve the expressiveness of features. The introduction of a graph convolutional neural network to calculate the fused features can effectively process unstructured data and improve the ability to understand and process data. By performing privacy processing on user mobile route data, the user's privacy information is protected and the acceptability and compliance of the model in practical applications are improved. By adopting a model iterative training method, the model parameters are continuously optimized, the performance and generalization ability of the model are improved, and the intelligent SMS voice conversion model is made more accurate and reliable.

[0193] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0194] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. An intelligent SMS voice conversion method based on machine learning, characterized in that: The following steps are involved: Step S1: Acquire SMS text data, SMS voice data and user movement route data; Step S2: performing feature extraction based on the SMS text data, SMS voice data and user movement route data to obtain SMS text feature data, SMS voice feature data and user movement route feature data, and performing graph fusion based on the SMS text feature data, SMS voice feature data and user movement route feature data to obtain SMS text and voice fusion graph feature data; Step S3: performing graph convolution calculation based on the SMS text-speech fusion graph feature data to obtain text-speech graph convolution data, performing graph pooling processing based on the text-speech graph convolution data to obtain text-speech graph pooling data, performing full connection calculation based on the text-language graph pooling data to obtain text-speech fully connected layer data; Step S4: Performing user location privacy processing according to the user movement route data to obtain the user movement location privacy data, and constructing and iteratively training a model according to the user movement location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS-to-speech conversion model to perform an intelligent SMS-to-speech conversion operation; Wherein step S2 is specifically: Extract features based on the SMS text data, SMS voice data, and user movement route data to obtain SMS text feature data, SMS voice feature data, and user movement route feature data; Constructing a correlation graph based on the SMS text feature data and the user's mobile route feature data to obtain SMS text route correlation feature graph data; Constructing a correlation graph based on the SMS voice feature data and the user's mobile route feature data to obtain SMS voice route correlation feature graph data; Graph fusion is performed based on the SMS text route associated feature graph data and the SMS voice route associated feature graph data to obtain SMS text and voice fusion graph feature data.

2. The method according to claim 1, characterized in that Step S1 is specifically as follows: Collecting primary text data from a short message service provider to obtain primary short message text data; Collect primary voice data from a public voice database to obtain primary SMS voice data; Alternatively, performing voice conversion according to the primary SMS text data to obtain primary SMS voice data; Collect user movement route data based on primary SMS text data to obtain user movement route data; Associating the primary SMS text data and the primary SMS voice data according to the user movement route data to obtain SMS voice associated data and SMS text associated data; The SMS voice associated data and the SMS text associated data are deduplicated to obtain SMS voice data and SMS text data.

3. The method according to claim 1, characterized in that: Step S3 is specifically as follows: Perform adjacency matrix normalization processing on the SMS text and voice fusion graph feature data to obtain graph feature normalized data; Calculate the information entropy of the graph feature normalized data to obtain the graph feature information entropy data; When it is determined that the graph feature information entropy data is greater than or equal to the preset graph feature information entropy threshold data, the graph feature information entropy data and the graph feature normalization data are feature spliced ​​to obtain graph feature adjustment data; When it is determined that the graph feature information entropy data is less than the preset graph feature information entropy threshold data, the graph feature information entropy data is determined as the graph feature adjustment data; According to the graph feature information entropy data and the preset graph convolution activation function data, the activation function parameters are mapped to obtain the graph convolution activation function data; According to the graph feature adjustment data, the graph convolution calculation is performed through the graph convolution activation function to obtain the text speech graph convolution data; Perform graph pooling processing based on graph feature information entropy data and text-speech graph convolution data to obtain text-speech graph pooling data; Perform full connection calculation based on the text language graph pooling data to obtain the text speech full connection layer data.

4. The method according to claim 1, characterized in that: Step S4 is specifically as follows: Performing privacy assessment based on user movement route data to obtain privacy assessment data; Performing privacy labeling on the user's mobile route data according to the privacy assessment data to obtain privacy labeling data; Performing privacy masking on high-privacy location data in the privacy-labeled data to obtain privacy masked data; Extract privacy features from the privacy mask data to obtain privacy data of the user's moving location; The model is constructed and iteratively trained based on the user's mobile location privacy data and the text-to-speech fully connected layer data to obtain an intelligent SMS speech conversion model for performing intelligent SMS speech conversion operations.

5. The method according to claim 2, characterized in that: The SMS voice data includes first SMS voice data and second SMS voice data, and the SMS text data includes first SMS text data and second SMS text data. The deduplication process is as follows: Performing high correlation deduplication processing on the SMS voice-related data and the SMS text-related data to obtain first SMS voice data and first SMS text data respectively; Calculate the information entropy of the SMS voice-related data and the SMS text-related data to obtain SMS voice-related information entropy data and SMS text-related information entropy data respectively; Information entropy deduplication processing is performed on the SMS voice-related data and the SMS text-related data according to the SMS voice-related information entropy data and the SMS text-related information entropy data to obtain second SMS voice data and second SMS text data.

6. The method according to claim 1, characterized in that The short message text and speech fusion graph feature data includes the first short message text and speech fusion graph feature data and the second short message text and speech fusion graph feature data. The graph fusion is specifically: According to the SMS text route associated feature graph data and the SMS voice route associated feature graph data, feature matrix concatenation and adjacency matrix addition are performed to obtain first SMS text and voice fusion graph feature data; The self-attention weight is calculated based on the SMS text route associated feature map data and the SMS voice route associated feature map data to obtain the text route feature self-attention weight data and the voice route feature self-attention weight data respectively; The text route feature self-attention weight data and the voice route feature self-attention weight data are used to perform feature graph weighted calculation on the SMS text route associated feature graph data and the SMS voice route associated feature graph data and add the adjacency matrix to obtain the second SMS text and voice fusion graph feature data.

7. The method according to claim 3, characterized in that The graph feature information entropy data includes the first graph feature information entropy data and the second graph feature information entropy data. The information entropy calculation is specifically as follows: Extract the changed nodes from the graph feature normalized data to obtain the changed node data; Calculate information entropy based on the changed node data to obtain feature information entropy data of the first graph; Perform Gaussian clustering calculation on the graph feature normalized data to obtain graph feature clustering data; Perform feature extraction on the graph feature clustering data to obtain graph feature clustering feature data; The information entropy of the graph feature clustering feature data is calculated to obtain second graph feature information entropy data.

8. The method according to claim 4, characterized in that The privacy assessment is as follows: Extracting user behavior features based on user movement route data to obtain user movement route behavior feature data, wherein the user movement route behavior feature data includes geographic location feature data, time feature data, and user behavior feature data, and the user behavior feature data includes visit location frequency data and stay time data; Perform feature distribution fitting based on user movement route behavior feature data to obtain a user privacy evaluation model; Calculate the privacy weight according to the user privacy assessment model to obtain the privacy weight data; Risk calculation is performed based on the privacy weight data and the user privacy assessment data to obtain the privacy assessment data.

9. The method according to claim 8, characterized in that The privacy assessment data includes node privacy assessment data and route privacy assessment data. The risk calculation is as follows: Calculate the local node risk based on the privacy weight data, user privacy assessment data, and user movement route behavior feature data to obtain node privacy assessment data; Route node risks are calculated based on privacy weight data, user privacy assessment data, and user mobile route behavior feature data to obtain route privacy assessment data.

Images