Method and system based on distributed multivariate data analysis

Abstract

According to the distributed multivariate data analysis-based method and system provided by the embodiment of the invention, the to-be-derived field data is subjected to hiding operation, the key change moment in the data is further extracted and accurately captured by dynamically updating nodes, and finer time dimension information is provided for data analysis. And the to-be-derived field data is marked, so that the data complexity is simplified, and the efficiency and accuracy of subsequent analysis are improved. The hidden data is subjected to feature enhancement based on the target derivative template knowledge data, the dynamic update node data and the derivative field mark data, so that the feature dimension of the data is enriched, the business logic and knowledge experience are integrated, and the depth and breadth of data analysis are remarkably improved. Finally, by constructing a training template of a machine learning network model, the model can more accurately learn data features, and the prediction accuracy and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system based on distributed multivariate data analysis. Background Art

[0002] With the rapid development of information technology, all kinds of data are growing at an unprecedented rate, especially multivariate data in distributed environments, such as mobile payment data, personnel database data, scene monitoring data, regulatory platform data, and behavior log platform data. These data come from a wide range of sources and in various formats, and contain rich business information and value. However, directly analyzing these complex and large distributed multivariate data faces many challenges, including but not limited to insufficient data privacy protection, difficulty in capturing dynamic changes, difficulty in extracting data features, and low model training efficiency.

[0003] First, distributed multivariate data often contains a lot of sensitive information, such as personal identity information, transaction amounts, etc. If this information is not properly processed and used directly for analysis, there will be a serious risk of privacy leakage. Therefore, how to effectively use this data for in-depth analysis while protecting data privacy has become an urgent problem to be solved.

[0004] Secondly, some fields in distributed multivariate data (such as transaction time, monitoring time, etc.) have dynamic changing characteristics. These dynamic field data are of great significance for understanding business development trends and predicting future trends. However, traditional data analysis methods often find it difficult to effectively capture the key nodes in these dynamic changes, resulting in delayed or biased analysis results.

[0005] Finally, when building a machine learning model for data analysis, the design of the training template directly affects the performance of the model. Traditional training templates are often designed based on a single type of data and are difficult to adapt to the complexity and diversity of distributed multivariate data. Therefore, how to design a suitable machine learning network model training template based on the characteristics of distributed multivariate data to improve the prediction accuracy and generalization ability of the model is also an important challenge facing the current data analysis field. Summary of the invention

[0006] In order to at least overcome the above-mentioned deficiencies in the prior art, an object of the embodiments of the present application is to provide a method and system based on distributed multivariate data analysis.

[0007] According to one aspect of the present application, a method based on distributed multivariate data analysis is provided, the method comprising:

[0008] Performing a hiding operation on the field data to be derived in the target distributed multivariate data to generate hidden distributed multivariate data, wherein the target distributed multivariate data includes at least two types of data among mobile payment data, personnel database data, scene monitoring data, supervision platform data, and behavior log platform data;

[0009] Dynamically updating node extraction is performed on the dynamic field data in the target distributed multivariate data to generate dynamic updating node data; the field data to be derived includes the dynamic field data to be derived in the dynamic field data;

[0010] Marking the to-be-derived field data in the target distributed multivariate data to generate derived field marked data;

[0011] Based on the target derived template knowledge data, the dynamically updated node data and the derived field tag data, feature enhancement is performed on the hidden distributed multivariate data to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data;

[0012] A training template for a corresponding machine learning network model is constructed based on the derived distributed multivariate data and the target distributed multivariate data.

[0013] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in various optional implementations of the above three aspects.

[0014] In the technical solutions provided in some embodiments of the present application, the embodiments of the present application effectively protect sensitive information in the data, enhance the security of the data, and avoid the risk of privacy leakage by hiding the derived field data. The method further accurately captures the key change moments in the data by dynamically updating node extraction, providing more refined time dimension information for data analysis. The labeling processing of the derived field data simplifies the complexity of the data and improves the efficiency and accuracy of subsequent analysis. Based on the target derived template knowledge data, dynamically updated node data and derived field labeling data, the hidden data is enhanced in features, which not only enriches the feature dimension of the data, but also incorporates business logic and knowledge experience, significantly improving the depth and breadth of data analysis. Finally, by constructing a training template for the machine learning network model, the model can learn data features more accurately, improving the prediction accuracy and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be extracted in combination with these drawings without creative work.

[0016] Figure 1 A schematic diagram of a process for a distributed multivariate data analysis method provided in an embodiment of the present application;

[0017] Figure 2 A schematic block diagram of the architecture of a system based on distributed multivariate data analysis provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following description is intended to enable one of ordinary skill in the art to implement and combine the present application, and the description is provided in the context of a specific application scenario and its requirements. It will be apparent to one of ordinary skill in the art that various changes may be made to the disclosed embodiments, and that the general principles defined in the present application may be applied to other embodiments and application scenarios without departing from the principles and scope of the present application. Therefore, the present application is not limited to the described embodiments, but should be given the broadest scope consistent with the claims.

[0019] Figure 1 This is a flow chart of a method based on distributed multivariate data analysis provided by an embodiment of the present application. The method based on distributed multivariate data analysis is introduced in detail below.

[0020] Step S110, performing a hiding operation on the field data to be derived in the target distributed multivariate data to generate hidden distributed multivariate data, wherein the target distributed multivariate data includes at least two types of data among mobile payment data, personnel database data, scene monitoring data, supervision platform data and behavior log platform data.

[0021] Specifically, the target distributed multivariate data refers to a data set containing multiple sources and types, which comes from different systems or platforms, including but not limited to mobile payment data, personnel database data, scene monitoring data, regulatory platform data, and behavior log platform data. These data usually have different structures and formats, but they all serve a certain business goal or analysis requirement.

[0022] The field data to be derived are those fields in the target distributed multivariate data that need to be further processed or transformed to extract more information or features. These fields to be derived may contain sensitive information or require specific processing to realize their potential value. The hiding operation is the process of encrypting, replacing or obfuscating the sensitive or protected field data to be derived to ensure the security or privacy of the data while retaining its possibility for analysis. The hidden distributed multivariate data is the target distributed multivariate data after the hiding operation, in which the field data to be derived has been encrypted, replaced or obfuscated to protect sensitive information.

[0023] For example, in this embodiment, the target distributed multivariate data received by the server includes mobile payment data, personnel database data and scene monitoring data. The mobile payment data includes fields such as transaction amount, transaction time, and payment method; the personnel database data includes fields such as name, age, and ID number; and the scene monitoring data includes fields such as monitoring location, monitoring time, and personnel flow.

[0024] Among these data, the field data to be derived is set to the transaction amount in the mobile payment data (for example, because the amount-related information may need to be specially processed to protect privacy or perform special feature engineering) and the age field in the personnel database data.

[0025] The server performs a hiding operation. For example, for the transaction amount field, an encryption algorithm (such as the AES encryption algorithm) is used to encrypt each transaction amount value. For the age field, a replacement strategy can be used to replace the real age with a random number generated according to certain rules (such as randomly generating a number within a range of 5 years above or below the real age). After such an operation, in the generated hidden distributed multivariate data, the values ​​of the transaction amount and age fields are hidden and exist in an encrypted or disguised form, while other fields remain unchanged.

[0026] Step S120, extracting dynamic update nodes from the dynamic field data in the target distributed multivariate data to generate dynamic update node data. The field data to be derived includes the dynamic field data to be derived in the dynamic field data.

[0027] In detail, the dynamic field data are those data fields in the target distributed multivariate data that change over time, such as transaction time, monitoring time, etc. These data reflect the real-time status or activity of the system. The dynamically updated node data are key time points or time period information extracted from the dynamic field data, which represent the moments when the data changes significantly or has special significance.

[0028] For example, in this embodiment, still taking the above-mentioned target distributed multivariate data as an example, the dynamic field data includes the transaction time in the mobile payment data (because the transaction time will be continuously updated with each transaction) and the personnel flow in the scene monitoring data (the personnel flow at different times changes dynamically).

[0029] The server extracts dynamic update nodes for transaction time in mobile payment data. For example, a day is divided into 24 nodes by hour, and the number of transactions in each hour is counted. When the number of transactions exceeds a certain threshold (such as 100 times), this hour is marked as a dynamic update node. For the flow of people in scene monitoring data, the server counts it in a time period of 15 minutes. If the change in the flow of people in two adjacent 15-minute time periods exceeds 50 people, this time point is marked as a dynamic update node. Through such operations, dynamic update node data is generated, which can reflect the key node information of the data in the dynamic change process.

[0030] Step S130 , marking the to-be-derived field data in the target distributed multivariate data to generate derived field marked data.

[0031] Continuing to use the target distributed multivariate data mentioned above, the server performs labeling processing on the derived field data (i.e., the transaction amount in the mobile payment data and the age in the personnel database data).

[0032] For example, for the transaction amount field, if the amount is less than 100 yuan, it is marked as "small transaction" and represented by the number 1; if the amount is between 100 yuan and 1,000 yuan, it is marked as "medium transaction" and represented by the number 2; if the amount is greater than 1,000 yuan, it is marked as "large transaction" and represented by the number 3. For the age field, if the age is less than 18 years old, it is marked as "minor" and represented by the number 1; if the age is between 18 and 60 years old, it is marked as "adult" and represented by the number 0; if the age is greater than 60 years old, it is marked as "elderly" and represented by the number 1. In this way, derived field labeled data is generated, which can play an auxiliary role in subsequent operations such as feature enhancement.

[0033] Step S140, based on the target derived template knowledge data, the dynamically updated node data and the derived field tag data, feature enhancement is performed on the hidden distributed multivariate data to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data.

[0034] In detail, the target derived template knowledge data is template knowledge data constructed based on business logic and data analysis experience, and contains knowledge patterns about the relationship between fields in the target distributed multivariate data, which is used to guide the feature enhancement process. Feature enhancement refers to the process of processing hidden distributed multivariate data using target derived template knowledge data, dynamically updated node data, and derived field tag data to add new features or improve existing features. This helps to improve the accuracy and effectiveness of data analysis.

[0035] The derived distributed multivariate data is hidden distributed multivariate data that has been subjected to feature enhancement processing, and is combined with the derived feature dimensions expressed by the target derived template knowledge data, and has richer and more valuable information.

[0036] For example, in this embodiment, it is assumed that the target derived template knowledge data is constructed in advance based on business needs and data analysis experience, and includes some relationship patterns between transaction amount, age and other data features, such as "large transactions and adult groups are more inclined to make mobile payments during specific time periods on weekdays".

[0037] The server first embeds the target derived template knowledge data, hidden distributed multivariate data, and dynamically updated node data. For example, for the target derived template knowledge data, the relational pattern is converted into a first embedded feature vector through a word vector model (such as a variant of Word2Vec suitable for processing such non-text but logically related data); for the hidden distributed multivariate data, each field (after the hiding operation) is converted into a second embedded feature vector through feature extraction through a multi-layer neural network (such as a fully connected neural network with 3 hidden layers); for the dynamically updated node data, it is converted into a third embedded feature vector through a time series-based embedding model (such as a variant of Transformer).

[0038] Then, the randomly generated distributed multivariate data is iteratively optimized based on the first embedded feature vector, the second embedded feature vector, the third embedded feature vector and the derived field tag data. For example, the randomly generated distributed multivariate data are initially some numerical vectors randomly generated according to a certain distribution (such as a normal distribution), representing the assumed feature data. The first embedded feature vector is feature transformed using a feature transformation model (assuming that this model contains 5 first feature focusing units) to generate feature transformation data. Each first feature focusing unit may focus on different relationship parts in the target derived template knowledge data, for example, one unit focuses on the relationship between transaction amount and time, and another unit focuses on the relationship between age and payment method, etc.

[0039] The second embedded feature vector, the third embedded feature vector and the derived field tag data are used as iterative transfer conditions (assuming that the business logic description document of the target distributed multivariate data is also included, which describes the source, purpose and business logic relationship between the data), and the iterative optimization model (including 5 second feature focusing units) is used for iterative optimization. For example, three rounds (Y=3) of optimization iterations are performed.

[0040] The first round of optimization iteration: obtain the reference embedding feature vector of the first round of optimization iteration, which is generated by fusing the second embedding feature vector, the third embedding feature vector, the derived field tag data, and the random embedding feature vector corresponding to the randomly generated distributed multivariate data.

[0041] In the first second feature focusing unit, interactive calculations are performed based on the first feature focusing result generated by the first first feature focusing unit (for example, the first focusing index points to the relationship between the transaction amount and the payment method, and the first focusing feature is a quantitative representation of this relationship) and the second feature focusing result of the first second feature focusing unit for the first round of optimization iteration (determined based on the reference embedded feature vector, for example, the index points to the part related to age, and the feature is a quantitative representation related to age). For example, an interactive operation (such as addition operation) is performed on the first focusing index and the second focusing index of the first second feature focusing unit for the first round of optimization iteration to generate a target focusing index of the first second feature focusing unit for the first round of optimization iteration; an interactive operation (such as multiplication operation) is performed on the first focusing feature and the second focusing feature of the first second feature focusing unit for the first round of optimization iteration to generate a target focusing feature of the first second feature focusing unit for the first round of optimization iteration; based on the target interaction index (assuming it is a coefficient index pre-set according to business logic) and the target focusing index, an interaction factor is determined (such as by looking up a coefficient table); based on the interaction factor, an interactive calculation (such as multiplication) is performed on the corresponding target focusing feature to generate an interactive embedded feature vector generated by the first second feature focusing unit for the first round of optimization iteration.

[0042] The interactive embedding feature vector is optimized iteratively (eg, by adjusting the values ​​of certain elements in the vector to make it more consistent with the expected optimization target) to generate the first temporary optimized iterative embedding feature vector for the first round of optimization iteration.

[0043] The second round of optimization iteration: Get the reference embedding feature vector of the second round of optimization iteration, which is the optimized embedding feature vector of the first round of optimization iteration. According to the similar operation as the first round, perform interactive calculation in the second second feature focusing unit, generate interactive embedding feature vector, and perform optimization iteration to generate the second temporary optimized iterative embedding feature vector.

[0044] Optimization iteration of the third round: Get the reference embedding feature vector of the optimization iteration of the third round, which is the optimized embedding feature vector of the optimization iteration of the second round. Perform similar operations in the fifth second feature focusing unit, and finally use the fifth temporary optimized embedding feature vector of the optimization iteration of the third round as the derived embedding feature vector.

[0045] Finally, the derived embedded feature vector is embedded and restored, for example, by an inverse transformation (the inverse operation corresponding to the previous embedding representation operation), to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data. This derived distributed multivariate data not only contains the hidden information of the original data, but also incorporates the relational patterns and feature dimensions in the target derived template knowledge data.

[0046] Step S150, constructing a training template of a corresponding machine learning network model based on the derived distributed multivariate data and the target distributed multivariate data.

[0047] In this embodiment, the server constructs a training template of the corresponding machine learning network model based on the derived distributed multivariate data and the target distributed multivariate data generated previously.

[0048] For example, a neural network model is used as a machine learning network model, and some features in the derived distributed multivariate data are input as part of the input layer, and some related features in the target distributed multivariate data are input as another part. Several hidden layers are set in the middle (such as 3 hidden layers, each hidden layer contains 100 neurons), and the output layer is set to a binary output (0 means no risk, 1 means risk) according to the business goal (such as predicting whether a certain transaction behavior is risky).

[0049] When constructing a training template, define a loss function (such as a cross entropy loss function) to measure the difference between the model's predicted output and the true label (assuming the actual transaction risk situation obtained through manual annotation or historical data verification). At the same time, set an optimization algorithm (such as the Adam optimization algorithm) to adjust the parameters of the network model (such as the connection weights and bias terms between neurons), so that the model is continuously optimized during the training process to adapt to the feature relationships in the derived distributed multivariate data and the target distributed multivariate data, and improve the accuracy of the prediction.

[0050] Based on the above steps, the embodiment of the present application effectively protects sensitive information in the data, enhances data security, and avoids the risk of privacy leakage by hiding the derived field data. The method further accurately captures the key change moments in the data by dynamically updating node extraction, providing more detailed time dimension information for data analysis. The labeling processing of the derived field data simplifies the complexity of the data and improves the efficiency and accuracy of subsequent analysis. Based on the target derived template knowledge data, dynamically updated node data and derived field labeling data, the hidden data is enhanced in features, which not only enriches the feature dimension of the data, but also incorporates business logic and knowledge experience, significantly improving the depth and breadth of data analysis. Finally, by constructing a training template for the machine learning network model, the model can learn data features more accurately, improving the prediction accuracy and generalization ability of the model.

[0051] In a possible implementation, step S140 includes:

[0052] Step S141, embedding the target derived template knowledge data, the hidden distributed multivariate data and the dynamically updated node data respectively, generating a first embedded feature vector of the target derived template knowledge data, a second embedded feature vector of the hidden distributed multivariate data and a third embedded feature vector of the dynamically updated node data.

[0053] Step S142: performing iterative optimization processing on the randomly generated distributed multivariate data according to the first embedded feature vector, the second embedded feature vector, the third embedded feature vector and the derived field tag data to generate a derived embedded feature vector.

[0054] Step S143, embedding and restoring the derived embedded feature vector to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data.

[0055] In this embodiment, it is assumed that the target derived template knowledge data in the server contains some predefined rules for mobile payment behavior, such as "during peak hours on weekdays (9:10 and 18:19), users aged between 25 and 45 are more likely to use mobile payment for shopping."

[0056] The server quantifies these rules for embedding. For example, the number 15 represents the working day, the peak hours can be represented by encoding the 24 hours of the day and using the corresponding numerical interval (assuming that 910 is encoded as 90100 and 1819 is encoded as 180190), and the age interval 2545 is also encoded with the corresponding numerical value.

[0057] The server then uses a specially designed embedding model (e.g., an embedding layer structure based on a neural network) to convert these quantized rules into a first embedding feature vector. This embedding model may be pre-trained and can convert these business logic-related data into a vector form suitable for subsequent calculations.

[0058] Taking the previously processed hidden distributed multivariate data as an example, the transaction amount of the mobile payment data was encrypted and hidden, and the age in the personnel database data was replaced and hidden.

[0059] The server extracts features from these hidden data. For mobile payment data, in addition to the hidden transaction amount, other fields include payment method (assuming cash payment is coded as 0 and electronic payment is coded as 1), transaction location (coded according to geographic location), etc. For personnel database data, names can be encoded as text, and ID numbers can be encoded after desensitization.

[0060] The multi-layer perceptron (MLP) model is used as a tool for embedding representation to transform these processed hidden distributed multivariate data into a second embedded feature vector. The input layer of the MLP receives the encoded value of each data field, and after nonlinear transformations of multiple hidden layers, the output layer outputs the second embedded feature vector.

[0061] Consider the dynamically updated node data previously extracted from the transaction time in the mobile payment data and the personnel flow in the scene monitoring data.

[0062] For the dynamic update nodes of transaction time, for example, the number of transactions is counted in a time window of half an hour, and when the number of transactions exceeds a certain threshold, it is marked as a dynamic update node. The server encodes these node information, such as marking the first half-hour window as 1, the second half-hour window as 2, etc. For the dynamic update nodes of personnel flow, they are encoded according to different flow threshold ranges.

[0063] The time series embedding technology (similar to the position encoding idea in the Transformer architecture for processing time series data) is used to transform these dynamically updated node data into a third embedded feature vector. This embedding technology can capture the characteristics of the dynamic update nodes in terms of time and value changes.

[0064] Next, we assume that the feature transformation model consists of multiple sub-modules, each of which corresponds to a type of rule or relation in the target derived template knowledge data.

[0065] For example, for the relationship rules between mobile payment time, age and payment probability mentioned earlier, one submodule specifically handles time-related feature conversions, and the other submodule handles age-related feature conversions.

[0066] When the first embedded feature vector is subjected to feature conversion, each submodule operates on the corresponding part in the vector. Taking the time-related submodule as an example, it may convert the time-related part in the first embedded feature vector according to a predefined time conversion function (such as transforming the time code according to a periodic function) to obtain a part of the feature conversion data. Similarly, the age-related submodule converts the age-related part, and finally combines to obtain the complete feature conversion data.

[0067] Here, the randomly generated distributed multivariate data are assumed to be initially random numeric vectors representing features assumed to be associated with the target data. For example, the dimensions of these vectors may be similar to the dimensions of the hidden distributed multivariate data, but the initial values ​​are randomly generated.

[0068] The iterative optimization model includes multiple second feature focusing units, assuming 3 units here.

[0069] In the first round of iterative optimization, first, a reference embedded feature vector of the first round of optimization iteration is generated. The second embedded feature vector (including the features of hidden distributed multivariate data), the third embedded feature vector (dynamically updated node data features), the derived field tag data (such as the previously marked age group and transaction amount level), and the random embedded feature vector corresponding to the randomly generated distributed multivariate data are fused. For example, a simple vector concatenation or weighted summation method can be used for fusion to obtain a reference embedded feature vector.

[0070] In the first second feature focusing unit, interactive calculations are performed based on the first feature focusing result generated by the first first feature focusing unit (assuming that the first feature focusing result includes the focusing index and feature representation of the relationship between age and payment probability in the target derived template knowledge data) and the second feature focusing result of the first second feature focusing unit for the first round of optimization iteration (age-related feature indices and feature values ​​determined based on the reference embedded feature vector).

[0071] For the interactive operation of the focus index, it is assumed that the age-related index in the first feature focus result is added to the age-related index in the second feature focus result to obtain a new target focus index.

[0072] For the interactive operation of the focused features, the age-payment probability relationship feature value in the first feature focused result is multiplied by the age-related feature value in the second feature focused result to obtain the target focused feature.

[0073] Based on a preset target interaction index (eg, a weight index of age in different scenarios determined according to business logic) and a target focus index, an interaction factor is determined (eg, by looking up a predefined interaction factor table).

[0074] Finally, the target focused features are interactively calculated (such as multiplication) based on the interaction factor to generate an interactive embedding feature vector generated by the first second feature focusing unit for the first round of optimization iteration.

[0075] The interactive embedding feature vector is optimized iteratively. For example, by adjusting the values ​​of certain elements in the vector, according to a predefined optimization rule (such as making the elements in the vector closer to the ideal feature values ​​in the target derived template knowledge data), the first temporary optimized iterative embedding feature vector for the first round of optimization iteration is generated.

[0076] During the second round of iterative optimization, a reference embedding feature vector of the second round of optimization iteration is obtained. This vector is the optimized embedding feature vector of the first round of optimization iteration.

[0077] In the second second feature focusing unit, similar operations to the first round are repeated. According to the feature focusing result generated by the second first feature focusing unit (assuming it is related to the payment method) and the second feature focusing result of the second round of optimization iteration by the second second feature focusing unit (features related to the payment method determined according to the reference embedded feature vector), interactive calculation is performed to generate an interactive embedded feature vector, and then optimization iteration is performed to obtain the second temporary optimized iterative embedded feature vector.

[0078] During the third round of iterative optimization, a reference embedding feature vector of the third round of optimization iteration is obtained. This vector is the optimized embedding feature vector of the second round of optimization iteration.

[0079] In the third second feature focusing unit, the operation is performed according to the previous process, and finally the third temporary optimized iterative embedding feature vector for the third round of optimization iteration is used as the derived embedding feature vector.

[0080] Finally, the server performs an embedding restoration operation on the generated derivative embedding feature vector.

[0081] Assuming that a multi-layer perceptron (MLP) was used to transform the original data into an embedded feature vector when embedding, the corresponding inverse operation is now used. For example, if the MLP performs a linear transformation (such as matrix multiplication and adding a bias term) when embedding, then the embedding is restored by inverse matrix multiplication and subtracting the bias term.

[0082] For the quantitative coding operations performed during the embedding representation process, such as the coding of transaction time, age range, etc., reverse decoding is performed according to the coding rules during embedding restoration. For example, the time coding is restored to the actual time interval, and the age coding is restored to the age range.

[0083] Through such an embedding restoration operation, the derived embedded feature vector is converted into derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data. This derived distributed multivariate data not only contains the hidden information in the original hidden distributed multivariate data, but also incorporates the relationship patterns and feature dimensions in the target derived template knowledge data, such as the association features between different ages, payment times, and payment methods.

[0084] In a possible implementation, step S142 includes:

[0085] Step S1421: perform feature conversion on the first embedded feature vector using a feature conversion model to generate feature conversion data.

[0086] Step S1422, using the second embedded feature vector, the third embedded feature vector and the derived field tag data as iterative transfer conditions, using an iterative optimization model, and iteratively optimizing the randomly generated distributed multivariate data according to the feature conversion data to generate the derived embedded feature vector.

[0087] In a possible implementation, the feature conversion model includes X first feature focusing units, the iterative optimization model includes X second feature focusing units, and the feature conversion data includes first feature focusing results loaded by each of the first feature focusing units.

[0088] Step S1422 includes:

[0089] The second embedded feature vector, the third embedded feature vector and the derived field label data are used as iterative transfer conditions, and an iterative optimization model is used to perform Y rounds of optimization iterations on the randomly generated distributed multivariate data according to the feature conversion data, and the optimized embedded feature vector generated by the Yth round of optimization iteration is used as the derived embedded feature vector.

[0090] The specific steps of the optimization iteration of the ath round include:

[0091] Obtain a reference embedding feature vector of the optimization iteration of the ath round, wherein the reference embedding feature vector of the optimization iteration of the first round is generated by fusing the second embedding feature vector, the third embedding feature vector, the derived field tag data, and the random embedding feature vector corresponding to the randomly generated distributed multivariate data. When 1<a≤Y, the reference embedding feature vector of the optimization iteration of the ath round is the optimized embedding feature vector of the optimization iteration of the a-1th round.

[0092] In the bth second feature focusing unit, interactive calculation is performed based on the first feature focusing result generated by the bth first feature focusing unit and the second feature focusing result of the bth second feature focusing unit for the ath round of optimization iteration, to generate an interactive embedded feature vector generated by the bth second feature focusing unit for the ath round of optimization iteration. 1≤b≤X, b is a positive integer. The second feature focusing result of the first second feature focusing unit for the ath round of optimization iteration is determined based on the reference embedded feature vector of the ath round of optimization iteration.

[0093] The interactive embedding feature vector generated by the b-th second feature focusing unit is optimized and iterated to generate a b-th temporary optimized iterative embedding feature vector for the a-th round of optimization iteration.

[0094] If b is less than X, determine the second feature focusing result of the b+1th second feature focusing unit for the ath round of optimization iteration based on the bth temporary optimization iteration embedded feature vector for the ath round of optimization iteration, count and accumulate b, and return to execute the step of interactively calculating in the bth second feature focusing unit according to the first feature focusing result generated by the bth first feature focusing unit and the second feature focusing result of the bth second feature focusing unit for the ath round of optimization iteration, to generate the interactive embedded feature vector generated by the bth second feature focusing unit for the ath round of optimization iteration.

[0095] If b=X, the bth temporary optimized iteration embedding feature vector for the ath round of optimization iteration is used as the optimized embedding feature vector for the ath round of optimization iteration.

[0096] In this embodiment, it is assumed that the feature conversion model in the server includes three first feature focusing units (X=3), and the first embedded feature vector includes the quantitative representation of the target derived template knowledge data mentioned above, such as the quantitative information of the relationship between age and payment probability, payment time and payment amount in the mobile payment scenario.

[0097] The first feature focusing unit 1 focuses on the feature conversion of the relationship between age and payment probability. It has a predefined age payment probability mapping table, which is obtained based on a large amount of historical mobile payment data and user age data. For example, the probability of the 2030 age group in high consumption payment (amount greater than 1,000 yuan) is 0.2, the probability of medium consumption payment (1,001,000 yuan) is 0.5, and the probability of low consumption payment (less than 100 yuan) is 0.3.

[0098] When the first embedded feature vector is processed, it identifies the part of the vector that is related to age and payment probability. Assume that the age part in the first embedded feature vector is coded with numbers to represent the age range (such as 25 years old is coded as 25), and the payment probability part is represented by a decimal. Unit 1 converts these data according to the mapping table, such as readjusting the high, medium, and low consumption payment probabilities corresponding to 25 years old (may be fine-tuned according to business logic, such as increasing the high consumption payment probability by 0.05), and obtains part of the first feature focus result, including the adjusted age payment probability relationship index (pointing to the new mapping relationship) and the new feature value.

[0099] The first feature focusing unit 2 focuses on the relationship between payment time and payment amount. It has a payment time-payment amount relationship model obtained based on time series analysis. For example, in the afternoon of weekdays (13:15), the proportion of small payments (less than 100 yuan) is higher, while in the evening (19:21), the proportion of large payments (greater than 1,000 yuan) is relatively high.

[0100] For the part of the first embedded feature vector related to payment time and payment amount, this unit converts it according to the relational model. If the payment time in the vector is coded as 14 (corresponding to 14 points) and the payment amount is coded as a medium amount (assuming 500 yuan), unit 2 may adjust the expectation of the payment amount according to the model (such as adjusting it to be more inclined to small payments based on the time trend, and the amount is adjusted to 80 yuan), and at the same time update the relational index to form the first feature focus result of this part.

[0101] The first feature focusing unit 3 is responsible for processing the relationship between payment methods and age. For example, young people (aged 18-30) are more inclined to use electronic payment, while the elderly (over 60 years old) may use cash payment or specific elderly-friendly payment methods.

[0102] Find the part related to payment method (such as electronic payment coded as 1) and age (such as 22 years old coded as 22) in the first embedded feature vector and transform it according to the predefined relationship pattern. If the current relationship indicates that the probability of using electronic payment at the age of 22 is 0.8, unit 3 may adjust the probability to 0.9 according to new market trends (such as young people are more receptive to new electronic payment methods), and update the relationship index at the same time to generate the first feature focus result of this part.

[0103] The operation results of the three units are integrated to obtain feature conversion data, which includes the first feature focusing results loaded by each first feature focusing unit, and covers the converted and adjusted information of different relationships in the target derived template knowledge data.

[0104] Next, assuming that the iterative optimization model also includes three second feature focusing units (X=3), and three rounds of optimization iterations are set (Y=3), the randomly generated distributed multivariate data are initially some random numerical vectors, for example, the dimension of each vector is 10, and the value is randomly generated between 01.

[0105] During the first round (a=1) of optimization iterations, the server merges the second embedded feature vector (containing features of hidden distributed multivariate data, such as hidden transaction amounts, personnel database data, etc.), the third embedded feature vector (dynamically updated node data features, such as dynamic node information of transaction time and personnel flow), derived field tag data (such as previously marked age groups and transaction amount levels), and the random embedded feature vector corresponding to the randomly generated distributed multivariate data.

[0106] For example, for each element in the second embedded feature vector, a weighted sum is performed with the elements at the corresponding positions in the third embedded feature vector, the derived field tag data, and the random embedded feature vector. Assuming that the second embedded feature vector element is v1, the corresponding element of the third embedded feature vector is v2, the derived field tag data element is v3, and the random embedded feature vector element is v4, the fused element is w = 0.3*v1+0.2*v2+0.1*v3+0.4*v4, and the reference embedded feature vector of the first round of optimization iteration is generated in this way.

[0107] During the operation in the first second feature focusing unit (b=1), the second feature focusing result of the first second feature focusing unit for the first round of optimization iteration is determined based on the reference embedded feature vector of the first round of optimization iteration. It is assumed that this unit mainly focuses on the relationship between age and payment method.

[0108] It extracts the parts related to age and payment method from the reference embedded feature vector, such as age coded as 25 years old and payment method coded as electronic payment. According to the predefined age-payment method relationship model (similar to the relationship model in the previous feature conversion unit, but may be different in details), the initial second feature focusing result is obtained, including an index pointing to a specific part of the age-payment method relationship (such as the specific area of ​​25-year-old using electronic payment in the model) and the corresponding feature value (such as a certain quantitative feature of using electronic payment in this case, assuming it is 0.7).

[0109] At the same time, interactive calculations are performed based on the first feature focusing result generated by the first first feature focusing unit (the adjusted result of the relationship between age and payment method obtained in the previous feature conversion stage, assuming that the adjusted eigenvalue is 0.8) and the second feature focusing result of the first round of optimization iteration by the first second feature focusing unit.

[0110] For the interactive operation of indexes, the age-payment method relationship index in the first feature focusing result is subjected to some logical operation, such as a bitwise AND operation (if the index is represented in binary), to obtain the target focusing index.

[0111] For the interactive operation of eigenvalues, the eigenvalue 0.8 in the first feature focusing result is multiplied by the eigenvalue 0.7 in the second feature focusing result to obtain the target focusing eigenvalue 0.56.

[0112] Based on the pre-set target interaction index (assuming the weight index of the age payment method in different scenarios determined according to the business logic, which is 0.6 here) and the target focus index, the interaction factor is determined (such as 0.8 obtained by looking up the predefined interaction factor table).

[0113] Finally, the target focused feature value is interactively calculated (such as multiplication) based on the interaction factor, 0.56*0.8=0.448, to generate the interactive embedding feature vector generated by the first second feature focusing unit for the first round of optimization iteration.

[0114] The interactive embedding feature vector is optimized iteratively. For example, the server adjusts each element in the vector to a direction that is more consistent with the business logic through a preset optimization rule. If the element value is less than 0.5, it is increased by 0.1 (adjusted within a certain range) to obtain the first temporary optimized iterative embedding feature vector for the first round of optimization iteration.

[0115] During the operation in the second second feature focusing unit (b=2), the second feature focusing result of the second second feature focusing unit for the first round of optimization iteration is determined according to the first temporary optimization iteration embedded feature vector for the first round of optimization iteration. For example, this unit focuses on the relationship between payment time and transaction amount, extracts information related to payment time and transaction amount from the temporary optimization iteration embedded feature vector, and obtains the initial second feature focusing result according to the predefined payment time transaction amount relationship model.

[0116] Then, based on the first feature focusing result generated by the second first feature focusing unit (the adjusted result of the relationship between payment time and transaction amount obtained in the previous feature conversion stage) and the second feature focusing result of the second second feature focusing unit for the first round of optimization iteration, interactive calculations are performed. The steps are similar to the operations in the first second feature focusing unit. An interactive embedded feature vector is generated and optimized iteratively to obtain the second temporary optimized iterative embedded feature vector.

[0117] During the operation in the third second feature focusing unit (b=3), in accordance with the previous method, the second feature focusing result of the third second feature focusing unit for the first round of optimization iteration is determined based on the third temporary optimization iterative embedded feature vector, and interactive calculation is performed based on the first feature focusing result generated by the third first feature focusing unit and the second feature focusing result of the third second feature focusing unit for the first round of optimization iteration to generate an interactive embedded feature vector and perform optimization iteration.

[0118] Since b=X=3, the third temporary optimized iteration embedding feature vector for the first round of optimization iteration is used as the optimized embedding feature vector for the first round of optimization iteration.

[0119] In the optimization iteration process of the second round (a=2), the reference embedding feature vector is the optimized embedding feature vector of the optimization iteration of the first round.

[0120] Repeat the operation flow of the first round in each second feature focusing unit, starting from the first second feature focusing unit (b=1), and perform interactive calculation, generate interactive embedded feature vectors, and optimize iterations in sequence, until the third second feature focusing unit (b=3) uses the third temporary optimized iterative embedding feature vector for the second round of optimization iteration as the optimized embedded feature vector for the second round of optimization iteration.

[0121] During the optimization iteration of the third round (a=3), the reference embedding feature vector is the optimized embedding feature vector of the optimization iteration of the second round.

[0122] Repeat the previous operation process again. After the third second feature focusing unit operation of the third round is completed, the third temporary optimized iterative embedding feature vector of the third round of optimization iteration is used as the derived embedding feature vector. This derived embedding feature vector has undergone three rounds of iterative optimization, integrating the target derived template knowledge data, hidden distributed multivariate data, dynamically updated node data, and derived field tag data information, and is continuously adjusted through iterative optimization to meet the requirements of business logic and data characteristics.

[0123] In a possible implementation, the first feature focusing result includes a first focusing index and a first focusing feature. The second feature focusing result includes a second focusing index, a second focusing feature and a target interaction index.

[0124] In the b-th second feature focusing unit, interactive calculation is performed according to the first feature focusing result generated by the b-th first feature focusing unit and the second feature focusing result of the b-th second feature focusing unit for the a-th round of optimization iteration, to generate the interactive embedding feature vector generated by the b-th second feature focusing unit for the a-th round of optimization iteration, including:

[0125] In the bth second feature focusing unit, the first focusing index and the second focusing index of the bth second feature focusing unit for the ath round of optimization iteration are interactively calculated to generate a target focusing index of the bth second feature focusing unit for the ath round of optimization iteration.

[0126] Interactive operations are performed on the first focusing feature and the second focusing feature of the bth second feature focusing unit for the ath round of optimization iteration to generate a target focusing feature of the bth second feature focusing unit for the ath round of optimization iteration.

[0127] Based on the target interaction index of the bth second feature focusing unit for the ath round of optimization iteration and the target focusing index, the interaction factor of the bth second feature focusing unit for the ath round of optimization iteration is determined.

[0128] Based on the interaction factor of the b-th second feature focusing unit for the a-th round of optimization iteration, the corresponding target focusing feature is interactively calculated to generate an interactive embedding feature vector generated by the b-th second feature focusing unit for the a-th round of optimization iteration.

[0129] In this embodiment, it is assumed that the second round (a=2) of optimization iterations is being performed and the first second feature focusing unit (b=1) is currently in progress.

[0130] When the first embedded feature vector was converted using the feature conversion model, the first first feature focusing unit generated a first feature focusing result. Assume that the first focusing index in this first feature focusing result is a code representing the relationship between age and payment method. For example, this code is "0101", which may correspond to the relationship mapping between the age between 2030 and a certain electronic payment method (such as Alipay payment) in a specific business scenario. The first focused feature is a quantitative value related to the relationship, such as 0.8, which indicates the strength of a certain business feature under this age-payment method relationship (which can be understood as the degree of inclination of this age group to use this payment method).

[0131] The second feature focusing result of the first second feature focusing unit for the second round of optimization iteration includes a second focusing index, a second focusing feature, and a target interaction index. Assuming that the second focusing index is determined based on the reference embedded feature vector of the current round, this index is "0110", which may be related to a specific representation of the relationship between age and payment method currently obtained from the fusion of hidden distributed multivariate data, dynamically updated node data, and derived field tag data. The second focusing feature is 0.6, which is another quantitative value related to the relationship extracted from the current round of data. The target interaction index is pre-set according to the business logic, for example, 0.5, which represents a weight adjustment factor when different data features interact in this specific business scenario.

[0132] Specifically, in the first second feature focusing unit, the first focusing index "0101" and the second focusing index "0110" of the first second feature focusing unit for the second round of optimization iteration are interactively operated. A bitwise XOR operation is used here (this is a possible interactive operation method, and the specific operation method depends on the design of the business logic and data representation). The result of the bitwise XOR operation is "0011", which generates the target focusing index of the first second feature focusing unit for the second round of optimization iteration. This target focusing index will be used for the subsequent determination of the interaction factor and the interactive calculation of the target focusing feature.

[0133] The first focusing feature 0.8 and the second focusing feature 0.6 of the first second feature focusing unit for the second round of optimization iteration are interactively operated. Here, a multiplication operation is used (again, the operation method depends on the business logic), and the target focusing feature is obtained as 0.8*0.6=0.48. This target focusing feature represents a new quantitative relationship after the interaction between the first focusing feature and the second focusing feature.

[0134] Based on the target interaction index 0.5 of the first second feature focusing unit for the second round of optimization iteration and the target focusing index "0011", the interaction factor of the first second feature focusing unit for the second round of optimization iteration is determined. Assume that there is a predefined interaction factor lookup table in the server, which determines the interaction factor according to different combinations of the target interaction index and the target focusing index. For the case where the target interaction index is 0.5 and the target focusing index is "0011", the interaction factor is found to be 0.7 from the lookup table. This interaction factor will be used for the final interaction calculation of the target focusing feature.

[0135] Based on the interaction factor 0.7 of the first second feature focusing unit for the second round of optimization iteration, the corresponding target focusing feature 0.48 is interactively calculated. Here, a multiplication operation is used to obtain 0.48*0.7=0.336. This result will be used as part of the interactive embedding feature vector generated by the first second feature focusing unit for the second round of optimization iteration (if the vector is multidimensional, this is only the calculation result of one dimension, and other dimensions may be calculated according to similar logic or combined with other related calculation logic).

[0136] Assume that the optimization iteration continues for the second round (a=2) and reaches the second second feature focusing unit (b=2).

[0137] The first focusing index in the first feature focusing result generated by the second first feature focusing unit is assumed to be "1010", which may correspond to a relationship encoding between the payment time (such as 14:16 pm) and the transaction amount (such as the medium-amount transaction range) in the target derived template knowledge data. The first focusing feature is 0.7, indicating the strength of a certain business feature under this relationship (such as the expected probability of a medium-amount transaction occurring within this time period).

[0138] The second focusing index in the second feature focusing result of the second round of optimization iteration of the second second feature focusing unit is assumed to be "1001", which is a specific representation related to the payment time transaction amount relationship determined based on the reference embedded feature vector of the current round. The second focusing feature is 0.5, and the target interaction index is 0.6 (pre-set according to the business logic).

[0139] Specifically, the first focusing index "1010" and the second focusing index "1001" of the second second feature focusing unit for the second round of optimization iteration are interactively operated, and a bitwise AND operation is used here to obtain the result "1000", which is the target focusing index of the second second feature focusing unit for the second round of optimization iteration.

[0140] The first focusing feature 0.7 and the second focusing feature 0.5 of the second second feature focusing unit for the second round of optimization iteration are multiplied to obtain a target focusing feature of 0.7*0.5=0.35.

[0141] According to the target interaction index 0.6 and the target focus index "1000", the interaction factor is found to be 0.8 from the predefined interaction factor lookup table.

[0142] The target focus feature 0.35 is multiplied by the interaction factor 0.8 to obtain 0.35*0.8=0.28. This result will be used as part of the interactive embedding feature vector generated by the second feature focusing unit for the second round of optimization iterations, and together with the calculation results of other dimensions (if any), it constitutes a complete interactive embedding feature vector.

[0143] According to this process, similar interactive calculations are performed in each second feature focusing unit (b) of each round (a), so as to gradually generate interactive embedding feature vectors for each round and each unit. After subsequent optimization iterative operations, these vectors finally generate derived embedding feature vectors, which provide a basis for subsequent operations such as embedding restoration and generation of derived distributed multivariate data.

[0144] In a possible implementation, the iterative transfer condition further includes a business logic description document for the target distributed multivariate data.

[0145] In a possible implementation, the method further includes:

[0146] Step A110, obtaining template-derived template knowledge data, template-hidden distributed multivariate data, template-derived field tag data, and template-dynamically updated node data. The template-derived template knowledge data is determined based on the template-to-be-derived field data in the first template-distributed multivariate data, the template-hidden distributed multivariate data is generated by performing a hiding operation on the template-to-be-derived field data in the first template-distributed multivariate data, the template-derived field tag data is the tagging result of the template-to-be-derived field data, and the template-dynamically-updated node data is obtained by dynamically updating the template dynamic field data in the first template-distributed multivariate data and extracting the node. The template-to-be-derived field data includes the dynamic-to-be-derived field data in the template dynamic field data.

[0147] Step A120, respectively embedding the template-derived template knowledge data, the template-hidden distributed multivariate data, and the template-dynamically updated node data to generate a first template-embedded feature vector for the template-derived template knowledge data, a second template-embedded feature vector for the template-hidden distributed multivariate data, and a third template-embedded feature vector for the template-dynamically updated node data.

[0148] Step A130: perform feature conversion on the first template embedding feature vector using a feature conversion model to generate template feature conversion data.

[0149] Step A140, using the second template embedded feature vector, the third template embedded feature vector and the template derived field label data as iterative transfer conditions, using the iterative optimization model, and performing iterative optimization processing on the template randomly generated distributed multivariate data according to the template feature conversion data to generate a template derived embedded feature vector.

[0150] Step A150, embedding and restoring the template-derived embedded feature vector to generate template-derived distributed multivariate data.

[0151] Step A160, training the feature conversion model and the iterative optimization model according to the loss function value between the template-derived distributed multivariate data and the first template distributed multivariate data until the first model convergence requirement is met.

[0152] In this embodiment, for the target distributed multivariate data (including mobile payment data, personnel database data, scene monitoring data, etc.), the server has a corresponding business logic description document. This document describes in detail the logical relationship and business rules between the data.

[0153] For example, in terms of mobile payment data, the document states that there is a correlation between transaction amounts and user age, payment time, and payment location. Specifically, the amount of mobile payments made by users aged 20-30 at night (18:00-22:00) may be relatively high, and the payment amounts in commercial center areas are generally higher than in other areas. For personnel database data, it is associated with scene monitoring data. For example, when the flow of people monitored in certain specific scenes (such as large event venues) is large, certain attributes of the relevant personnel in the personnel database (such as activity tags) may be updated.

[0154] During the entire iterative optimization process (such as the iterative optimization of randomly generated distributed multivariate data based on the first embedded feature vector, the second embedded feature vector, etc. mentioned above), the server will refer to this business logic description document. For example, when determining the interaction factor or adjusting the feature value, according to the rules in the document, if the current iteration involves the relationship between age and payment amount, and the age is between 20 and 30 years old and the payment time is at night (18:22), the relevant feature value is adjusted or the corresponding weight bias is given when determining the interaction factor according to the logic mentioned in the document that the payment amount is relatively high in this case.

[0155] Assume that the first template distributed multivariate data includes simulated mobile payment, personnel database and scene monitoring data. For the template field data to be derived, for example, the transaction amount in the mobile payment data and the age in the personnel database data are determined as the template field data to be derived (because they may have a potential derivative relationship with other data features).

[0156] The server determines the template derived template knowledge data based on the field data to be derived from these templates. For example, the relationship between the transaction amount and the payment time, payment location and the age of the person is analyzed. If it is found that the payment amount of users aged 30-40 in the business district at noon (12-14 o'clock) on weekdays is mostly between 200-500 yuan, this relationship model is constructed as part of the template derived template knowledge data.

[0157] The template to-be-derived field data (such as transaction amount and age) in the first template distributed multivariate data is hidden. Taking the transaction amount as an example, the server can use a simple offset encryption method to add a random number (this random number is within a certain range, such as 0100) to each transaction amount value for hiding. For the age field, a fixed code can be used to replace the real age, such as replacing all ages with random numbers between 0100 to generate a template to hide distributed multivariate data.

[0158] Next, the data of the template's derived fields are marked. For the transaction amount, if the amount is less than 100 yuan, it is marked as "low amount" and represented by the number 1; 100-500 yuan is marked as "medium amount" and represented by the number 2; more than 500 yuan is marked as "high amount" and represented by the number 3. For age, those under 18 are marked as "minor" and represented by 1; those aged 18-60 are marked as "adult" and represented by 0; those aged over 60 are marked as "elderly" and represented by 1, thus obtaining the template's derived field mark data.

[0159] Next, in the first template distributed multivariate data, such as the transaction time in the mobile payment data, it is a dynamic field data. The server counts the number of transactions in a time window of half an hour. When the number of transactions exceeds 100, this half-hour time window is marked as a dynamic update node. For the flow of people in the scene monitoring data, when the change in the flow of people in two adjacent 10-minute time periods exceeds 50 people, this time point is marked as a dynamic update node, thereby obtaining the template dynamic update node data.

[0160] Next, the server quantifies the relational patterns in the template-derived template knowledge data. For example, for the previously mentioned relationship between users aged 30-40 and the amount of payment in the business district at noon (12-14 o'clock) on weekdays, most of them are between 200-500 yuan. The age range, payment time interval, payment location category, and amount range are numerically coded respectively, such as 30-40 years old is coded as 3040, 12-14 o'clock is coded as 1214, business district is coded as 1 (assuming different areas have different codes), and 200-500 yuan is coded as 200-500.

[0161] These quantized relational patterns are converted into first template embedding feature vectors using a neural network-based embedding model (such as a multi-layer perceptron with specific structure and parameters). This embedding model maps the input quantized relational pattern into a low-dimensional vector space by learning the intrinsic relationship in the data to obtain the first template embedding feature vector.

[0162] For template-hidden distributed multivariate data, take the mobile payment data after hiding as an example. In addition to the hidden transaction amount (such as the value after offset encryption), other fields include payment method (assuming that electronic payment is coded as 1 and cash payment is coded as 0), transaction location (coded according to geographic location), etc.

[0163] This data is embedded using an autoencoder (a neural network structure that can automatically learn data feature representation). The input layer of the autoencoder receives the encoding of each field of the template hidden distributed multivariate data. Through the encoding and decoding process of the hidden layer, the output layer outputs the second template embedded feature vector, which can capture the feature information in the hidden data.

[0164] For template dynamic update node data, such as the dynamic update nodes extracted from transaction time and personnel flow, the dynamic update nodes of transaction time are coded in chronological order, such as the first dynamic update node is coded as 1, the second is coded as 2, etc. For the dynamic update nodes of personnel flow, they are coded according to different flow threshold ranges, such as the first node with flow changes between 50100 people is coded as 11, the second is coded as 12, etc.

[0165] An embedding technology specifically for time series and discrete event coding (such as an embedding algorithm based on time series analysis) is used to convert these template dynamic update node data into a third template embedding feature vector, which can represent the characteristics of the dynamic update node in time and event logic.

[0166] Assume that the feature conversion model contains 3 first feature focusing units.

[0167] The first unit focuses on the feature conversion of the relationship between age and payment amount. It has an age-payment amount mapping table based on statistical analysis. For example, the probability distribution of the 30-40 age group in different payment amount intervals. When processing the part related to age and payment amount in the first template embedded feature vector, the features are adjusted according to the mapping table to obtain the first feature focus result of this part, including the adjusted age-payment amount relationship index and the new feature value.

[0168] The second unit focuses on the relationship between payment time and payment location, and transforms the relevant part of the first template embedded feature vector according to the pre-built payment time and payment location relationship model to obtain the corresponding first feature focusing result.

[0169] The third unit processes the relationship between payment method and age, and obtains the first feature focusing result of this part in a similar way, and finally combines them to obtain the template feature conversion data.

[0170] In addition, the iterative optimization model also includes three second feature focusing units.

[0171] The second template embedded feature vector, the third template embedded feature vector and the template derived field marker data are used as iterative transfer conditions, and the template randomly generates distributed multivariate data (initialized as a random numerical vector, for example, each vector dimension is 8, and the value is randomly generated between 01) according to the template feature conversion data for iterative optimization processing.

[0172] For example, in the first round of optimization iterations, a reference embedding feature vector is first generated, and the second template embedding feature vector, the third template embedding feature vector, the template-derived field label data, and the random embedding feature vector corresponding to the template randomly generated distributed multivariate data are fused (such as weighted summation).

[0173] In the first second feature focusing unit, an interactive calculation is performed based on the first feature focusing result generated by the first first feature focusing unit and the second feature focusing result of the first round of optimization iteration of the first second feature focusing unit (such as the index interaction operation, feature interaction operation, determination of interaction factors and final feature interaction calculation described previously) to generate an interactive embedded feature vector, and then an optimization iteration is performed to obtain a first temporary optimized iterative embedded feature vector.

[0174] A similar process is followed in the second and third second feature focusing units. After multiple rounds (assuming 3 rounds) of iterative optimization, the optimized embedding feature vector generated by the last round of optimization iteration is used as the template to derive the embedding feature vector.

[0175] Assume that in the previous embedding representation process, the template-derived template knowledge data was converted using a neural network embedding model, the template hidden distributed multivariate data was embedded using an autoencoder, and the template dynamically updated node data was embedded using a time series analysis embedding algorithm.

[0176] When embedding and restoring the template-derived embedding feature vector, the part converted by the neural network embedding model is restored using the inverse neural network operation. For example, if the neural network embedding model performs matrix multiplication and nonlinear activation function operations during forward propagation, it is restored by the reverse matrix multiplication and inverse activation function operations (if any) during restoration.

[0177] For the part processed by the autoencoder, the decoder part of the autoencoder is used to restore the encoded features to the representation in the original data space. For the part processed by the time series analysis embedding algorithm, the encoding is restored to the original dynamically updated node data representation according to the inverse process of the algorithm, thereby generating template-derived distributed multivariate data.

[0178] Thus, the loss function value between the template-derived distributed multivariate data and the first template distributed multivariate data is calculated. For example, the mean square error (MSE) is used as the loss function to calculate the sum of squared errors between the predicted value (such as the predicted transaction amount, age, etc.) in the template-derived distributed multivariate data and the true value in the first template distributed multivariate data.

[0179] According to this loss function value, the feature conversion model and the iterative optimization model are trained. If the loss function value is large, it means that the prediction result of the model is quite different from the real data, and the parameters of the model need to be adjusted. For example, the internal parameters of each first feature focusing unit in the feature conversion model (such as the probability value in the age payment amount mapping table, the coefficient in the relationship model, etc.) are adjusted, and the parameters of each second feature focusing unit in the iterative optimization model (such as the weight in the interactive calculation, the step size of the optimization iteration, etc.) are adjusted.

[0180] This process is repeated continuously, the loss function value is calculated and the model parameters are adjusted until the first model convergence requirement is met. For example, when the loss function value is less than a preset threshold (such as 0.01), the model is considered to have converged and training is stopped.

[0181] In a possible implementation, step A110 includes:

[0182] Step A111 , extracting derivable fields from the first template distributed multivariate data, and determining template field data to be derived in the first template distributed multivariate data.

[0183] Step A112, marking the field data to be derived from the template to generate the template derived field marking data.

[0184] Step A113, based on the template derived field marking data, hide the template to be derived field data in the first template distributed multivariate data to generate template hidden distributed multivariate data.

[0185] Step A114, processing the first template distributed multivariate data according to the template-derived field tag data to generate the template-derived template knowledge data.

[0186] Step A115, extracting the dynamic update nodes of the template dynamic field data in the first template distributed multivariate data to generate the template dynamic update node data.

[0187] In a possible implementation, step A113 includes:

[0188] Step A1131, converting the first template distributed multivariate data into a first eigenvalue interval in a regularized manner to generate first temporary template distributed multivariate data.

[0189] Step A1132, converting the template-derived field tag data into a second feature value interval in a regularized manner to generate first template-derived field tag data, wherein the second feature value interval is within the first feature value interval.

[0190] Step A1133: fuse the first temporary template distributed multivariate data and the first template derived field tag data element by element to generate second temporary distributed multivariate data.

[0191] Step A1134, performing inverse regularization conversion on the second temporary distributed multivariate data to generate the template hidden distributed multivariate data.

[0192] In a possible implementation, step A114 includes:

[0193] Step A1141, converting the template-derived field tag data into a second feature value interval in a regularized manner to generate second template-derived field tag data.

[0194] Step A1142, fusing the second template-derived field tag data and the first template distributed multivariate data element by element to generate the template-derived template knowledge data.

[0195] In a possible implementation, step A111 includes:

[0196] Step A1111: perform structural feature conversion on the first template distributed multivariate data using a first feature mapping structure to generate a template structure embedding feature vector.

[0197] Step A1112: perform logical feature conversion on the first template distributed multivariate data using a second feature mapping structure to generate a template logical embedding feature vector.

[0198] Step A1113, performing dimensionality reduction processing on the template structure embedding feature vector to generate a reduced-dimensionality structure embedding feature vector.

[0199] Step A1114, performing dimension-upgrading processing on the template logical embedding feature vector to generate a dimension-upgraded logical embedding feature vector.

[0200] Step A1115: Combine the template logic embedding feature vector and the dimension reduction structure embedding feature vector to generate a first template combined embedding feature vector.

[0201] Step A1116, combining the template structure embedding feature vector with the dimension-raising logic embedding feature vector to generate a second template combination embedding feature vector.

[0202] Step A1117: merge the first template combination embedding feature vector and the second template combination embedding feature vector to generate a template embedding feature vector.

[0203] Step A1118, determining the template to-be-derived field data in the first template distributed multivariate data based on the template embedded feature vector.

[0204] In this embodiment, it is continued to be assumed that the first template distributed multivariate data includes mobile payment data, personnel database data and scene monitoring data.

[0205] For mobile payment data, the first feature mapping structure can be a feature extractor based on a neural network. For example, the transaction amount, payment time, payment method and other fields in the mobile payment data are input, and the neural network maps the value of each field to a high-dimensional space. Taking the transaction amount as an example, 100 yuan may be mapped to a 10-dimensional vector [0.1, 0.2, -0.1, 0.3, ...], and the payment time may be mapped to another different 10-dimensional vector. Similar mapping operations are also performed for fields such as name, age, and gender in the personnel database data. Finally, these mapped vectors are combined to form a template structure embedded feature vector. Assuming that the dimension of the vector after the mobile payment data is mapped is 100, the dimension of the vector after the personnel database data is mapped is 80, and the dimension of the vector after the scene monitoring data is mapped is 60, the dimension of the combined template structure embedded feature vector is 240.

[0206] The second feature mapping structure is a mapper based on logical rules. For mobile payment data, it may be mapped according to the logical relationship between the payment amount and the payment time. For example, if the payment amount is greater than 500 yuan and the payment time is at night (18:22), a logical value of 1 is assigned, otherwise it is 0. For the personnel database data, it is mapped according to the logical relationship between age and gender. For example, if the age is between 18 and 30 years old and is male, a logical value of 1 is assigned. These logical values ​​are combined to form a template logical embedding feature vector. Assume that the dimension of this vector is determined to be 50 based on the number of logical relationships.

[0207] Next, the template structure embedding feature vector is subjected to dimensionality reduction processing. For example, the principal component analysis (PCA) algorithm is used to reduce the 240-dimensional template structure embedding feature vector to 120 dimensions, thereby obtaining a reduced-dimensional structure embedding feature vector. For the template logic embedding feature vector, a neural network-based dimensionality increase technique (such as a specific structure of a multilayer perceptron) is used to increase the 50-dimensional vector to 100 dimensions, thereby generating an increased-dimensional logic embedding feature vector.

[0208] The template logic embedding feature vector (100 dimensions) after dimension increase is combined with the template structure embedding feature vector (120 dimensions) after dimension reduction to obtain the first template combination embedding feature vector, whose dimension is 220 dimensions. At the same time, the original template structure embedding feature vector (240 dimensions) is combined with the template logic embedding feature vector (100 dimensions) after dimension increase to obtain the second template combination embedding feature vector, whose dimension is 340 dimensions.

[0209] Then, the two combined embedding feature vectors are merged. For example, the first template combined embedding feature vector and the second template combined embedding feature vector are sequentially concatenated to obtain a template embedding feature vector with a dimension of 560.

[0210] The server determines the field data to be derived from the template according to the numerical features in the template embedded feature vector and predefined rules. For example, if the values ​​related to the payment amount and age in some dimensions in the template embedded feature vector have a higher weight or a specific numerical pattern, the payment amount in the mobile payment data and the age in the personnel database data are determined as the field data to be derived from the template.

[0211] For the previously determined template field data to be derived, such as the payment amount in mobile payment data and the age in personnel database data.

[0212] For the payment amount, the server marks it according to the following rules: if the payment amount is less than 100 yuan, it is marked as "small amount" and represented by the number 1; if the amount is between 100 yuan and 500 yuan, it is marked as "medium amount" and represented by the number 2; if the amount is greater than 500 yuan, it is marked as "large amount" and represented by the number 3.

[0213] Age is marked according to the following rules: if the age is less than 18 years old, it is marked as "minor" and represented by 1; if the age is between 18 and 60 years old, it is marked as "adult" and represented by 0; if the age is greater than 60 years old, it is marked as "elderly" and represented by 1.

[0214] Through such tagging processing, template-derived field tag data is generated.

[0215] For the mobile payment data in the first template distributed multivariate data, assume that the transaction amount ranges from 0 to 1,000 yuan, the payment time is 0 to 24 hours (in hours), and the payment method is represented by 0 (cash) and 1 (electronic payment). The server converts the transaction amount to the first eigenvalue interval of 0 to 1 according to the linear transformation rule. For example, a transaction amount of 100 yuan is converted to 0.1 (100 / 1000). The payment time is also converted similarly, such as 12 o'clock is converted to 0.5 (12 / 24). For the age in the personnel database data, assume that the age range is 0 to 100 years old, and convert the age value to the interval of 0 to 1. For example, 30 years old is converted to 0.3 (30 / 100), thus generating the first temporary template distributed multivariate data.

[0216] For the previously generated template derived field tag data, such as the payment amount tag (1, 2, 3) and the age tag (1, 0, 1), convert these tags to a smaller second feature value interval, such as 0 to 0.5. If the payment amount is marked as 1 (small amount), convert it to 0.1; if it is marked as 2 (medium amount), convert it to 0.3; if it is marked as 3 (large amount), convert it to 0.5. For the age tag, 1 (minor) is converted to 0.1, 0 (adult) is converted to 0.3, and 1 (elderly) is converted to 0.5, and the first template derived field tag data is obtained.

[0217] Each element in the distributed multivariate data of the first temporary template (such as 0.1 after the transaction amount is converted in the mobile payment data, 0.5 after the payment time is converted, etc.) and the corresponding element in the derived field tag data of the first template (such as 0.1 after the payment amount tag is converted) are fused element by element. For example, a simple addition operation can be used to add 0.1 after the transaction amount is converted and 0.1 after the payment amount tag is converted to get 0.2. The same operation is performed on the age-related elements in the personnel database data to generate the second temporary distributed multivariate data.

[0218] Perform an inverse regularization conversion on each element in the second temporary distributed multivariate data. For example, for 0.2 obtained by merging the transaction amount and the payment amount mark, restore it according to the inverse process of the previous regularization conversion. Since the transaction amount was previously converted from 0 to 1,000 yuan to 0 to 1, now the inverse conversion is reversed, and the transaction amount corresponding to 0.2 is 200 yuan (0.2*1000). Similar inverse conversion operations are also performed on elements such as payment time and age, and finally the template hidden distributed multivariate data is obtained.

[0219] Similar to the previous operation, the template-derived field tag data (such as payment amount tags 1, 2, 3 and age tags 1, 0, 1) are converted to a second characteristic value interval of 0 to 0.5 to obtain the second template-derived field tag data (such as payment amount tags converted to 0.1, 0.3, 0.5 and age tags converted to 0.1, 0.3, 0.5).

[0220] For each element in the distributed multivariate data of the first template, such as the transaction amount, payment time, and payment method in the mobile payment data, and the name, age, and gender in the personnel database data, each element is fused with the corresponding element in the derived field tag data of the second template element by element. For example, for a transaction amount of 100 yuan, the corresponding payment amount tag is converted to 0.1, and the fused result can be a new value (such as using a simple weighted sum, 100*0.9+0.1*100=91). For the age of 30 in the personnel database data, the corresponding age tag is converted to 0.3, and the fused result may be 30*0.7+0.3*30=30 (here is just an example of a fusion method). Through such an element-by-element fusion operation, template-derived template knowledge data is generated.

[0221] It is assumed that the template dynamic field data in the first template distributed multivariate data includes the transaction time in the mobile payment data and the personnel flow in the scene monitoring data.

[0222] For the transaction time in the mobile payment data, the server counts the number of transactions in a time window of 15 minutes. If the number of transactions in a 15-minute time window exceeds 100, the time window is marked as a dynamic update node. For example, if the number of transactions in the time window from 10:00 to 10:15 am is 120, then the time window is marked as a dynamic update node.

[0223] For the flow of people in the scene monitoring data, the change of flow of people is counted every 10 minutes. If the change of flow of people in two adjacent 10-minute time periods exceeds 50 people, this time point is marked as a dynamic update node. For example, the flow of people from 9:00 to 9:10 is 100 people, and the flow of people from 9:10 to 9:20 is 160 people. The change of flow of people is 60 people, which exceeds 50 people, so 9:10 is marked as a dynamic update node. Through such operations, the template dynamic update node data is generated.

[0224] In a possible implementation, the iterative transfer condition also includes a business logic description document generated by a business logic construction model based on the target distributed multivariate data.

[0225] The method further comprises:

[0226] Step B110, using a data structure conversion unit to perform feature conversion on the second template distributed multivariate data to generate a template data structure vector.

[0227] Step B120: Use a logic conversion unit to perform feature conversion on the template logic description document corresponding to the second template distributed multivariate data to generate a template logic conversion vector.

[0228] Step B130, using a determination unit to perform logic relevance determination on the template data structure vector and the template logic description document, and generating a template logic relevance determination result.

[0229] Step B140, using the business logic to build a model based on the template data structure vector and the template logic description document to generate a prediction logic description document.

[0230] Step B150, determining the business logic construction error based on the loss function value between the prediction logic description document and the template logic description document, the loss function value between the template data structure vector and the template logic conversion vector, and the template logic correlation judgment result.

[0231] Step B160, using the business logic construction error to train the data structure conversion unit, the logic conversion unit, the discrimination unit and the business logic construction model until the second model convergence requirement is met.

[0232] In this embodiment, for the target distributed multivariate data (including mobile payment data, personnel database data, scene monitoring data, etc.), the business logic construction model will analyze these data to generate a business logic description document.

[0233] For example, the business logic construction model analyzes the relationship between transaction amount, payment time, payment method in mobile payment data, and age, gender and other information in personnel database data. It finds that users aged 20-30 are more inclined to use electronic payment methods for small purchases (less than 100 yuan) at night (18:00-22:00), so this relationship pattern is recorded in the business logic description document. Throughout the processing, this document participates in related operations as an iterative transfer condition. For example, when performing feature enhancement or model training on data, if it involves processing features related to age, payment time and payment amount, the logical relationship in this document will be referenced to ensure that the processing process conforms to the actual business logic of the data.

[0234] Assume that the second template distributed multivariate data includes simulated mobile payment and personnel database data. The mobile payment data has fields such as transaction amount, payment time, and payment method, and the personnel database data has fields such as age, gender, and occupation.

[0235] The data structure conversion unit is a neural network based structure, such as a multi-layer perceptron (MLP).

[0236] For the transaction amount in the mobile payment data, assuming that the original value range is 01000 yuan, the data structure conversion unit maps it to a new feature space. For example, through the first hidden layer of MLP, the transaction amount of 100 yuan may be converted into a 3-dimensional vector [0.1, 0.2, 0.3]. This conversion process is calculated based on the weights and activation functions inside the MLP. For the payment time, originally in hours (024), after conversion, it may become a 4-dimensional vector, representing the feature representation of different time periods. If the payment method is binary (cash or electronic payment), it may be converted into a 2-dimensional vector.

[0237] For the age in the personnel database data, assuming the range is 0-100 years old, it is converted into a 5-dimensional vector after being converted by the data structure conversion unit. This vector can better represent the characteristic relationship of age in the entire data structure. If gender is binary (male or female), it is converted into a different vector representation, and occupation may be converted into a multidimensional weight vector according to the pre-defined occupation category code.

[0238] Finally, these converted vectors are combined to form a template data structure vector. For example, if the total dimension of the converted vector of mobile payment data is 10 dimensions and the total dimension of the converted vector of personnel database data is 15 dimensions, then the dimension of the template data structure vector is 25 dimensions.

[0239] The template logic description document corresponding to the second template distributed multivariate data contains a description of the logical relationship of these data. For example, the document describes that in a specific scenario, men aged 30-40 are more likely to use a certain payment method to make transactions of medium amounts (100,500 yuan) at noon (12:00-14:00) on weekdays.

[0240] The logical conversion unit uses a technology based on word vectors and logical rule encoding. For each logical element in the document, the encoding conversion is performed. For example, "age 30-40 years old" may be encoded as a specific vector [0.1, 0.2, -0.1], "male" is encoded as [0.3, -0.2, 0.1], "noon on weekdays (12-14 o'clock)" is encoded as [0.2, 0.3, 0.4], "a certain payment method" is encoded as [0.1, 0.2, 0.3], and "medium amount (100500 yuan)" is encoded as [0.1, -0.1, 0.2].

[0241] These encoded vectors are then combined in a certain logical order, such as by weighted summation or concatenation, to generate a template logical transformation vector. Assume that the dimension of this vector is ultimately 20 dimensions according to the encoding and combination method.

[0242] The discrimination unit receives the template data structure vector (25 dimensions) and the template logic description document (in the form of previously generated logic coding).

[0243] The discriminant unit has a predefined logical correlation discriminant model inside, which may be based on some statistical rules and logical algorithms. For example, it calculates the distance measure (such as Euclidean distance or cosine similarity) between certain dimensions in the template data structure vector and the corresponding logical element encoding vector in the template logical description document.

[0244] For the age-related dimension in the template data structure vector, calculate its distance with the "age between 30 and 40 years old" encoding vector in the template logic description document. If the distance is small, it means that the age-related part in the data structure and logic description has a high correlation. Similar distance calculations are performed for other elements such as payment time, payment amount, gender, etc.

[0245] Based on these distance measurement results, the template logical correlation judgment results are generated. For example, if the distance between most elements is less than a preset threshold (such as 0.5), the judgment result is "high correlation", represented by the number 1; if the distance between some elements is large, the judgment result is "medium correlation", represented by the number 0; if the distance between many elements is large, the judgment result is "low correlation", represented by the number 1.

[0246] The business logic construction model takes the template data structure vector (25 dimensions) and the template logic description document (logical coding form) as input.

[0247] Assume that the business logic construction model is a neural network-based generative model, such as a recurrent neural network (RNN) with an attention mechanism.

[0248] The model first analyzes the template data structure vector and identifies the characteristics of each element (such as age, payment amount and other related vectors). Then, it combines the logical relationship in the template logic description document to generate a prediction logic description document.

[0249] For example, based on the vector representation of age in the template data structure vector and the logical relationship between age and payment behavior in the template logic description document, the model predicts that in a new scenario, users aged 25-35 may use a new payment method to make small purchases at a specific time (such as a weekend afternoon). This prediction logic description document contains a predictive description of the relationship between data and has a similar structure to the original template logic description document, but the content is the prediction result based on the data structure vector and the original logic relationship.

[0250] Then, the loss function value between the predicted logic description document and the template logic description document is calculated. For example, using the cross entropy loss function, if a predicted relationship in the predicted logic description document (such as the relationship between age and payment behavior) is inconsistent with the actual relationship in the template logic description document, a large loss value will be generated. Assuming that the payment behavior of a user with a predicted age of 25-35 on weekend afternoons is quite different from the relationship in the actual template logic description document, then this part will contribute greatly to the cross entropy loss value.

[0251] At the same time, the loss function value between the template data structure vector and the template logic conversion vector is calculated. For example, the mean square error (MSE) loss function is used to calculate the sum of squared errors between the corresponding elements of the two vectors. If there is a large difference between the age-related vector in the template data structure vector and the age-related encoding vector in the template logic conversion vector, this MSE loss value will increase.

[0252] Finally, the business logic construction error is determined based on the loss function value between the prediction logic description document and the template logic description document, the loss function value between the template data structure vector and the template logic conversion vector, and the template logic relevance judgment result. For example, if the loss value between the prediction logic description document and the template logic description document is large, the loss value between the template data structure vector and the template logic conversion vector is also large, and the template logic relevance judgment result is "low relevance", then the business logic construction error will be very high, indicating that the model performs poorly in building business logic.

[0253] Therefore, the data structure conversion unit, the logic conversion unit, the discrimination unit, and the business logic construction model are trained using the business logic construction error. For the data structure conversion unit (such as a multi-layer perceptron), its internal weights and bias items are adjusted according to the error. For example, if it is found that the age-related conversion result leads to a large error, the age-related input to the hidden layer and the hidden layer to the output layer weights are adjusted.

[0254] For the logic conversion unit, the encoding method or weight of the logic element is adjusted according to the error. For example, if the correlation between the encoding vector of a certain logic element and the data structure vector is not good, resulting in a large error, the encoding method of this logic element is adjusted.

[0255] For the discrimination unit, adjust the threshold or internal statistical rules of the discrimination logic relevance according to the error. For the business logic construction model (such as RNN with attention mechanism), adjust the weight of its neural network, the parameters of the attention mechanism, etc. according to the error. Repeat this training process until the convergence requirements of the second model are met. For example, when the loss function value between the prediction logic description document and the template logic description document is less than a preset threshold (such as 0.01), and the loss function value between the template data structure vector and the template logic conversion vector is also less than another threshold (such as 0.05), and the template logic relevance discrimination result reaches "high correlation" (such as the discrimination result is 1), it is considered that the convergence requirements of the second model are met and training is stopped.

[0256] Figure 2 The system 100 based on distributed multivariate data analysis shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the system 100 based on distributed multivariate data analysis may also include a transceiver 1004, which may be used for data interaction between the server and other servers, such as data transmission and / or data reception. It should be noted that the transceiver 1004 is not limited to one in actual scheduling, and the structure of the system 100 based on distributed multivariate data analysis does not constitute a limitation on the embodiments of the present application.

[0257] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0258] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0259] The memory 1003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0260] The memory 1003 is used to store program codes for executing the embodiments of the present application, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the above method embodiments.

[0261] In addition, an embodiment of the present application further provides a readable storage medium, in which computer executable instructions are pre-installed. When a processor executes the computer executable instructions, the above method based on distributed multivariate data analysis is implemented.

[0262] Similarly, it should be noted that in order to simplify the description disclosed in this application, thereby helping to understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof. Similarly, it should be noted that in order to simplify the description disclosed in this application, thereby helping to understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof.

Claims

1. A method based on distributed multivariate data analysis, characterized in that: The method comprises: Performing a hiding operation on the field data to be derived in the target distributed multivariate data to generate hidden distributed multivariate data, wherein the target distributed multivariate data includes at least two types of data among mobile payment data, personnel database data, scene monitoring data, supervision platform data, and behavior log platform data; Dynamically updating node extraction is performed on the dynamic field data in the target distributed multivariate data to generate dynamic updating node data; the field data to be derived includes the dynamic field data to be derived in the dynamic field data; Marking the to-be-derived field data in the target distributed multivariate data to generate derived field marked data; Based on the target derived template knowledge data, the dynamically updated node data and the derived field tag data, feature enhancement is performed on the hidden distributed multivariate data to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data; A training template for a corresponding machine learning network model is constructed based on the derived distributed multivariate data and the target distributed multivariate data.

2. The method based on distributed multivariate data analysis according to claim 1, characterized in that: The method of enhancing the features of the hidden distributed multivariate data based on the target derived template knowledge data, the dynamically updated node data and the derived field tag data to generate derived distributed multivariate data combining the derived feature dimensions expressed by the target derived template knowledge data includes: Embedding the target derived template knowledge data, the hidden distributed multivariate data and the dynamically updated node data respectively to generate a first embedded feature vector of the target derived template knowledge data, a second embedded feature vector of the hidden distributed multivariate data and a third embedded feature vector of the dynamically updated node data; According to the first embedded feature vector, the second embedded feature vector, the third embedded feature vector and the derived field tag data, iteratively optimize the randomly generated distributed multivariate data to generate a derived embedded feature vector; The derived embedded feature vector is embedded and restored to generate derived distributed multivariate data that combines the derived feature dimensions expressed by the target derived template knowledge data.

3. The method based on distributed multivariate data analysis according to claim 2, characterized in that: The iterative optimization processing of randomly generated distributed multivariate data is performed according to the first embedded feature vector, the second embedded feature vector, the third embedded feature vector and the derived field tag data to generate a derived embedded feature vector, including: Performing feature conversion on the first embedded feature vector using a feature conversion model to generate feature conversion data; The second embedded feature vector, the third embedded feature vector and the derived field label data are used as iterative transfer conditions, and an iterative optimization model is used to iteratively optimize the randomly generated distributed multivariate data according to the feature conversion data to generate the derived embedded feature vector.

4. The method based on distributed multivariate data analysis according to claim 3, characterized in that: The feature conversion model includes X first feature focusing units, and the iterative optimization model includes X second feature focusing units; The feature conversion data includes the first feature focusing results loaded by each of the first feature focusing units; The method uses the second embedded feature vector, the third embedded feature vector and the derived field tag data as iterative transfer conditions, uses an iterative optimization model, and performs iterative optimization processing on the randomly generated distributed multivariate data according to the feature conversion data to generate the derived embedded feature vector, including: Using the second embedded feature vector, the third embedded feature vector and the derived field tag data as iterative transfer conditions, using an iterative optimization model, performing Y rounds of optimization iterations on the randomly generated distributed multivariate data according to the feature conversion data, and using the optimized embedded feature vector generated by the Yth round of optimization iteration as the derived embedded feature vector; The specific steps of the optimization iteration of the ath round include: Obtaining a reference embedded feature vector of the optimization iteration of the ath round, wherein the reference embedded feature vector of the optimization iteration of the first round is generated by fusing the second embedded feature vector, the third embedded feature vector, the derived field tag data, and the random embedded feature vector corresponding to the randomly generated distributed multivariate data; when 1<a≤Y, the reference embedded feature vector of the optimization iteration of the ath round is the optimized embedded feature vector of the optimization iteration of the a-1th round; In the bth second feature focusing unit, interactive calculation is performed based on the first feature focusing result generated by the bth first feature focusing unit and the second feature focusing result of the bth second feature focusing unit for the ath round of optimization iteration, to generate an interactive embedding feature vector generated by the bth second feature focusing unit for the ath round of optimization iteration; 1≤b≤X, b is a positive integer; the second feature focusing result of the first second feature focusing unit for the ath round of optimization iteration is determined based on the reference embedding feature vector of the ath round of optimization iteration; Performing optimization iteration on the interactive embedding feature vector generated by the b-th second feature focusing unit to generate a b-th temporary optimized iterative embedding feature vector for the a-th round of optimization iteration; If b is less than X, determine the second feature focusing result of the b+1th second feature focusing unit for the ath round of optimization iteration according to the bth temporary optimization iteration embedded feature vector for the ath round of optimization iteration, count and accumulate b, and return to execute the step of interactively calculating in the bth second feature focusing unit according to the first feature focusing result generated by the bth first feature focusing unit and the second feature focusing result of the bth second feature focusing unit for the ath round of optimization iteration, to generate the interactive embedded feature vector generated by the bth second feature focusing unit for the ath round of optimization iteration; If b=X, the bth temporary optimized iteration embedding feature vector for the ath round of optimization iteration is used as the optimized embedding feature vector for the ath round of optimization iteration.

5. The method based on distributed multivariate data analysis according to claim 4, characterized in that: The first feature focusing result includes a first focusing index and a first focusing feature; the second feature focusing result includes a second focusing index, a second focusing feature and a target interaction index; In the b-th second feature focusing unit, interactive calculation is performed according to the first feature focusing result generated by the b-th first feature focusing unit and the second feature focusing result of the b-th second feature focusing unit for the a-th round of optimization iteration, to generate the interactive embedding feature vector generated by the b-th second feature focusing unit for the a-th round of optimization iteration, including: In the bth second feature focusing unit, the first focusing index and the second focusing index of the bth second feature focusing unit for the ath round of optimization iteration are interactively calculated to generate a target focusing index of the bth second feature focusing unit for the ath round of optimization iteration; Performing interactive operations on the first focusing feature and the second focusing feature of the b-th second feature focusing unit for the a-th round of optimization iterations to generate a target focusing feature of the b-th second feature focusing unit for the a-th round of optimization iterations; Determine an interaction factor of the bth second feature focusing unit for the ath round of optimization iteration based on the target interaction index of the bth second feature focusing unit for the ath round of optimization iteration and the target focusing index; Based on the interaction factor of the b-th second feature focusing unit for the a-th round of optimization iteration, the corresponding target focusing feature is interactively calculated to generate an interactive embedding feature vector generated by the b-th second feature focusing unit for the a-th round of optimization iteration.

6. The method based on distributed multivariate data analysis according to claim 3, characterized in that: The iterative transfer condition also includes a business logic description document for the target distributed multivariate data.

7. The method based on distributed multivariate data analysis according to any one of claims 3 to 6, characterized in that: The method further comprises: Acquire template-derived template knowledge data, template-hidden distributed multivariate data, template-derived field tag data, and template-dynamically updated node data; the template-derived template knowledge data is determined based on the template-to-be-derived field data in the first template-distributed multivariate data, the template-hidden distributed multivariate data is generated by performing a hiding operation on the template-to-be-derived field data in the first template-distributed multivariate data, the template-derived field tag data is the tagging result of the template-to-be-derived field data, and the template-dynamically-updated node data is obtained by dynamically updating the template dynamic field data in the first template-distributed multivariate data and extracting the node; the template-to-be-derived field data includes the dynamic-to-be-derived field data in the template dynamic field data; The template-derived template knowledge data, the template-hidden distributed multivariate data, and the template-dynamically updated node data are respectively embedded to generate a first template-embedded feature vector for the template-derived template knowledge data, a second template-embedded feature vector for the template-hidden distributed multivariate data, and a third template-embedded feature vector for the template-dynamically updated node data; Performing feature conversion on the first template embedding feature vector using a feature conversion model to generate template feature conversion data; Using the second template embedding feature vector, the third template embedding feature vector and the template derived field tag data as iterative transfer conditions, using an iterative optimization model, and performing iterative optimization processing on the template randomly generated distributed multivariate data according to the template feature conversion data to generate a template derived embedding feature vector; Embedding and restoring the template-derived embedded feature vector to generate template-derived distributed multivariate data; The feature conversion model and the iterative optimization model are trained according to the loss function value between the template-derived distributed multivariate data and the first template distributed multivariate data until the first model convergence requirement is met.

8. The method based on distributed multivariate data analysis according to claim 7, characterized in that: The obtaining of template-derived template knowledge data, template-hidden distributed multivariate data, template-derived field tag data, and template-dynamically updated node data includes: Extracting derivable fields from the first template distributed multivariate data to determine template field data to be derived in the first template distributed multivariate data; Marking the template to-be-derived field data to generate the template derived field marking data; According to the template derived field mark data, a hiding operation is performed on the template to be derived field data in the first template distributed multivariate data to generate template hidden distributed multivariate data; Processing the first template distributed multivariate data according to the template-derived field tag data to generate the template-derived template knowledge data; Extracting dynamic update nodes from the template dynamic field data in the first template distributed multivariate data to generate the template dynamic update node data; The step of performing a hiding operation on the template to-be-derived field data in the first template distributed multivariate data according to the template derived field mark data to generate template hidden distributed multivariate data includes: Regularly converting the first template distributed multivariate data into a first eigenvalue interval to generate first temporary template distributed multivariate data; Regularly converting the template-derived field tag data into a second feature value interval to generate first template-derived field tag data, wherein the second feature value interval is within the first feature value interval; The first temporary template distributed multivariate data and the first template derived field tag data are merged element by element to generate second temporary distributed multivariate data; Performing inverse regularization conversion on the second temporary distributed multivariate data to generate the template hidden distributed multivariate data; The step of processing the first template distributed multivariate data according to the template-derived field tag data to generate the template-derived template knowledge data includes: Regularly converting the template-derived field tag data into a second characteristic value interval to generate second template-derived field tag data; Fusing the second template-derived field tag data and the first template-distributed multivariate data element by element to generate the template-derived template knowledge data; The extracting derivable fields from the first template distributed multivariate data to determine the template field data to be derived in the first template distributed multivariate data includes: Using a first feature mapping structure to perform structural feature conversion on the first template distributed multivariate data to generate a template structure embedding feature vector; Performing logical feature conversion on the first template distributed multivariate data using a second feature mapping structure to generate a template logical embedding feature vector; Performing dimensionality reduction processing on the template structure embedding feature vector to generate a reduced-dimensionality structure embedding feature vector; Performing dimension-upgrading processing on the template logic embedding feature vector to generate a dimension-upgraded logic embedding feature vector; Combining the template logic embedding feature vector with the dimension reduction structure embedding feature vector to generate a first template combined embedding feature vector; Combining the template structure embedding feature vector with the dimension-raising logic embedding feature vector to generate a second template combination embedding feature vector; Merging the first template combination embedding feature vector and the second template combination embedding feature vector to generate a template embedding feature vector; Based on the template embedding feature vector, the template to-be-derived field data in the first template distributed multivariate data is determined.

9. The method based on distributed multivariate data analysis according to any one of claims 3 to 6, characterized in that: The iterative transfer condition also includes a business logic description document generated by the business logic construction model based on the target distributed multivariate data; The method further comprises: Using a data structure conversion unit to perform feature conversion on the second template distributed multivariate data to generate a template data structure vector; Using a logic conversion unit to perform feature conversion on a template logic description document corresponding to the second template distributed multivariate data to generate a template logic conversion vector; Using a discriminating unit to perform logical correlation discrimination on the template data structure vector and the template logic description document, and generate a template logic correlation discrimination result; Using the business logic to build a model based on the template data structure vector and the template logic description document, generate a prediction logic description document; Determine the business logic construction error according to the loss function value between the prediction logic description document and the template logic description document, the loss function value between the template data structure vector and the template logic conversion vector, and the template logic correlation judgment result; The data structure conversion unit, the logic conversion unit, the discrimination unit and the business logic construction model are trained using the business logic construction error until a second model convergence requirement is met.

10. A system based on distributed multivariate data analysis, characterized in that: The system based on distributed multivariate data analysis includes a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and the machine-executable instructions are loaded and executed by the processor to implement the method based on distributed multivariate data analysis described in any one of claims 1-9.

Images