Artificial intelligence model building method and device, equipment and storage medium

Abstract

The invention relates to an artificial intelligence model building method and device, equipment and a storage medium. The method comprises the following steps: preprocessing input original data to obtain a privacy protection data set; performing feature extraction through a multi-scale convolutional neural network connected in parallel in the target model to obtain a target multi-level local feature; dividing the multi-level local feature into two parts to obtain a first part of local feature and a second part of local feature, and inputting the second part of local feature into a bidirectional long-short-term memory network and a bidirectional gating loop unit network which are connected in parallel in a target model for feature extraction to obtain a global semantic feature; splicing and weighting the first part of local features and the global semantic features to obtain fusion feature representation; and performing model parameter optimization on the target model based on the fusion feature representation to obtain a trained artificial intelligence model. According to the invention, the training speed and convergence stability of the artificial intelligence model are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence model building method, device, equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, the demand for AI models in all walks of life is growing. However, traditional AI model building methods often have problems such as insufficient data privacy protection, limited feature extraction capabilities, and poor model generalization performance. In the era of big data, how to fully utilize massive data to train high-performance AI models while protecting user privacy has become a difficult problem that needs to be solved urgently. In addition, existing AI models often find it difficult to effectively capture multi-scale features and long-range dependencies in the data, resulting in poor performance of the model when processing complex tasks.

[0003] Another significant challenge is how to balance the model's local feature learning ability and global semantic understanding ability. A single network structure is difficult to take both aspects into account at the same time, and simple feature concatenation cannot fully utilize the advantages of different types of features. How to design a model architecture that can organically integrate multiple deep learning technologies to achieve better feature representation and stronger model performance has become a difficult problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide an artificial intelligence model building method, device, equipment and storage medium to improve the training speed and convergence stability of the artificial intelligence model.

[0005] To achieve the above object, the present invention provides an artificial intelligence model building method, comprising the following steps: Preprocessing the input raw data to obtain a preprocessed data set, adding Laplace noise to the preprocessed data set, and performing data standardization and normalization processing to obtain a privacy-preserving data set; By using a multi-scale convolutional neural network in parallel in the target model, feature extraction is performed on the privacy-preserving dataset to obtain multi-level local features of the target; Dividing the multi-level local features into two parts to obtain a first part of local features and a second part of local features, and inputting the second part of local features into a bidirectional long short-term memory network and a bidirectional gated recurrent unit network connected in parallel in the target model for feature extraction to obtain a global semantic feature; The first part of local features and the global semantic features are concatenated and weighted by a multi-head attention mechanism to obtain a fused feature representation; Based on the fused feature representation, the model parameters of the target model are optimized by batch gradient descent method with adaptive learning rate to obtain a trained artificial intelligence model.

[0006] The present invention also provides an artificial intelligence model building device, comprising: A preprocessing module is used to preprocess the input raw data to obtain a preprocessed data set, add Laplace noise to the preprocessed data set, and perform data standardization and normalization processing to obtain a privacy-preserving data set; A first extraction module is used to extract features from the privacy-preserving dataset through a multi-scale convolutional neural network connected in parallel in a target model to obtain multi-level local features of the target; A second extraction module is used to divide the multi-level local features into two parts to obtain a first part of local features and a second part of local features, and input the second part of local features into a bidirectional long short-term memory network and a bidirectional gated recurrent unit network connected in parallel in the target model for feature extraction to obtain a global semantic feature; A concatenation module, used for concatenating the first part of local features with the global semantic features, and weighting them through a multi-head attention mechanism to obtain a fused feature representation; The optimization module is used to optimize the model parameters of the target model based on the fusion feature representation through a batch gradient descent method with an adaptive learning rate to obtain a trained artificial intelligence model.

[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0008] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0009] In summary, the technical solution provided by the present invention effectively protects the privacy of the original data by adding Laplace noise to the preprocessed data set, and combines data standardization and normalization processing, while retaining the statistical characteristics of the data, and adopts a parallel multi-scale convolutional neural network, including convolution branches of small, medium and large scales, which can fully capture the multi-scale local features in the data and improve the model's recognition ability of different scale patterns. Through the parallel bidirectional long short-term memory network and the bidirectional gated recurrent unit network, the long-term and short-term dependencies in the data are effectively captured, and the model's ability to understand sequence data is enhanced. The multi-head attention mechanism is used to perform weighted fusion of local features and global semantic features, realize the organic combination of different types of features, and improve the richness and expression of feature representation. The batch gradient descent method with adaptive learning rate is adopted, combined with the Adam optimizer and gradient clipping technology, to achieve efficient optimization of model parameters and improve the training speed and convergence stability of the model. By introducing residual connection, layer normalization and other technologies, and applying weight decay and early stopping strategies, the overfitting problem is effectively alleviated and the generalization ability of the model is improved. The design of the multi-head attention mechanism and feature fusion network provides a more intuitive explanation for model decisions and enhances the interpretability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a schematic diagram of the steps of a method for building an artificial intelligence model in one embodiment of the present invention; Figure 2 This is a structural block diagram of an artificial intelligence model building device in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0011] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0013] Reference Figure 1 , this embodiment provides an artificial intelligence model building method, comprising the following steps: S1, preprocess the input raw data to obtain a preprocessed data set, add Laplace noise to the preprocessed data set, and perform data standardization and normalization to obtain a privacy-preserving data set; Among them, the input raw data is cleaned to remove noise, incomplete data or outliers, and a cleaned data set is obtained. Feature extraction is performed based on the cleaned data set to obtain a preliminary feature data set. Data encoding is performed on the preliminary feature data set, and the categorical variables therein are converted into numerical variables to generate an encoded feature data set. In order to meet the requirements of differential privacy, the Laplace noise parameter ε that meets the requirements of differential privacy is calculated based on the NN-FC framework. The calculation formula of this parameter is ε=ln(1+σ / Δf), where σ represents the sensitivity of the data and Δf represents the global sensitivity of the function. This formula ensures that the generated noise can minimize the impact on data accuracy while ensuring data privacy. According to the calculated Laplace noise parameter ε, random noise with Laplace distribution is generated for the encoded feature data set to obtain a noise matrix. The noise matrix is ​​added to the encoded feature data set at the element level to obtain a data set with added noise. By introducing noise to enhance the privacy of the data, the sensitive information of the original data is effectively protected. The data set after adding noise is normalized. The data is converted into a standardized data set with a uniform scale by returning the data to zero according to its mean and scaling it according to the standard deviation. This process helps to eliminate the dimensional differences between different features, making model training more stable and efficient. Normalization is performed on the basis of the standardized data set, scaling the data to a specific range, usually [0, 1], to obtain a normalized data set. Normalization can improve the convergence speed and accuracy of the model, while reducing the impact of certain large eigenvalues ​​on the model training process. In order to optimize the distribution characteristics of the data, the normalized data set is balanced to ensure that the data samples of each category are evenly distributed and reduce the deviation of the model during the training process. The balanced data set has good data distribution characteristics, reduces the bias in model training, and enhances the model's ability to recognize different categories. Feature selection is performed based on the balanced data set to eliminate redundant or irrelevant features and retain the features with the strongest correlation with model prediction, and finally obtain a privacy-preserving data set.

[0014] S2, extracts features from the privacy-preserving dataset through a multi-scale convolutional neural network in parallel in the target model to obtain multi-level local features of the target; Among them, the privacy-preserving dataset is segmented into training set, validation set and test set to ensure that the performance of the model on different datasets is fully evaluated and obtain the dataset for model training. Based on this dataset for model training, a multi-scale convolutional neural network is constructed. The network includes three parallel convolution branches, namely small-scale branch, medium-scale branch and large-scale branch, to extract different levels of features in the data. In the process of constructing the small-scale branch, the network uses three consecutive convolution layers. The convolution kernel size of each convolution layer is 3x3, the step size is set to 1, the padding is 1, and the number of output channels is 64, 128 and 256 respectively. Each convolution layer is followed by a batch normalization layer and a ReLU activation function to ensure the stability and nonlinear expression ability of the network. In order to aggregate features, a global maximum pooling layer is added to obtain a small-scale feature map by extracting the most significant global features. Meanwhile, in the mesoscale branch, the network also consists of three consecutive convolutional layers, but the kernel size of each convolutional layer is 5x5, the stride is 1, the padding is 2, and the number of output channels is 64, 128, and 256. Each convolutional layer is also followed by a batch normalization layer and a ReLU activation function to maintain the training stability of the network. A global average pooling layer is added to extract the global mean features and generate the mesoscale feature map. For the large-scale branch, its structure also contains three consecutive convolutional layers, each with a kernel size of 7x7, a stride of 1, a padding of 3, and the number of output channels is still 64, 128, and 256. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function to maintain the nonlinear expression ability and training stability of the model. In order to aggregate features in a larger range, a spatial pyramid pooling layer is added to the large-scale branch at the end to form a large-scale feature map through multi-scale spatial information extraction. Based on the small-scale feature map, medium-scale feature map and large-scale feature map, feature fusion is performed, and the channel attention mechanism is used to perform weighted summation on the feature maps of the three scales to fully utilize the importance of each scale feature to obtain a multi-scale fusion feature map. The residual connection technology is applied to the multi-scale fusion feature map, and the result of the input data processed by the 1x1 convolution layer is added to the multi-scale fusion feature map at the element level to generate a residual enhanced feature map. By introducing the residual structure, the expression ability of the feature map and the depth of training are enhanced. Based on the residual enhanced feature map, a feature reconstruction module is constructed, which includes a 1x1 convolution layer for feature dimensionality reduction, followed by a 3x3 separable convolution layer for efficient feature extraction, and the last 1x1 convolution layer for feature dimensionality increase to generate a reconstructed feature map. In this way, the feature reconstruction module can reduce the computational complexity while retaining important information. The self-attention mechanism is applied to the reconstructed feature map. By calculating the correlation between each position in the feature map and all other positions, and weighted summing the features based on the correlation, the attention enhanced feature map is obtained.This process improves the model's feature selection capability, ensuring that only the features that are most critical to model decision-making are retained, and ultimately obtaining the target multi-level local features.

[0015] S3, divide the multi-level local features into two parts, obtain the first part of local features and the second part of local features, and input the second part of local features into the parallel bidirectional long short-term memory network and bidirectional gated recurrent unit network in the target model for feature extraction to obtain global semantic features; Among them, feature segmentation is performed on the multi-level local features, and the feature space is divided into two subspaces to obtain the first part of local features and the second part of local features, so as to process the local information and global semantic information of different features respectively. Based on the second part of local features, a bidirectional long short-term memory network (LSTM) is constructed, which includes a forward LSTM layer and a backward LSTM layer, each of which contains 128 hidden units. The calculation process of the forward LSTM layer includes the input gate, the forget gate, the output gate, and the update of the unit state. The gate control unit uses the Sigmoid activation function for nonlinear transformation, and the update of the unit state uses the tanh activation function to ensure the effective expression of the features and the long-term dependence of the information. Through calculation, the forward LSTM feature sequence is obtained. At the same time, the backward LSTM layer is calculated, and the calculation process is the same as the forward LSTM layer, but because the order of the input sequence is reversed, the time series information different from the forward sequence can be captured to obtain the backward LSTM feature sequence. Based on the local features in the second part, a bidirectional gated recurrent unit network (GRU) is constructed. The network also includes a forward GRU layer and a backward GRU layer, and each layer also contains 128 hidden units. The calculation process of the forward GRU layer involves the calculation of the update gate and the reset gate. The gated unit also uses the Sigmoid activation function, and the candidate hidden state uses the tanh activation function to obtain the forward GRU feature sequence. The gating mechanism effectively controls the information flow, enabling the GRU to handle longer dependencies and reduce the risk of gradient disappearance. The backward GRU layer is calculated. The calculation process is the same as the forward GRU layer, but the input sequence order is reversed, capturing feature information different from the forward GRU layer, and obtaining the backward GRU feature sequence. The forward LSTM feature sequence is concatenated with the backward LSTM feature sequence, and the features are fused through the adaptive weight pooling layer to obtain the LSTM global feature. The adaptive weight pooling layer dynamically adjusts the weight of each feature according to the importance of the feature, and integrates the global features extracted by the LSTM network in the best way. The forward GRU feature sequence is concatenated with the backward GRU feature sequence, and these features are weighted through the attention mechanism to highlight the most important part for the final semantic feature extraction, and the GRU global feature is obtained. The attention mechanism can effectively identify the most informative part of the input feature and give it a higher weight. A feature fusion network is constructed based on the LSTM global feature and the GRU global feature. The network includes a multi-layer perceptron (MLP) and a residual connection module. Through the nonlinear transformation and feature interaction of the multi-layer perceptron, the deep correlation between the LSTM and GRU features is fully explored to improve the richness of feature expression and the generalization ability of the model. At the same time, the residual connection effectively alleviates the gradient vanishing problem that may occur in the deep network by introducing the jump connection of the input feature in the feature fusion process, and finally obtains the global semantic feature.

[0016] The LSTM global features and the GRU global features are concatenated to combine the features from two different networks to form an initial fused feature vector. Based on the initial fused feature vector, a multilayer perceptron (MLP) is constructed. The multilayer perceptron consists of three fully connected layers with 512, 256, and 128 neurons, respectively. The LeakyReLU activation function is used between each layer to handle the nonlinear relationship in the features. At the same time, the BatchNorm normalization layer is applied to accelerate model convergence and reduce overfitting to obtain the output features of the multilayer perceptron. The initial fused feature vector is reduced in dimension. Through the 1x1 convolution operation, the dimension of the initial fused feature vector is adjusted to the same size as the output feature of the multilayer perceptron to generate residual features. The dimensionality reduction process not only retains the important information of the initial features, but also reduces the computational burden, making the subsequent feature fusion more efficient. Feature fusion is performed based on the output features of the multilayer perceptron and the residual features, and the fused feature representation is obtained through element-wise addition. The new feature information obtained by nonlinear transformation in the multi-layer perceptron is retained, and the structure of the original features is retained by residual connection, so that the fused feature representation is more robust and diverse. The Self-Attention mechanism is applied to the fused feature representation, and the features that are most important to the final task are identified and highlighted by calculating the attention weights within the features. The Self-Attention mechanism can consider the dependencies between features globally, and reweight the features according to the attention weights to obtain the feature representation with enhanced attention. Based on the feature representation with enhanced attention, nonlinear transformation is performed, and the features are mapped through the Swish activation function to obtain the final global semantic features. The Swish activation function effectively enhances the expressive power of the model with its smooth nonlinear characteristics, and often performs better than traditional activation functions (such as ReLU) in complex tasks, and finally obtains the global semantic features.

[0017] S4, concatenates the local features of the first part with the global semantic features, and weights them through a multi-head attention mechanism to obtain a fused feature representation; Among them, the local features and global semantic features of the first part are dimensionally aligned. They are projected into the same feature space by linear projection to obtain the aligned local features and global features. Based on the aligned features, a feature interaction matrix is ​​constructed. The matrix quantifies the relationship between local features and global features by calculating the dot product similarity between each pair of features. The similarity values ​​are normalized by applying the Softmax function to generate an attention weight matrix, which reflects the relative importance of each feature when interacting with other features. The generated attention weight matrix is ​​multi-head split and evenly divided into 8 sub-matrices, each of which corresponds to an attention head. In this way, multiple independent multi-head attention weights are obtained. Based on the multi-head attention weights, a weighted sum operation is performed on the aligned features. It is performed independently on each attention head to obtain 8 head-specific context vectors. The context vector represents the specific perspective of different attention heads in the feature space, which helps to capture richer feature representations. The 8 head-specific context vectors are concatenated and dimensionality reduced by a linear transformation layer to reduce the dimension of the features and improve the computational efficiency of the model. The multi-head attention output is obtained through linear transformation. In order to enhance the stability and expressiveness of feature representation, a residual connection and layer normalization structure is constructed based on multi-head attention output. The original input features are added to the multi-head attention output, and the original input information is retained through residual connection while enhancing the deep learning ability of the network. The layer normalization is performed on the added result to improve the convergence speed and overall performance of the model by standardizing the features, while also reducing the training fluctuation of the model. Through the multi-head attention mechanism, the deep interactive relationship between the local features of the first part and the global semantic features is fully explored, and a fused feature representation is generated.

[0018] S5, based on the fusion feature representation, optimizes the model parameters of the target model through the batch gradient descent method with adaptive learning rate to obtain the trained artificial intelligence model.

[0019] That is, batch sampling is performed on the fused feature representation. By using a fixed-size sliding window, a subset of samples is randomly selected from the training data to form the training data of the current batch, thereby effectively reducing the computational burden during the training process while maintaining the representativeness of the model for the entire data set. Based on the current batch of training data, the forward propagation process of the target model is calculated. In this process, the training data is passed through each layer of the model's network structure in sequence, and the intermediate activation values ​​are calculated and stored to obtain the output of the model, so that the model can map the input data to the prediction result. According to the difference between the model output and the true label, the loss function value is calculated. The cross entropy loss function is used, which is suitable for classification tasks and can measure the distance between the probability distribution of the model output and the actual label distribution. By calculating the cross entropy loss function, the loss value of the current batch is obtained, which reflects the performance of the model on the current training data. Back propagation is performed based on the loss value of the current batch. By using automatic differentiation technology, the gradient of the loss function to the parameters of each layer of the model is automatically calculated to obtain the parameter gradient matrix. The matrix contains information on how to adjust each parameter to reduce the loss value. In order to prevent the occurrence of gradient explosion, the gradient clipping technology is applied to the parameter gradient matrix. In this process, a threshold of the gradient norm is set, and the gradients exceeding the threshold are scaled to obtain the clipped gradient matrix, ensuring that the model training process is more stable and avoiding training failure caused by excessive gradient values. The adaptive learning rate is calculated based on the clipped gradient matrix, and the Adam optimizer is used. The Adam optimizer can dynamically adjust the learning rate of each parameter by combining the first-order moment estimation and the second-order moment estimation, so that different parameters can be updated with the most appropriate step size according to their gradient changes. The adaptive learning rate adjustment method not only speeds up the convergence speed of the model, but also improves the efficiency and effect of training. According to the calculated parameter update step size, the parameters of the target model are updated. In the update process, the weight decay technology is applied to prevent the model parameters from being too large by introducing a regularization term in the update process, thereby effectively preventing the occurrence of overfitting. As the model parameters are continuously updated, the model gradually learns a better feature representation. The validation set is evaluated based on the updated model parameters. By calculating the performance indicators on the validation set, the performance of the model on unseen data is objectively evaluated. In order to avoid the model from stopping training too early or overtraining, an early stopping strategy is adopted. The early stopping strategy monitors the changes in performance indicators on the validation set. When the performance of the model no longer improves significantly, it can be judged that the training process has reached the convergence condition, and the training is stopped immediately to obtain the final trained artificial intelligence model.

[0020] In one example, the input raw data is preprocessed to obtain a preprocessed data set, Laplace noise is added to the preprocessed data set, and data standardization and normalization are performed at the same time to obtain a privacy-preserving data set, including: Perform data cleaning on the input raw data to obtain a cleaned data set, and perform feature extraction based on the cleaned data set to obtain a preliminary feature data set; Encode the preliminary feature data set, convert the categorical variables into numerical variables, and obtain the encoded feature data set; Based on the NN-FC framework, the Laplace noise parameter ε that satisfies differential privacy is calculated. The calculation formula of the Laplace noise parameter ε is: ε=ln(1+σ / Δf), σ is the sensitivity, and Δf is the global sensitivity of the function; According to the Laplace noise parameter ε, a Laplace distributed random noise is generated for the encoded feature data set to obtain a noise matrix; Perform element-wise addition operation on the noise matrix and the encoded feature data set to obtain the data set after adding noise; Performing data standardization processing on the data set after adding noise to obtain a standardized data set, and performing normalization processing based on the standardized data set to obtain a normalized data set; The normalized data set is subjected to data balancing to obtain a balanced data set, and feature selection is performed based on the balanced data set to obtain a privacy-preserving data set.

[0021] In this example, the input raw data is cleaned to remove noise, process missing values, and correct outliers in the data to ensure the integrity, accuracy, and consistency of the data, and a cleaned data set is obtained. Feature extraction is performed. Representative information is extracted from the cleaned data set so that subsequent models can better use this information for learning, and a preliminary feature data set is obtained. Data encoding is performed on the preliminary feature data set. Categorical variables (such as gender, region, occupation, etc.) are converted into numerical variables, which are accepted by most machine learning models. Common encoding methods include one-hot encoding and label encoding. For example, when processing the categorical variable of user gender, "male" is encoded as 0, "female" is encoded as 1, or it is converted into two binary variables using one-hot encoding. In this process, categorical variables are converted into numerical variables to form an encoded feature data set. Based on the NN-FC framework, the Laplace noise parameter ε that satisfies differential privacy is calculated. Differential privacy ensures that the information of individuals in the data will not be leaked even in some cases by adding random noise to the data. Calculation The formula is: ε=ln(1+σ / Δf), where is a parameter that controls the degree of privacy protection. Represents the sensitivity of the data, indicating the degree of influence of a certain feature on the result, and is the global sensitivity of the function, which represents the maximum impact that changing a data point may have on the overall result. By measuring the global sensitivity and data sensitivity, a suitable Laplace noise parameter is calculated to maintain the validity of the data as much as possible while protecting privacy. Parameters, generate Laplace distributed random noise for the encoded feature data set. This noise is randomly generated through Laplace distribution to form a noise matrix. By adding noise, sensitive information in the data is masked, protecting the privacy of the data, but the intensity of the noise should be appropriate and should not excessively affect the overall structure and validity of the data. The generated noise matrix is ​​added to the encoded feature data set at the element level to obtain the data set after adding noise. In this process, noise will introduce a certain degree of randomness in each element of the data, thereby achieving privacy protection. For example, if the user is 30 years old, the age after adding noise may become 31 or 29 years old. Perform data standardization on the data set after adding noise. Scale the data to a uniform scale to eliminate the dimensional differences between different features and obtain a standardized data set. Normalize the data. Scale the data to a specific range, usually between 0 and 1. Through normalization, ensure that all features are in the same numerical range, further reduce the deviation between features, and obtain a normalized data set. Perform data balancing on the normalized data set. Data balancing can be achieved by undersampling or oversampling. For example, if the number of positive samples is much smaller than that of negative samples in a binary classification problem, the dataset may be balanced by copying positive samples or synthesizing new positive samples. After data balancing, the resulting balanced dataset has a more even distribution of categories, which helps the model learn the features of each category fairly during training. Feature selection is performed based on the balanced dataset. The most representative and predictive features are selected from a large number of features to simplify the model and improve its generalization ability. Feature selection can be achieved through statistical methods (such as analysis of variance), model-based methods (such as importance analysis of decision trees), or embedded methods (such as Lasso regression). After feature selection, a privacy-preserving dataset is finally obtained.

[0022] In one example, a multi-scale convolutional neural network in parallel in the target model is used to extract features from a privacy-preserving dataset to obtain multi-level local features of the target, including: Perform data segmentation on the privacy-preserving dataset and divide it into a training set, a validation set, and a test set to obtain a dataset for model training; A multi-scale convolutional neural network is constructed based on the data set used for model training. The multi-scale convolutional neural network includes three parallel convolution branches, namely a small-scale branch, a medium-scale branch, and a large-scale branch. The small-scale branch is constructed, including three consecutive convolutional layers. The convolution kernel size of each convolutional layer is 3x3, the stride is 1, the padding is 1, and the number of output channels is 64, 128, and 256 respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a global maximum pooling layer is added to obtain a small-scale feature map. The mid-scale branch is constructed, including three consecutive convolutional layers. The convolution kernel size of each convolutional layer is 5x5, the stride is 1, the padding is 2, and the number of output channels is 64, 128, and 256 respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a global average pooling layer is added to obtain the mid-scale feature map. The large-scale branch is constructed, including three consecutive convolutional layers. The convolution kernel size of each convolutional layer is 7x7, the stride is 1, the padding is 3, and the number of output channels is 64, 128, and 256 respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a spatial pyramid pooling layer is added to obtain a large-scale feature map. Based on the small-scale feature map, medium-scale feature map and large-scale feature map, feature fusion is performed, and the channel attention mechanism is used to perform weighted summation on the feature maps of the three scales to obtain a multi-scale fused feature map; Apply residual connection to the multi-scale fusion feature map, perform element-level addition operation on the result of the input data processed by the 1x1 convolution layer and the multi-scale fusion feature map to obtain the residual enhanced feature map; A feature reconstruction module is constructed based on the residual enhanced feature map, including a 1x1 convolution layer for dimensionality reduction, a 3x3 separable convolution layer for feature extraction, and a 1x1 convolution layer for dimensionality increase to obtain the reconstructed feature map; The self-attention mechanism is applied to the reconstructed feature map, the correlation between each position in the feature map and all other positions is calculated, and the features are weighted summed based on the correlation to obtain the attention-enhanced feature map. Feature selection is performed based on the attention-enhanced feature map to obtain the target multi-level local features.

[0023] In this example, the privacy-preserving dataset is split. The entire privacy-preserving dataset is divided into three parts: training set, validation set, and test set, to ensure that the model can evaluate its performance in the training, validation, and testing stages respectively. The split usually follows a certain ratio, such as 70% of the data is used for training, 15% of the data is used for validation, and the remaining 15% of the data is used for testing. In this way, the training set is used to learn model parameters, the validation set is used to adjust hyperparameters and prevent model overfitting, and the test set is used to evaluate the generalization ability of the model. A multi-scale convolutional neural network is constructed based on the training dataset to extract features from different spatial scales and capture multi-level information in the data. The network structure consists of three parallel convolution branches, namely small-scale branch, medium-scale branch, and large-scale branch. These three branches correspond to different receptive fields to capture features from details to the global. The small-scale branch is constructed. The small-scale branch is designed to extract fine-grained features, which contains three consecutive convolutional layers. The convolution kernel size of each convolutional layer is set to 3x3, the stride is 1, and the padding is 1. The convolution kernel size is small, which enables this branch to capture the details in the input data. The number of output channels is 64, 128, and 256, respectively. The increasing number of output channels layer by layer enables the network to gradually extract richer features. After each convolution layer, a batch normalization layer and a ReLU activation function are connected. The batch normalization layer is used to standardize the output of each batch to accelerate training and stabilize the model, while the ReLU activation function introduces nonlinear factors to enable the model to learn more complex patterns. A global maximum pooling layer is added, which performs feature dimensionality reduction by selecting the maximum value in the feature map to obtain a small-scale feature map. Similar to the small-scale branch, the structure of the medium-scale branch contains three consecutive convolution layers, but the convolution kernel size is increased to 5x5, the stride is also 1, and the padding is 2. The increase in the convolution kernel size is to expand the receptive field, so that it can capture a wider range of features than the small-scale branch. Similarly, the number of output channels of each convolution layer is 64, 128, and 256, respectively, and a batch normalization layer and a ReLU activation function are connected after each convolution layer. At the end of the medium-scale branch, a global average pooling layer is added, which performs dimensionality reduction by calculating the average value of the feature map, and finally generates a medium-scale feature map. The design of the large-scale branch is similar to the previous two branches, but its convolution kernel size is further increased to 7x7, and the padding is set to 3. Since the convolution kernel is larger, the receptive field is also correspondingly larger, allowing this branch to extract global information from the data. The number of output channels of each convolution layer is still 64, 128, and 256, and the layers are still processed by batch normalization and ReLU activation. In order to enhance the expressive power of large-scale features, a spatial pyramid pooling layer is added at the end of the large-scale branch, and a fixed-length feature vector is generated through pooling operations of different scales, and finally a large-scale feature map is formed. The small-scale, medium-scale, and large-scale feature maps are feature fused.In order to effectively integrate features from different scales, the channel attention mechanism is used to perform weighted summation on the feature maps of the three scales. The channel attention mechanism can automatically learn the importance of each channel and perform weighted summation on the features according to their importance to obtain a multi-scale fusion feature map. This process can be expressed as formula: ; in Represents the output of the cth channel of the final multi-scale fusion feature map, is the feature map of the i-th scale, is the weight learned by the channel attention mechanism, which is used to indicate the importance of the i-th channel. Residual connection is applied to the multi-scale fusion feature map, and the result of the input data processed by the 1x1 convolution layer is added to the multi-scale fusion feature map at the element level to obtain the residual enhanced feature map. 1x1 convolution is used to adjust the dimension of the feature to match the fusion feature, while retaining the information of the original input data. Through residual connection, the model can better avoid the gradient vanishing problem in the deep network and make the feature representation more robust. Based on the residual enhanced feature map, a feature reconstruction module is constructed. This module consists of a 1x1 convolution layer, a 3x3 separable convolution layer and a 1x1 convolution layer. Dimensionality reduction is performed through the 1x1 convolution layer to reduce the dimension of the feature to reduce the computational complexity. The 3x3 separable convolution layer is used for feature extraction to fully explore the information in the residual enhanced feature map. Then the 1x1 convolution layer is used to increase the dimension to restore the feature to the original dimension, and finally the reconstructed feature map is obtained. The self-attention mechanism is applied to the reconstructed feature map to capture long-range dependencies by calculating the correlation between each position in the feature map and all other positions. The self-attention mechanism weights the information in the feature map through the correlation matrix to strengthen the expression of important features and weaken irrelevant information. The dot product similarity between each feature point is calculated and normalized by the Softmax function to obtain the attention weight matrix, and then the features are weighted and summed according to these weights to finally obtain the attention-enhanced feature map. By performing feature selection on the attention-enhanced feature map, redundant and irrelevant features are eliminated, and finally the target multi-level local features are obtained.

[0024] In one example, the multi-level local features are divided into two parts to obtain the first part of local features and the second part of local features, and the second part of local features are input into the parallel bidirectional long short-term memory network and the bidirectional gated recurrent unit network in the target model for feature extraction to obtain global semantic features, including: Perform feature segmentation on the multi-level local features, divide the feature space into two subspaces, and obtain the first part of local features and the second part of local features; A bidirectional long short-term memory network is constructed based on the local features of the second part. The bidirectional long short-term memory network includes a forward LSTM layer and a backward LSTM layer, each layer contains 128 hidden units. The forward LSTM layer of the bidirectional long short-term memory network is calculated, including the update of the input gate, forget gate, output gate and unit state. The gate unit uses the Sigmoid activation function and the unit state update uses the tanh activation function to obtain the forward LSTM feature sequence. The backward LSTM layer of the bidirectional long short-term memory network is calculated. The calculation process is the same as the forward LSTM layer, but the order of the input sequence is reversed to obtain the backward LSTM feature sequence; A bidirectional gated recurrent unit network is constructed based on the local features of the second part. The bidirectional gated recurrent unit network includes a forward GRU layer and a backward GRU layer, and each layer contains 128 hidden units. Calculate the forward GRU layer of the bidirectional gated recurrent unit network, including the calculation of the update gate and the reset gate, where the gated unit uses the sigmoid activation function and the candidate hidden state uses the tanh activation function to obtain the forward GRU feature sequence; The backward GRU layer of the bidirectional gated recurrent unit network is calculated in the same way as the forward GRU layer, but the order of the input sequence is reversed to obtain the backward GRU feature sequence; The forward LSTM feature sequence and the backward LSTM feature sequence are concatenated, and the features are fused through the adaptive weight pooling layer to obtain the LSTM global feature; The forward GRU feature sequence and the backward GRU feature sequence are concatenated, and the features are weighted through the attention mechanism to obtain the GRU global feature; A feature fusion network is constructed based on LSTM global features and GRU global features, including a multi-layer perceptron and a residual connection. Global semantic features are obtained through feature interaction and nonlinear transformation.

[0025] In this example, feature segmentation is performed on multi-level local features, and the high-dimensional feature space is divided into two subspaces to generate the first part of local features and the second part of local features. Through segmentation, different network structures are used for processing different subspaces to better extract information from the features. Based on the second part of the local features after segmentation, a bidirectional long short-term memory network (Bi-LSTM) is constructed, which includes a forward LSTM layer and a backward LSTM layer, each containing 128 hidden units. LSTM (Long Short-Term Memory Network) is a special recurrent neural network (RNN) that is good at processing and storing long-term dependent information. Bidirectional LSTM can simultaneously consider information in both the forward and backward directions in the sequence, thereby more comprehensively capturing the temporal relationship of the input features. For the forward LSTM layer, the calculation process includes four main parts: input gate, forget gate, output gate, and update of the unit state. The input gate determines how much information of the current input can pass; the forget gate controls how much information of the previous time step is retained; and the output gate determines the output of the current time step. All these gated units use the Sigmoid activation function, which can compress the input value to the range of 0 to 1, thus achieving effective control of the information flow. The unit state update uses the tanh activation function to compress the input value to the range of -1 to 1, making the state change smooth and controllable. After calculation, the forward LSTM feature sequence is obtained, which contains the time sequence information from left to right. At the same time, the calculation process of the backward LSTM layer is the same as that of the forward LSTM layer, but the order of the input sequence is reversed. The backward LSTM can capture the time sequence information from right to left and obtain the backward LSTM feature sequence. While constructing the bidirectional long short-term memory network, a bidirectional gated recurrent unit network (Bi-GRU) is constructed based on the local features of the second part. GRU (gated recurrent unit) is a simplified version of LSTM with similar advantages, but with a simpler structure and lower computational cost. The bidirectional GRU also includes a forward GRU layer and a backward GRU layer, each containing 128 hidden units. For the forward GRU layer, the calculation process mainly involves two gating units: the update gate and the reset gate. The update gate determines the proportion of information from the previous moment retained at the current moment, while the reset gate determines how much of the current input will be used to reset the information from the previous moment. Like LSTM, the gating unit of GRU uses the Sigmoid activation function, while the update of the candidate hidden state uses the tanh activation function. After calculation, the forward GRU feature sequence is obtained. The calculation process of the backward GRU layer is the same as that of the forward GRU layer, but the order of the input sequence is reversed to obtain the backward GRU feature sequence. The forward and backward LSTM feature sequences are concatenated to combine the output of the bidirectional LSTM to form a comprehensive feature representation containing contextual information.The feature fusion is performed through an adaptive weight pooling layer, which dynamically adjusts the weight of each feature according to its importance to obtain the LSTM global feature. Similarly, the forward GRU feature sequence and the backward GRU feature sequence are concatenated, and the features are weighted through the attention mechanism to obtain the GRU global feature. The attention mechanism can automatically identify which parts of the input sequence are most important to the current task and give these parts higher weights. This mechanism improves the model's ability to focus on key features by calculating the attention weights of the features. A feature fusion network is constructed based on the LSTM global features and the GRU global features. The network includes a multi-layer perceptron (MLP) and a residual connection. The multi-layer perceptron realizes nonlinear transformation of features through multiple fully connected layers to mine the complex interactive relationships between features. The residual connection alleviates the gradient vanishing problem in the deep network by directly adding the input features to the features processed by the multi-layer perceptron, while enhancing the expressiveness of the model to obtain global semantic features.

[0026] In one example, a feature fusion network is constructed based on LSTM global features and GRU global features, including a multi-layer perceptron and a residual connection. Through feature interaction and nonlinear transformation, global semantic features are obtained, including: Concatenate the LSTM global features and the GRU global features to obtain the initial fused feature vector; A multilayer perceptron is constructed based on the initial fusion feature vector. The multilayer perceptron contains three fully connected layers, with 512, 256, and 128 neurons in each layer. The LeakyReLU activation function and BatchNorm normalization layer are used between each layer to obtain the output features of the multilayer perceptron. Perform dimensionality reduction processing on the initial fused feature vector, adjust the feature dimension to the same as the output feature of the multi-layer perceptron through a 1x1 convolution operation, and obtain the residual feature; Feature fusion is performed based on the output features of the multi-layer perceptron and the residual features, and the fused feature representation is obtained through element-level addition operation; Apply the Self-Attention mechanism to the fused feature representation, calculate the attention weight within the feature, and reweight the feature according to the attention weight to obtain the attention-enhanced feature representation; Based on the attention-enhanced feature representation, nonlinear transformation is performed and feature mapping is performed through the Swish activation function to obtain global semantic features.

[0027] In this example, In one example, the first part of local features is concatenated with the global semantic features and weighted by a multi-head attention mechanism to obtain a fused feature representation, including: Align the dimensions of the local features and global semantic features in the first part, project them into the same feature space through linear projection, and obtain the aligned local features and global features; The feature interaction matrix is ​​constructed based on the aligned local features and global features. The attention weight matrix is ​​obtained by calculating the dot product similarity between each pair of features and applying the Softmax function for normalization. Perform multi-head segmentation on the attention weight matrix and evenly divide it into 8 sub-matrices, each sub-matrix corresponds to an attention head, and obtain the multi-head attention weight; We perform weighted summation of the aligned features based on the multi-head attention weights, and perform this operation independently for each attention head to obtain 8 head-specific context vectors; The 8 head-specific context vectors are concatenated and reduced in dimension through a linear transformation layer to obtain the multi-head attention output; Based on the multi-head attention output, a residual connection and layer normalization structure are constructed, the original input features are added to the multi-head attention output, and then layer normalization is performed to obtain a fused feature representation.

[0028] In this example, the first part of local features and global semantic features are dimensionally aligned. Local features may contain fine-grained information of the input data, while global semantic features reflect broader contextual information. They are projected into the same feature space through linear projection, which can be expressed as a matrix multiplication plus a bias operation. Let the first part of local features be , the global semantic feature is , the alignment feature after linear projection is and . This process can be expressed as the following two formulas: ; ; in, and are the weight matrices of linear projection, and is the bias vector. Through linear transformation, local features and global features are mapped to feature spaces of the same dimension to ensure that they can interact effectively in the same space. The feature interaction matrix is ​​constructed based on the aligned local features and global features. The feature interaction matrix is ​​used to represent the relationship between local features and global features. Each element in the matrix is ​​obtained by calculating the dot product of the corresponding local feature and the global feature: ; in, Indicates The local features and The similarity between the global features, the dot product operation can capture the similarity between two feature vectors, the larger the value, the more similar the features are. After obtaining the feature interaction matrix, use the Softmax function to convert it into an attention weight matrix ; In this formula, represents an exponential function. The purpose of Softmax normalization is to convert the similarity into a probability distribution so that the mutual influence between each pair of local and global features can be expressed by a probability value. Perform multi-head segmentation and divide it evenly into 8 sub-matrices, each sub-matrix corresponds to an attention head ,in Represents different attention heads. The multi-head attention mechanism allows the model to process feature relationships independently in different subspaces, capturing the diversity and complexity of features. Based on the weight of each attention head , for the global features after alignment Perform weighted summation to generate a specific context vector For each attention head, the formula for calculating the context vector is: ; in, Indicates The attention head is at position Through this weighted summation operation, 8 context vectors are generated independently on each attention head. . The eight head-specific context vectors are concatenated to form a high-dimensional feature vector. The high-dimensional feature vector contains information from all attention heads, indicating the diversity of global features captured by different attention heads. In order to convert the high-dimensional vector into a more tractable form, a linear transformation layer is used to reduce its dimensionality to obtain the final multi-head attention output. ; in, is the reduced-dimensional weight matrix, is the bias term. Through linear transformation, the concatenated high-dimensional features are compressed to a dimension suitable for model processing while retaining important context information. Based on multi-head attention output Construct residual connection and layer normalization structure. With multi-head attention output Add together to form a residual connection output ; The purpose of residual connection is to retain the original input information in the deep structure of the network, while superimposing the results of multi-head attention output to improve the learning ability and stability of the model. This structure helps the model to effectively transmit information in the deep network and alleviates the gradient vanishing problem. In order to stabilize the output and ensure that features at different levels are processed at the same scale, layer normalization is applied to the residual output. Perform standardization to obtain the final fusion feature representation ; The role of layer normalization is to make the output of each layer have the same mean and variance, thereby speeding up training and reducing the possibility of overfitting. The final fusion feature representation combines the complex relationship between local features and global features, and captures the diverse information through the multi-head attention mechanism. After residual connection and layer normalization, the model performs more stable and efficient when processing complex tasks.

[0029] In one example, based on the fusion feature representation, the model parameters of the target model are optimized by the batch gradient descent method with adaptive learning rate to obtain a trained artificial intelligence model, including: Batch sampling is performed on the fused feature representation, and a fixed-size sliding window is used to randomly select a sample subset from the training data to obtain the current batch of training data; Calculate the forward propagation of the target model based on the current batch of training data, calculate and store the intermediate activation values ​​in turn through each layer of the target model network structure, and obtain the model output; Calculate the loss function value based on the model output and the true label, and use the cross entropy loss function to calculate and get the loss value of the current batch; Back propagates based on the loss value of the current batch, uses automatic differentiation technology to calculate the gradient of the loss function to the parameters of each layer, and obtains the parameter gradient matrix; Apply gradient clipping technology to the parameter gradient matrix, set the gradient norm threshold, scale the gradients exceeding the threshold, and obtain the clipped gradient matrix; The adaptive learning rate is calculated based on the clipped gradient matrix, and the Adam optimizer is used to dynamically adjust the learning rate of each parameter according to the first-order moment estimation and the second-order moment estimation to obtain the parameter update step size; Update the parameters of the target model according to the parameter update step size, and apply weight decay technology to prevent overfitting to obtain updated model parameters; The validation set is evaluated based on the updated model parameters, the performance indicators on the validation set are calculated, and the early stopping strategy is used to determine whether the convergence conditions have been met. If so, the training is stopped to obtain a trained artificial intelligence model.

[0030] In this example, batch sampling is performed on the fused feature representation. In order to efficiently utilize data and prevent overfitting, a mini-batch stochastic gradient descent method is usually used. A fixed-size sliding window is used to randomly select a subset of samples from the training data to form the current batch of training data. The sliding window slides over the training data and randomly selects data points to ensure that each batch of data samples can effectively represent the entire data set. Assume that the training data set is ,in represents the input features, Indicates the corresponding true label. In each training step, a batch is selected from ,in is the batch size. Based on the current batch of training data , calculate the forward propagation of the target model. The forward propagation process is to pass the input data through each layer of the model's network structure in sequence and calculate the intermediate activation value at each layer. Assume that the model contains Layer, the parameters of each layer are (in ), the output of the model can be expressed as ,in is the mapping function of the entire network, is the input data, is the set of parameters of all layers. Through forward propagation, the predicted output of the current batch of data is obtained . Model-based prediction output Calculate the loss function value. The loss function is used to measure the gap between the model prediction result and the true label. In classification tasks, the commonly used loss function is the cross entropy loss function. Assume that the true label of the current batch is , cross entropy loss function The definition is as follows: ; in, is the batch size, is the number of categories, Representation sample In category The real label, Indicates the category predicted by the model The probability value of . By calculating the cross entropy loss, the loss value of the current batch is obtained. Back propagation is performed based on the loss value of the current batch. The core of back propagation is to calculate the gradient of the loss function to the parameters of each layer, that is, This process is usually implemented using automatic differentiation technology. Automatic differentiation can efficiently calculate the partial derivatives of model parameters with respect to loss and obtain the parameter gradient matrix. These gradients will guide the model on how to adjust the parameters to reduce loss. In order to prevent the gradient explosion problem, gradient clipping technology is applied to the parameter gradient matrix. Gradient explosion will cause the model parameters to update too much, making the model training unstable. By setting the gradient norm threshold , the gradient exceeding the threshold can be scaled. If the norm of the gradient Exceeding the threshold , then scale it to The size of the clipped gradient matrix is ​​obtained . Based on the clipped gradient matrix Calculate the adaptive learning rate. Use the Adam optimizer to dynamically adjust the learning rate of each parameter. The Adam optimizer combines the first-order moment estimate and the second-order moment estimate so that the learning rate is adjusted according to the gradient history of each parameter. The update rule of the Adam optimizer is as follows: ; ; ; in, and are the first-order and second-order moment estimates, and is the momentum parameter, is the learning rate, is a small constant used to prevent division by zero errors. Through these steps, we get the update step size of the parameters , and update the model parameters accordingly. At the same time, in order to prevent the model from overfitting, the weight decay technology is applied, that is, when the parameters are updated, a penalty term proportional to the parameter value is added. Weight decay prevents the model from performing too well on the training data and performing poorly on the test data by gradually reducing the model parameters. The update rule for this step can be expressed as: ; in, is the weight decay coefficient. The validation set is evaluated based on the updated parameters. The validation set is used to evaluate the performance of the model on unseen data to ensure that the model has good generalization ability. In the validation phase, performance indicators such as accuracy, precision, recall or F1 score are calculated on the validation set. These indicators help determine whether the model has achieved the desired performance level. In order to prevent the model from overfitting or training time being too long, an early stopping strategy is applied. The early stopping strategy monitors the changes in performance indicators on the validation set. When the performance indicators no longer improve within several training cycles, training is stopped in advance to prevent the model from overfitting on the training data and ensure the best performance of the model. When the early stopping strategy is triggered or the preset training cycle is reached, the training process ends, and the trained artificial intelligence model is finally obtained. After multiple parameter updates and verifications, this model has the ability to make predictions on new data, and its parameters have been optimized to a state close to the global optimal state.

[0031] Reference Figure 2 , this embodiment provides an artificial intelligence model building device, including: Preprocessing module 1 is used to preprocess the input raw data to obtain a preprocessed data set, add Laplace noise to the preprocessed data set, and perform data standardization and normalization processing to obtain a privacy-preserving data set; The first extraction module 2 is used to extract features of the privacy protection data set through a multi-scale convolutional neural network in parallel in the target model to obtain multi-level local features of the target; The second extraction module 3 is used to divide the multi-level local features into two parts to obtain the first part of local features and the second part of local features, and input the second part of local features into the parallel bidirectional long short-term memory network and the bidirectional gated recurrent unit network in the target model for feature extraction to obtain the global semantic features; A concatenation module 4 is used to concatenate the first part of local features with the global semantic features, and weight them through a multi-head attention mechanism to obtain a fused feature representation; The optimization module 5 is used to optimize the model parameters of the target model based on the fusion feature representation through the batch gradient descent method with adaptive learning rate to obtain a trained artificial intelligence model.

[0032] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0033] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0034] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0035] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0036] In summary, an artificial intelligence model building method is proposed in an embodiment of the present invention. By adding Laplace noise to the preprocessed data set and combining data standardization and normalization processing, the privacy of the original data is effectively protected, while retaining the statistical characteristics of the data. A parallel multi-scale convolutional neural network is adopted, including convolution branches of small, medium and large scales, which can fully capture the multi-scale local features in the data and improve the model's recognition ability of different scale patterns. Through the parallel bidirectional long short-term memory network and the bidirectional gated recurrent unit network, the long-term and short-term dependencies in the data are effectively captured, and the model's ability to understand sequence data is enhanced. The multi-head attention mechanism is used to perform weighted fusion of local features and global semantic features, realize the organic combination of different types of features, and improve the richness and expression of feature representation. The batch gradient descent method with adaptive learning rate is adopted, combined with the Adam optimizer and gradient clipping technology, to achieve efficient optimization of model parameters and improve the training speed and convergence stability of the model. By introducing residual connection, layer normalization and other technologies, and applying weight decay and early stopping strategies, the overfitting problem is effectively alleviated and the generalization ability of the model is improved. The design of the multi-head attention mechanism and feature fusion network provides a more intuitive explanation for model decisions and enhances the interpretability of the model.

[0037] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0038] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0039] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for building an artificial intelligence model, characterized in that: The following steps are involved: Preprocessing the input raw data to obtain a preprocessed dataset, adding Laplace noise to the preprocessed dataset, and performing data standardization and normalization processing to obtain a privacy-preserving dataset; Performing feature extraction on the privacy-preserving dataset through a multi-scale convolutional neural network connected in parallel in the target model to obtain target multi-level local features; Dividing the multi-level local features into two parts to obtain a first part of local features and a second part of local features, and inputting the second part of local features into a bidirectional long short-term memory network and a bidirectional gated recurrent unit network connected in parallel in the target model for feature extraction to obtain a global semantic feature; The first part of local features and the global semantic features are concatenated and weighted through a multi-head attention mechanism to obtain a fused feature representation; Based on the fusion feature representation, the model parameters of the target model are optimized by the batch gradient descent method with adaptive learning rate to obtain a trained artificial intelligence model.

2. The artificial intelligence model building method according to claim 1, characterized in that: The input raw data is preprocessed to obtain a preprocessed data set, Laplace noise is added to the preprocessed data set, and data standardization and normalization are performed to obtain a privacy-preserving data set, including: Performing data cleaning on the input raw data to obtain a cleaned data set, and performing feature extraction based on the cleaned data set to obtain a preliminary feature data set; Performing data encoding on the preliminary feature data set, converting categorical variables into numerical variables, and obtaining an encoded feature data set; Based on the NN-FC framework, the Laplace noise parameter ε that satisfies differential privacy is calculated. The calculation formula of the Laplace noise parameter ε is: ε=ln(1+σ / Δf), where σ is the sensitivity and Δf is the global sensitivity of the function. Generating Laplace distributed random noise for the encoded feature data set according to the Laplace noise parameter ε to obtain a noise matrix; Performing an element-wise addition operation on the noise matrix and the encoded feature data set to obtain a data set after adding noise; performing data standardization processing on the noise-added data set to obtain a standardized data set, and performing normalization processing based on the standardized data set to obtain a normalized data set; Data balancing is performed on the normalized dataset to obtain a balanced dataset, and feature selection is performed based on the balanced dataset to obtain a privacy-preserving dataset.

3. The artificial intelligence model building method according to claim 1, characterized in that: The multi-scale convolutional neural network connected in parallel in the target model is used to extract features from the privacy-preserving dataset to obtain target multi-level local features, including: Performing data segmentation on the privacy-preserving dataset into a training set, a validation set, and a test set to obtain a dataset for model training; Constructing a multi-scale convolutional neural network based on the model training data set, wherein the multi-scale convolutional neural network includes three parallel convolution branches, namely a small-scale branch, a medium-scale branch, and a large-scale branch; The small-scale branch is constructed, including three consecutive convolutional layers. The convolution kernel size of each convolutional layer is 3x3, the stride is 1, the padding is 1, and the number of output channels is 64, 128, and 256 respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a global maximum pooling layer is added to obtain a small-scale feature map; The mid-scale branch is constructed by three consecutive convolutional layers. Each convolutional layer has a convolution kernel size of 5x5, a stride of 1, a padding of 2, and output channels of 64, 128, and 256, respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a global average pooling layer is added to obtain the mid-scale feature map. The large-scale branch is constructed, including three consecutive convolutional layers. The convolution kernel size of each convolutional layer is 7x7, the stride is 1, the padding is 3, and the number of output channels is 64, 128, and 256 respectively. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and finally a spatial pyramid pooling layer is added to obtain a large-scale feature map. Perform feature fusion based on the small-scale feature map, the medium-scale feature map, and the large-scale feature map, and use a channel attention mechanism to perform weighted summation on the feature maps of the three scales to obtain a multi-scale fused feature map; Applying a residual connection to the multi-scale fusion feature map, performing an element-wise addition operation on the result of the input data processed by the 1x1 convolution layer and the multi-scale fusion feature map to obtain a residual enhanced feature map; Constructing a feature reconstruction module based on the residual enhanced feature map, including a 1x1 convolution layer for dimensionality reduction, a 3x3 separable convolution layer for feature extraction, and a 1x1 convolution layer for dimensionality increase, to obtain a reconstructed feature map; A self-attention mechanism is applied to the reconstructed feature map, the correlation between each position in the feature map and all other positions is calculated, and the features are weighted summed based on the correlation to obtain an attention-enhanced feature map. Feature selection is performed based on the attention-enhanced feature map to obtain target multi-level local features.

4. The artificial intelligence model building method according to claim 1, characterized in that: The multi-level local features are divided into two parts to obtain a first part of local features and a second part of local features, and the second part of local features are input into a parallel bidirectional long short-term memory network and a bidirectional gated recurrent unit network in a target model for feature extraction to obtain a global semantic feature, including: Performing feature segmentation on the multi-level local features, dividing the feature space into two subspaces, and obtaining a first part of local features and a second part of local features; Constructing a bidirectional long short-term memory network based on the local features of the second part, wherein the bidirectional long short-term memory network includes a forward LSTM layer and a backward LSTM layer, each layer including 128 hidden units; Calculating the forward LSTM layer of the bidirectional long short-term memory network, including updating the input gate, forget gate, output gate, and unit state, wherein the gate unit uses a sigmoid activation function and the unit state update uses a tanh activation function to obtain a forward LSTM feature sequence; The backward LSTM layer of the bidirectional long short-term memory network is calculated in the same way as the forward LSTM layer, but the order of the input sequence is reversed to obtain a backward LSTM feature sequence; Constructing a bidirectional gated recurrent unit network based on the second part of local features, wherein the bidirectional gated recurrent unit network includes a forward GRU layer and a backward GRU layer, each layer including 128 hidden units; Calculating the forward GRU layer of the bidirectional gated recurrent unit network, including calculating the update gate and the reset gate, wherein the gated unit uses a sigmoid activation function and the candidate hidden state uses a tanh activation function to obtain a forward GRU feature sequence; Calculating the backward GRU layer of the bidirectional gated recurrent unit network, the calculation process is the same as the forward GRU layer, but the input sequence order is reversed, to obtain a backward GRU feature sequence; The forward LSTM feature sequence and the backward LSTM feature sequence are concatenated, and feature fusion is performed through an adaptive weight pooling layer to obtain LSTM global features; The forward GRU feature sequence and the backward GRU feature sequence are concatenated, and feature weighting is performed through an attention mechanism to obtain a GRU global feature; A feature fusion network is constructed based on the LSTM global features and the GRU global features, including a multi-layer perceptron and a residual connection, and global semantic features are obtained through feature interaction and nonlinear transformation.

5. The artificial intelligence model building method according to claim 4, characterized in that: The feature fusion network is constructed based on the LSTM global features and the GRU global features, including a multi-layer perceptron and a residual connection, and global semantic features are obtained through feature interaction and nonlinear transformation, including: Performing a splicing operation on the LSTM global feature and the GRU global feature to obtain an initial fused feature vector; Constructing a multilayer perceptron based on the initial fusion feature vector, wherein the multilayer perceptron includes three fully connected layers, the number of neurons in each layer is 512, 256, and 128, respectively, and a LeakyReLU activation function and a BatchNorm normalization layer are used between each layer to obtain the output features of the multilayer perceptron; Performing dimensionality reduction processing on the initial fused feature vector, adjusting the feature dimension to be the same as the output feature of the multi-layer perceptron through a 1x1 convolution operation, and obtaining a residual feature; Perform feature fusion based on the multi-layer perceptron output features and the residual features, and obtain the fused feature representation through element-level addition operation; Applying a Self-Attention mechanism to the fused feature representation, calculating the attention weight within the feature, and reweighting the feature according to the attention weight to obtain an attention-enhanced feature representation; A nonlinear transformation is performed based on the attention-enhanced feature representation, and feature mapping is performed through the Swish activation function to obtain global semantic features.

6. The artificial intelligence model building method according to claim 1, characterized in that: The concatenating of the first part of local features and the global semantic features and weighting them through a multi-head attention mechanism to obtain a fused feature representation includes: Dimensionally aligning the first part of local features and the global semantic features, projecting them into the same feature space through linear projection, and obtaining aligned local features and global features; A feature interaction matrix is ​​constructed based on the aligned local features and global features, and an attention weight matrix is ​​obtained by calculating the dot product similarity between each pair of features and applying a Softmax function for normalization; Perform multi-head splitting on the attention weight matrix, evenly dividing it into 8 sub-matrices, each sub-matrix corresponds to an attention head, and obtain the multi-head attention weight; Perform weighted summation of the aligned features based on the multi-head attention weights, and perform this operation independently for each attention head to obtain 8 head-specific context vectors; The eight head-specific context vectors are concatenated and passed through a linear transformation layer for dimensionality reduction to obtain the multi-head attention output; Based on the multi-head attention output, a residual connection and layer normalization structure are constructed, the original input features are added to the multi-head attention output, and then layer normalization is performed to obtain a fused feature representation.

7. The artificial intelligence model building method according to claim 1, characterized in that: The method of optimizing the model parameters of the target model based on the fusion feature representation by a batch gradient descent method with an adaptive learning rate to obtain a trained artificial intelligence model includes: Batch sampling is performed on the fused feature representation, and a sample subset is randomly selected from the training data using a sliding window of a fixed size to obtain the current batch of training data; Calculating the forward propagation of the target model based on the current batch of training data, sequentially calculating and storing intermediate activation values ​​through each layer of the target model network structure to obtain the model output; Calculate the loss function value based on the model output and the true label, and use the cross entropy loss function to calculate the loss value of the current batch; Backpropagation is performed based on the loss value of the current batch, and the gradient of the loss function with respect to the parameters of each layer is calculated using automatic differentiation technology to obtain a parameter gradient matrix; Applying a gradient clipping technique to the parameter gradient matrix, setting a gradient norm threshold, and scaling gradients exceeding the threshold to obtain a clipped gradient matrix; Calculating an adaptive learning rate based on the clipped gradient matrix, and using the Adam optimizer to dynamically adjust the learning rate of each parameter according to the first-order moment estimation and the second-order moment estimation to obtain the parameter update step size; The parameters of the target model are updated according to the parameter update step size, and weight decay technology is applied to prevent overfitting to obtain updated model parameters; A validation set evaluation is performed based on the updated model parameters, the performance indicators on the validation set are calculated, and the early stopping strategy is used to determine whether the convergence conditions are met. If so, the training is stopped to obtain a trained artificial intelligence model.

8. An artificial intelligence model building device, characterized in that: For executing the artificial intelligence model building method according to any one of claims 1 to 7, the device comprises: A preprocessing module is used to preprocess the input raw data to obtain a preprocessed dataset, add Laplace noise to the preprocessed dataset, and perform data standardization and normalization to obtain a privacy-preserving dataset; A first extraction module is configured to extract features from the privacy-preserving dataset using a multi-scale convolutional neural network connected in parallel in a target model to obtain target multi-level local features; A second extraction module is configured to divide the multi-level local features into two parts to obtain a first part of local features and a second part of local features, and input the second part of local features into a bidirectional long short-term memory network and a bidirectional gated recurrent unit network connected in parallel in the target model for feature extraction to obtain a global semantic feature; A splicing module, configured to splice the first part of local features with the global semantic features, and weight them through a multi-head attention mechanism to obtain a fused feature representation; An optimization module is used to optimize the model parameters of the target model based on the fusion feature representation through a batch gradient descent method with an adaptive learning rate to obtain a trained artificial intelligence model.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images