A digital heart-to-heart pavilion information processing method and system

Abstract

The present invention discloses a digital Xinlianxinting information processing method and system, the method includes data collection, CNN layer design, bidirectional GRU design, attention layer design, SCL method design and model determination. The present invention belongs to the field of communication technology, specifically refers to a digital Xinlianxinting information processing method and system, this scheme adopts weighted summation of local regions by weight matrix, nonlinear transformation and unilateral inhibition operation by ReLU function to improve the expression ability and robustness of the model; adopts forward and reverse GRU calculation to capture forward and reverse context information in text data, and captures sentence structure and semantic relationship based on activation function and nonlinear factors; adopts attention mechanism to perform weighted selection of hidden states in input sequence, performs feature extraction, and reduces model dimension and calculation amount by improving projection network and parallel processing.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a digital heart-to-heart pavilion information processing method and system. Background Art

[0002] Xinlianxin Pavilion is a communication and information sharing platform. In order to effectively process the information in the Xinlianxin Pavilion and provide accurate, timely and efficient services, it is necessary to use technologies and methods for processing text information.

[0003] However, conventional text processing methods are unable to capture local feature information, resulting in the model having no local correlation, which in turn affects the model's expressiveness; general text processing methods are unable to involve the context of the context, unable to capture complex sentence structures and semantic relationships, and thus unable to fully understand and encode text data; traditional processing models are unable to capture global context information and cannot adapt to the needs of different tasks and data, resulting in poor model flexibility, and the model dimension is too large, resulting in excessive computational effort, thereby reducing model performance. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a digital Xinlianxinting information processing method and system. In view of the problem that conventional text processing methods cannot capture local feature information, making the model lack of local correlation, and thus affecting the model's expressiveness, this solution adopts a weight matrix to weight the sum of local areas, and performs nonlinear transformation and unilateral inhibition operations through the ReLU function to improve the expressiveness and robustness of the model; in view of the problem that general text processing methods cannot involve the context of the context, cannot capture complex sentence structures and semantic relationships, and thus cannot fully understand and encode text data, this solution uses forward and reverse GRU calculations to capture text data. According to the forward and reverse context information in the sentence, the sentence structure and semantic relationship are captured based on the activation function and nonlinear factors, and the output is weighted and fused to enhance the model performance and generalization ability; the traditional processing model has the problems of being unable to capture global context information and unable to adapt to the needs of different tasks and data, resulting in poor model flexibility, and the large model dimension resulting in excessive calculation and thus reducing the model performance. This scheme adopts the attention mechanism to perform weighted selection of hidden states in the input sequence, perform feature extraction, and learn adjustable weight parameters to adapt to different tasks and data, thereby improving the model flexibility and generalization ability, and reducing the model dimension and calculation by improving the projection network and parallel processing, thereby improving the model performance.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides a digital heart-to-heart pavilion information processing method, the method comprising the following steps:

[0006] Step S1: Data collection, collecting historical data of Xinlianxin Pavilion and corresponding labels as sample data;

[0007] Step S2: CNN layer design, preprocessing the sample data, performing convolution operation and nonlinear transformation on the input through convolution kernel and ReLU function, generating local feature vectors, and generating the final feature sequence based on unilateral inhibition operation;

[0008] Step S3: Bidirectional GRU design, using BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, reverse calculation is performed from m to 1. Finally, the bidirectional outputs are merged into the result of BiGRU.

[0009] Step S4: Attention layer design, based on the attention mechanism, weighted selection of hidden states in the input sequence is performed, and feature information that fits the current task is extracted by calculating the attention weights;

[0010] Step S5: SCL method design, based on data enhancement, encoder network and projection network, maps the output of the attention layer to the feature vector and normalizes it, measures the similarity between different feature representations through the inner product metric, and obtains the loss function equation by calculating the contrast loss;

[0011] Step S6: Determine the model, divide the space required by the model into subspaces, use different operations at different stages to adjust the position of the solution and the gradient ascent algorithm to find the local optimal parameters of each subspace, judge the global optimality based on the fitness value of the combined parameters, and then determine the final model, and regularly process the Xinlianxinting information based on the model.

[0012] Furthermore, in step S1, the data collection is to collect the historical data of Xinlianxinting and the corresponding labels as sample data, and the corresponding labels are the corresponding categories of the data.

[0013] Furthermore, in step S2, the CNN layer design specifically includes the following steps:

[0014] Step S21: preprocessing the text data, the input text of CNN is represented as T1{x1, x2, ..., xn}, the text length is n, each word is represented as a vector xn, after the embedding layer, the word is represented as a vector of dimension d, and the embedding layer is a built-in layer of CNN; to ensure that the length of the input text matrix is ​​the same, the part with a length greater than n is truncated, and the part with a length less than n is filled; for the selection of n-gram features, the height of the convolution kernel is set to 3, 4 and 5, and the step size is set to 1;

[0015] Step S22: Generate a local feature vector. Perform convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function to generate a local feature vector. The formula used is as follows:

[0016] ;

[0017] In the formula, f m is the mth local feature, r() is the ReLU function, W i is the weight matrix of the embedding layer, x t is the tth row in matrix T1, and b is the bias vector of the embedding layer;

[0018] Step S23: Generate the final feature sequence. In the unilateral suppression operation, the ReLU function is used to convert negative values ​​to zero, and all positive values ​​and zero remain unchanged. The pooling layer is used to retain the main features while reducing parameters and calculations. Finally, the derived feature sequence is L f {f1, f2, ..., f m}, connect the output of the three-layer CNN as the downstream input, and the formula used is as follows:

[0019] ;

[0020] Where F is the feature sequence matrix, F1, F2, ..., F m is the feature vector of different sizes extracted from the feature sequence output by the CNN network in the third layer, L f （3） is the feature sequence generated in the third layer of CNN, which consists of feature vectors obtained by convolution kernels of size 3. f （4） is the feature sequence generated in the fourth layer of CNN, which consists of feature vectors obtained by convolution kernels of size 4. f （5） is the feature sequence generated in the fifth layer of CNN, which consists of the feature vector obtained by the convolution kernel of size 5, f1 （3） is the first feature vector calculated in the third layer of CNN, f1 （4） is the first feature vector calculated in the fourth layer of CNN, f1 （5） is the first eigenvector calculated in the fifth layer of CNN, f m （3） is the mth feature vector calculated in the third layer of CNN, f m （4） is the mth eigenvector calculated in the fourth layer of CNN, is the mth eigenvector calculated in the fifth layer of CNN, and ⊕ is the concatenation operator.

[0021] Furthermore, in step S3, the bidirectional GRU design architecture uses BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, the reverse calculation is performed from m to 1. Finally, the bidirectional output is merged into the result of BiGRU. The whole calculation process is described by the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] Where Y m and are the forward GRU output and the reverse GRU output respectively, BiGRU() is the function of the bidirectional gate controlled recurrent unit, is the reverse GRU output at time step m-1, f() is the activation function, o m is the output result of BiGRU at time m. The final output of BiGRU network is O=[o1, o2, …, o m ], ω is the weight corresponding to the forward hidden state and the reverse hidden state, ω1 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU, ω2 is the weight parameter of the weighted sum of the hidden state at the previous moment and the current hidden state in the forward GRU, ω3 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU; ω4 is the weight parameter of the weighted sum of the reverse hidden state at the previous moment and the current hidden state in the forward GRU; ω5 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU; ω6 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU.

[0026] Furthermore, in the design of the attention layer in step S4, the attention mechanism is a feature selection method. At this time, the input sequence is the output O of the BiGRU network. The attention mechanism is used to perform weighted selection on the hidden states in the input sequence to obtain a contextual representation based on the attention weights to extract important features related to the current task. The attention mechanism is simplified as follows:

[0027] ;

[0028] ;

[0029] ;

[0030] In the formula, e t is the weight value of the attention mechanism, a() is a learnable function, α tis the normalized result of the attention weight value, m is the input sequence length, k is the input sequence index, c is the attention weight embedding sequence, parameters t and k represent different positions, o t is the hidden state of the t-th position, e k is the attention weight value of the k-th position.

[0031] Further, in step S5, the SCL method is designed to enhance each sample data by embedding penalty gradient method to generate data enhanced data, input the enhanced data into the encoder network, and map the attention layer output to a feature vector, z=project(c)∈R Dp , R Dp is the dimension of the feature vector, z is normalized so that the output is on the unit hypersphere, and the output in the projection space is measured using the inner product, that is, the encoded feature representation is mapped to a low-dimensional space, and the similarity between different feature representations is measured by the inner product metric. The loss function equation is obtained by calculating the contrast loss. The formula used is as follows:

[0032] ;

[0033] ;

[0034] Where, L scl is the contrast loss, N is the total number of sample data categories after data enhancement, i and j are category indexes, and Z i represents the feature vector of the i-th category, T is the transpose, is a negative example, is a positive example, τ is the temperature parameter, L total is the loss function equation, ε is a hyperparameter used to control the degree of gradient penalty, is the contrast loss L scl The gradient of the eigenvector z;

[0035] Furthermore, in step S6, the determining model specifically includes the following steps:

[0036] Step S61: For the remaining design, use the pooling layer, the dropout layer, and the normalization layer to avoid over-filtering, so that the softmax layer generates the probability distribution of the text label class, and select the Adam optimizer to optimize the loss function of the network;

[0037] Step S62: parameter optimization, a fitness value threshold and a subspace maximum iteration number T2 are pre-set, the parameter space required by the model is divided into multiple subspaces and the solution sets are initialized for each subspace, the classification accuracy of the model for the sample data is used as the parameter fitness value, the position of the solution is adjusted based on the parameter fitness value, the gradient ascent algorithm is used to find the maximum point of the parameter fitness value curve in each subspace as the local optimal parameter, and the parameters of the local optimal combination of all subspaces are used as the pre-global solution. If the fitness value of the pre-global solution is higher than the fitness value threshold, the model is established using the parameters at this time, otherwise the subspace is re-divided and the solution set is initialized to find the next generation of pre-global solutions. The formula used to adjust the position of the solution is as follows:

[0038] ;

[0039] In the formula, is the position of the t+1th iteration of the ith solution, is the position of the tth iteration, is the optimal solution for the tth iteration, is the optimal solution for t+1 iterations, rand is a random number between 0 and 1, δ is the minimum value to prevent the denominator from being 0, α is a constant value of 0.1, and ES is a decreasing random value in the interval [-2, 2]. is the random solution in the tth iteration, UB and LB are the upper and lower boundaries of the search space respectively;

[0040] Step S63: Specific operation is to regularly collect information received by the Xinlianxin Pavilion and input it into the model corresponding to the parameters outputted in step S62, and take the ranking of the category proportions outputted by the model as the information with the highest demand, check the corresponding areas, and solve the corresponding demands in turn.

[0041] The present invention provides a digital heart-to-heart pavilion information processing system, comprising a data acquisition module, a CNN layer design module, a bidirectional GRU design module, an attention layer design module, an SCL method design module and a determination model module;

[0042] The data acquisition module collects the historical data of Xinlianxin Pavilion and the corresponding labels as sample data, and sends the collected data to the CNN layer design module;

[0043] The CNN layer design module receives the data sent by the data acquisition module, pre-processes the sample data, performs convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function, generates a local feature vector, generates a final feature sequence based on a unilateral inhibition operation, and sends the data to the bidirectional GRU design module;

[0044] The bidirectional GRU design module receives the data sent by the CNN layer design module, uses BiGRU to encode the forward and reverse information of the sentence, the forward GRU calculates the hidden state of each time step from 1 to m, and in the reverse GRU, performs reverse calculation from m to 1. Finally, the bidirectional output is merged into the result of BiGRU, and the data is sent to the attention layer design module;

[0045] The attention layer design module receives the data sent by the bidirectional GRU design module, performs weighted selection on the hidden states in the input sequence based on the attention mechanism, extracts feature information that fits the current task by calculating the attention weights, and sends the data to the SCL method design module;

[0046] The SCL method design module receives data sent by the attention layer design module, maps the attention layer output to a feature vector and normalizes it based on data enhancement, an encoder network and a projection network, measures the similarity between different feature representations by an inner product metric, obtains a loss function equation by calculating the contrast loss, and sends the data to the determination model module;

[0047] The model determination module receives the data sent by the SCL method design module, divides the space required by the model into subspaces, adjusts the position of the solution by different operations at different stages and uses the gradient ascent algorithm to find the local optimal parameters of each subspace, judges the global optimality according to the fitness value of the combined parameters, and then determines the final model, and regularly processes the Xinlianxinting information based on the model.

[0048] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0049] (1) To address the problem that conventional text processing methods are unable to capture local feature information, resulting in the model having no local correlation, which in turn affects the model's expressiveness, this solution uses a weight matrix to weight the sum of local regions, and uses the ReLU function to perform nonlinear transformation and unilateral suppression operations to improve the model's expressiveness and robustness.

[0050] (2) In view of the problem that general text processing methods cannot involve the context of the context, cannot capture complex sentence structures and semantic relationships, and thus cannot fully understand and encode text data, this solution uses forward and reverse GRU calculations to capture the forward and reverse context information in the text data, captures the sentence structure and semantic relationships based on activation functions and nonlinear factors, and weightedly fuses the outputs to enhance model performance and generalization capabilities.

[0051] (3) In order to address the problems that traditional processing models cannot capture global context information and cannot adapt to the needs of different tasks and data, resulting in poor model flexibility, and the model dimension is too large, resulting in excessive computation and thus reducing model performance, this scheme uses an attention mechanism to perform weighted selection of hidden states in the input sequence, perform feature extraction, and learn adjustable weight parameters to adapt to different tasks and data, thereby improving model flexibility and generalization ability. By improving the projection network and parallel processing, the model dimension and computation are reduced, thereby improving model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a flow chart of a digital heart-to-heart pavilion information processing method provided by the present invention;

[0053] Figure 2 A schematic diagram of a digital heart-to-heart pavilion information processing system provided by the present invention;

[0054] Figure 3 is a schematic flow chart of step S2;

[0055] Figure 4 It is a flowchart diagram of step S6.

[0056] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0058] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0059] Example 1, see Figure 1 The present invention provides a digital heart-to-heart pavilion information processing method, the method comprising the following steps:

[0060] Step S1: Data collection, collecting historical data of Xinlianxin Pavilion and corresponding labels as sample data, where the corresponding labels are data corresponding categories;

[0061] Step S2: CNN layer design, preprocessing the sample data, performing convolution operation and nonlinear transformation on the input through convolution kernel and ReLU function, generating local feature vectors, and generating the final feature sequence based on unilateral inhibition operation;

[0062] Step S3: Bidirectional GRU design, using BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, reverse calculation is performed from m to 1. Finally, the bidirectional outputs are merged into the result of BiGRU.

[0063] Step S4: Attention layer design, based on the attention mechanism, weighted selection of hidden states in the input sequence is performed, and feature information that fits the current task is extracted by calculating the attention weights;

[0064] Step S5: SCL method design, based on data enhancement, encoder network and projection network, maps the output of the attention layer to the feature vector and normalizes it, measures the similarity between different feature representations through the inner product metric, and obtains the loss function equation by calculating the contrast loss;

[0065] Step S6: Determine the model, divide the space required by the model into subspaces, use different operations at different stages to adjust the position of the solution and the gradient ascent algorithm to find the local optimal parameters of each subspace, judge the global optimality based on the fitness value of the combined parameters, and then determine the final model, and regularly process the Xinlianxinting information based on the model.

[0066] Example 2, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S2, the CNN layer design specifically includes the following steps:

[0067] Step S21: preprocessing the text data, the input text of CNN is represented as T1{x1, x2, ..., xn}, the text length is n, each word is represented as a vector xn, after the embedding layer, the word is represented as a vector of dimension d, and the embedding layer is a built-in layer of CNN; to ensure that the length of the input text matrix is ​​the same, the part with a length greater than n is truncated, and the part with a length less than n is filled; for the selection of n-gram features, the height of the convolution kernel is set to 3, 4 and 5, and the step size is set to 1;

[0068] Step S22: Generate a local feature vector. Perform convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function to generate a local feature vector. The formula used is as follows:

[0069] ;

[0070] In the formula, f m is the mth local feature, r() is the ReLU function, W i is the weight matrix of the embedding layer, x t is the tth row in matrix T1, and b is the bias vector of the embedding layer;

[0071] Step S23: Generate the final feature sequence. In the unilateral suppression operation, the ReLU function is used to convert negative values ​​to zero, and all positive values ​​and zero remain unchanged. The pooling layer is used to retain the main features while reducing parameters and calculations. Finally, the derived feature sequence is L f {f1, f2, ..., f m}, connect the output of the three-layer CNN as the downstream input, and the formula used is as follows:

[0072] ;

[0073] Where F is the feature sequence matrix, F1, F2, ..., F m is the feature vector of different sizes extracted from the feature sequence output by the CNN network in the third layer, L f （3） is the feature sequence generated in the third layer of CNN, which consists of feature vectors obtained by convolution kernels of size 3. f （4） is the feature sequence generated in the fourth layer of CNN, which consists of feature vectors obtained by convolution kernels of size 4. f （5） is the feature sequence generated in the fifth layer of CNN, which consists of the feature vector obtained by the convolution kernel of size 5, f1 （3） is the first feature vector calculated in the third layer of CNN, f1 （4） is the first feature vector calculated in the fourth layer of CNN, f1 （5） is the first eigenvector calculated in the fifth layer of CNN, f m （3） is the mth feature vector calculated in the third layer of CNN, f m （4） is the mth eigenvector calculated in the fourth layer of CNN, is the mth eigenvector calculated in the fifth layer of CNN, and ⊕ is the concatenation operator.

[0074] By performing the above operations, in order to solve the problem that conventional text processing methods cannot capture local feature information, making the model lack of local correlation and thus affecting the model's expressiveness, this solution uses a weight matrix to weight the sum of local areas, and performs nonlinear transformation and unilateral suppression operations through the ReLU function to improve the model's expressiveness and robustness.

[0075] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S3, the bidirectional GRU design architecture uses BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, reverse calculation is performed from m to 1. Finally, the bidirectional output is merged into the result of BiGRU. The whole calculation process is described by the following formula:

[0076] ;

[0077] ;

[0078] ;

[0079] Where Y m and are the forward GRU output and the reverse GRU output respectively, BiGRU() is the function of the bidirectional gate controlled recurrent unit, is the reverse GRU output at time step m-1, f() is the activation function, o m is the output result of BiGRU at time m. The final output of BiGRU network is O=[o1, o2, …, o m ], ω is the weight corresponding to the forward hidden state and the reverse hidden state, ω1 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU, ω2 is the weight parameter of the weighted sum of the hidden state at the previous moment and the current hidden state in the forward GRU, ω3 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU; ω4 is the weight parameter of the weighted sum of the reverse hidden state at the previous moment and the current hidden state in the forward GRU; ω5 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU; ω6 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU.

[0080] By performing the above operations, in order to address the problem that general text processing methods cannot involve the context of the context, cannot capture complex sentence structures and semantic relationships, and thus cannot fully understand and encode text data, this solution uses forward and reverse GRU calculations to capture the forward and reverse context information in text data, captures sentence structures and semantic relationships based on activation functions and nonlinear factors, and weightedly fuses the outputs to enhance model performance and generalization capabilities.

[0081] Example 4, see Figure 1 This embodiment is based on the above embodiment. In the design of the attention layer in step S4, the attention mechanism is a feature selection method. At this time, the input sequence is the output O of the BiGRU network. The attention mechanism is used to perform weighted selection on the hidden state in the input sequence to obtain a context representation based on the attention weight to extract important features related to the current task. The attention mechanism is simplified as follows:

[0082] ;

[0083] ;

[0084] ;

[0085] In the formula, e t is the weight value of the attention mechanism, a() is a learnable function, α t is the normalized result of the attention weight value, m is the input sequence length, k is the input sequence index, c is the attention weight embedding sequence, parameters t and k represent different positions, o t is the hidden state of the t-th position, e k is the attention weight value of the k-th position.

[0086] Example 5, see Figure 1 , This embodiment is based on the above embodiment. In step S5, the SCL method is designed to enhance each sample data by using the embedded penalty gradient method to generate data after data enhancement, input the enhanced data into the encoder network, and map the output of the attention layer to a feature vector, z=project(c)∈R Dp , R Dp is the dimension of the feature vector, z is normalized so that the output is on the unit hypersphere, and the output in the projection space is measured using the inner product, that is, the encoded feature representation is mapped to a low-dimensional space, and the similarity between different feature representations is measured by the inner product metric. The loss function equation is obtained by calculating the contrast loss. The formula used is as follows:

[0087] ;

[0088] ;

[0089] Where, L scl is the contrast loss, N is the total number of sample data categories after data enhancement, i and j are category indexes, and Z i represents the feature vector of the i-th category, T is the transpose, is a negative example, is a positive example, τ is the temperature parameter, L total is the loss function equation, ε is a hyperparameter used to control the degree of gradient penalty, is the contrast loss L scl The gradient of the eigenvector z.

[0090] Example 6, see Figure 1 and Figure 4 This embodiment is based on the above embodiment. In step S6, determining the model specifically includes the following steps:

[0091] Step S61: For the remaining design, use the pooling layer, the dropout layer, and the normalization layer to avoid over-filtering, so that the softmax layer generates the probability distribution of the text label class, and select the Adam optimizer to optimize the loss function of the network;

[0092] Step S62: parameter optimization, a fitness value threshold and a subspace maximum iteration number T2 are pre-set, the parameter space required by the model is divided into multiple subspaces and the solution sets are initialized for each subspace, the classification accuracy of the model for the sample data is used as the parameter fitness value, the position of the solution is adjusted based on the parameter fitness value, the gradient ascent algorithm is used to find the maximum point of the parameter fitness value curve in each subspace as the local optimal parameter, and the parameters of the local optimal combination of all subspaces are used as the pre-global solution. If the fitness value of the pre-global solution is higher than the fitness value threshold, the model is established using the parameters at this time, otherwise the subspace is re-divided and the solution set is initialized to find the next generation of pre-global solutions. The formula used to adjust the position of the solution is as follows:

[0093] ;

[0094] In the formula, is the position of the t+1th iteration of the ith solution, is the position of the tth iteration, is the optimal solution for the tth iteration, is the optimal solution for the t+1th iteration, rand is a random number between 0 and 1, δ is the minimum value to prevent the denominator from being 0, α is a constant value of 0.1, and ES is a decreasing random value in the interval [-2, 2]. is the random solution in the tth iteration, UB and LB are the upper and lower boundaries of the search space respectively;

[0095] Step S63: Specific operation is to regularly collect information received by the Xinlianxin Pavilion and input it into the model corresponding to the parameters outputted by step S63, and take the ranking of the category proportions outputted by the model as the information with the highest demand, check the corresponding areas, and solve the corresponding demands in turn.

[0096] By performing the above operations, this solution uses an attention mechanism to perform weighted selection of hidden states in the input sequence, extract features, and learn adjustable weight parameters to adapt to different tasks and data, thereby improving model flexibility and generalization ability. It also reduces model dimension and computational complexity by improving projection networks and parallel processing, thereby improving model performance.

[0097] Embodiment 7, see Figure 2 , This embodiment is based on the above embodiment. The present invention provides a digital heart-to-heart pavilion information processing system, including a data acquisition module, a CNN layer design module, a bidirectional GRU design module, an attention layer design module, an SCL method design module and a determination model module;

[0098] The data acquisition module collects the historical data of Xinlianxin Pavilion and the corresponding labels as sample data, and sends the collected data to the CNN layer design module;

[0099] The CNN layer design module receives the data sent by the data acquisition module, pre-processes the sample data, performs convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function, generates a local feature vector, generates a final feature sequence based on a unilateral inhibition operation, and sends the data to the bidirectional GRU design module;

[0100] The bidirectional GRU design module receives the data sent by the CNN layer design module, uses BiGRU to encode the forward and reverse information of the sentence, the forward GRU calculates the hidden state of each time step from 1 to m, and in the reverse GRU, performs reverse calculation from m to 1. Finally, the bidirectional output is merged into the result of BiGRU, and the data is sent to the attention layer design module;

[0101] The attention layer design module receives the data sent by the bidirectional GRU design module, performs weighted selection on the hidden states in the input sequence based on the attention mechanism, extracts feature information that fits the current task by calculating the attention weights, and sends the data to the SCL method design module;

[0102] The SCL method design module receives data sent by the attention layer design module, maps the attention layer output to a feature vector and normalizes it based on data enhancement, an encoder network and a projection network, measures the similarity between different feature representations by an inner product metric, obtains a loss function equation by calculating the contrast loss, and sends the data to the determination model module;

[0103] The model determination module receives the data sent by the SCL method design module, divides the space required by the model into subspaces, adjusts the position of the solution by different operations at different stages and uses the gradient ascent algorithm to find the local optimal parameters of each subspace, judges the global optimality according to the fitness value of the combined parameters, and then determines the final model, and regularly processes the Xinlianxinting information based on the model.

[0104] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0105] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0106] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A digital heart-to-heart kiosk information processing method, characterized by: The method comprises the following steps: Step S1: Data collection, collecting historical data of Xinlianxin Pavilion and corresponding labels as sample data; Step S2: CNN layer design, preprocessing the sample data, performing convolution operation and nonlinear transformation on the input through convolution kernel and ReLU function, generating local feature vectors, and generating the final feature sequence based on unilateral inhibition operation; Step S3: Bidirectional GRU design, using BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, reverse calculation is performed from m to 1. Finally, the bidirectional outputs are merged into the result of BiGRU. Step S4: Attention layer design, based on the attention mechanism, weighted selection of hidden states in the input sequence is performed, and feature information that fits the current task is extracted by calculating the attention weights; Step S5: SCL method design, based on data enhancement, encoder network and projection network, maps the output of the attention layer to the feature vector and normalizes it, measures the similarity between different feature representations through the inner product metric, and obtains the loss function equation by calculating the contrast loss; Step S6: Determine the model, divide the space required by the model into subspaces, use different operations at different stages to adjust the position of the solution and the gradient ascent algorithm to find the local optimal parameters of each subspace, determine the global optimality based on the fitness value of the combined parameters, and then determine the final model, and regularly process the Xinlianxinting information based on the model; In step S5, the SCL method is designed to enhance each sample data by embedding penalty gradient method to generate data enhanced data, input the enhanced data into the encoder network, and map the attention layer output to a feature vector, z=project(c)∈R Dp , R Dp is the dimension of the feature vector, z is normalized so that the output is on the unit hypersphere, and the output in the projection space is measured using the inner product, that is, the encoded feature representation is mapped to a low-dimensional space, and the similarity between different feature representations is measured by the inner product metric. The loss function equation is obtained by calculating the contrast loss. The formula used is as follows: ； ； Where, L scl is the contrast loss, N is the total number of sample data categories after data enhancement, i and j are category indexes, and Z i represents the feature vector of the i-th category, T is the transpose, is a negative example, is a positive example, τ is the temperature parameter, L total is the loss function equation, ε is a hyperparameter used to control the degree of gradient penalty, is the contrast loss L scl The gradient of the eigenvector z; In step S1, the data collection is to collect the historical data of Xinlianxinting and the corresponding labels as sample data, and the corresponding labels are the corresponding categories of the data.

2. A digital heart-to-heart kiosk information processing method according to claim 1, characterized in that: In step S2, the CNN layer design specifically includes the following steps: Step S21: preprocessing the text data, the input text of CNN is represented as T1{x1, x2, ..., xn}, the text length is n, each word is represented as a vector xn, after the embedding layer, the word is represented as a vector of dimension d, and the embedding layer is a built-in layer of CNN; to ensure that the length of the input text matrix is ​​the same, the part with a length greater than n is truncated, and the part with a length less than n is filled; for the selection of n-gram features, the height of the convolution kernel is set to 3, 4 and 5, and the step size is set to 1; Step S22: Generate a local feature vector. Perform convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function to generate a local feature vector. The formula used is as follows: ； In the formula, f m is the mth local feature, r() is the ReLU function, W i is the weight matrix of the embedding layer, x t is the tth row in matrix T1, and b is the bias vector of the embedding layer; Step S23: Generate the final feature sequence. In the unilateral suppression operation, the ReLU function is used to convert negative values ​​to zero, and all positive values ​​and zero remain unchanged. The pooling layer is used to retain the main features while reducing parameters and calculations. Finally, the derived feature sequence is L f {f1, f2, ..., f m }, connect the output of the three-layer CNN as the downstream input, and the formula used is as follows: ； Where F is the feature sequence matrix, F1, F2, ..., F m is the feature vector of different sizes extracted from the feature sequence output by the CNN network in the third layer, L f （3） is the feature sequence generated in the third layer of CNN, which consists of feature vectors obtained by convolution kernels of size 3. f （4） is the feature sequence generated in the fourth layer of CNN, which consists of feature vectors obtained by convolution kernels of size 4. f （5） is the feature sequence generated in the fifth layer of CNN, which consists of the feature vector obtained by the convolution kernel of size 5, f1 （3） is the first feature vector calculated in the third layer of CNN, f1 （4） is the first feature vector calculated in the fourth layer of CNN, f1 （5） is the first eigenvector calculated in the fifth layer of CNN, f m （3） is the mth feature vector calculated in the third layer of CNN, f m （4） is the mth eigenvector calculated in the fourth layer of CNN, is the mth eigenvector calculated in the fifth layer of CNN, and ⊕ is the concatenation operator.

3. A digital heart-to-heart kiosk information processing method according to claim 1, characterized in that: In step S3, the bidirectional GRU design architecture uses BiGRU to encode the forward and reverse information of the sentence. The forward GRU calculates the hidden state of each time step from 1 to m. In the reverse GRU, the reverse calculation is performed from m to 1. Finally, the bidirectional output is merged into the result of BiGRU. The whole calculation process is described by the following formula: ； ； ； Where Y m and are the forward GRU output and the reverse GRU output respectively, BiGRU() is the function of the bidirectional gate controlled recurrent unit, is the reverse GRU output at time step m-1, f() is the activation function, o m is the output result of BiGRU at time m. The final output of BiGRU network is O=[o1, o2, …, o m ], ω is the weight corresponding to the forward hidden state and the reverse hidden state, ω1 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU, ω2 is the weight parameter of the weighted sum of the hidden state at the previous moment and the current hidden state in the forward GRU, ω3 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU; ω4 is the weight parameter of the weighted sum of the reverse hidden state at the previous moment and the current hidden state in the forward GRU; ω5 is the weight parameter of the weighted sum of the input feature sequence and the current hidden state in the forward GRU; ω6 is the weight parameter of the weighted sum of the input feature sequence and the current reverse hidden state in the reverse GRU.

4. A digital heart-to-heart kiosk information processing method according to claim 1, characterized in that: In step S6, the determining model specifically includes the following steps: Step S61: For the remaining design, use the pooling layer, the dropout layer, and the normalization layer to avoid over-filtering, so that the softmax layer generates the probability distribution of the text label class, and select the Adam optimizer to optimize the loss function of the network; Step S62: parameter optimization, a fitness value threshold and a subspace maximum iteration number T2 are pre-set, the parameter space required by the model is divided into multiple subspaces and the solution sets are initialized for each subspace, the classification accuracy of the model for the sample data is used as the parameter fitness value, the position of the solution is adjusted based on the parameter fitness value, the gradient ascent algorithm is used to find the maximum point of the parameter fitness value curve in each subspace as the local optimal parameter, and the parameters of the local optimal combination of all subspaces are used as the pre-global solution. If the fitness value of the pre-global solution is higher than the fitness value threshold, the model is established using the parameters at this time, otherwise the subspace is re-divided and the solution set is initialized to find the next generation of pre-global solutions. The formula used to adjust the position of the solution is as follows: ； In the formula, is the position of the t+1th iteration of the ith solution, is the position of the tth iteration, is the optimal solution for the tth iteration, is the optimal solution for the t+1th iteration, rand is a random number between 0 and 1, δ is the minimum value to prevent the denominator from being 0, α is a constant value of 0.1, and ES is a decreasing random value in the interval [-2, 2]. is the random solution in the tth iteration, UB and LB are the upper and lower boundaries of the search space respectively; Step S63: Specific operation is to regularly collect information received by the Xinlianxin Pavilion and input it into the model corresponding to the parameters outputted in step S62, and take the ranking of the category proportions outputted by the model as the information with the highest demand, check the corresponding areas, and solve the corresponding demands in turn.

5. The method for processing digital heart-to-heart booth information according to claim 1, characterized in that: In the design of the attention layer in step S4, the attention mechanism is a feature selection method. At this time, the input sequence is the output O of the BiGRU network. The attention mechanism is used to perform weighted selection on the hidden states in the input sequence to obtain a contextual representation based on the attention weights to extract important features related to the current task. The attention mechanism is simplified as follows: ； ； ； In the formula, e t is the weight value of the attention mechanism, a() is a learnable function, α t is the normalized result of the attention weight value, m is the input sequence length, k is the input sequence index, c is the attention weight embedding sequence, parameters t and k represent different positions, o t is the hidden state of the t-th position, e k is the attention weight value of the k-th position.

6. A digital heart-to-heart pavilion information processing system, used to implement a digital heart-to-heart pavilion information processing method as claimed in any one of claims 1 to 5, characterized in that: It includes data acquisition module, CNN layer design module, bidirectional GRU design module, attention layer design module, SCL method design module and model determination module; The data acquisition module collects the historical data of Xinlianxin Pavilion and the corresponding labels as sample data, and sends the collected data to the CNN layer design module; The CNN layer design module receives the data sent by the data acquisition module, pre-processes the sample data, performs convolution operation and nonlinear transformation on the input through the convolution kernel and ReLU function, generates a local feature vector, generates a final feature sequence based on a unilateral inhibition operation, and sends the data to the bidirectional GRU design module; The bidirectional GRU design module receives the data sent by the CNN layer design module, uses BiGRU to encode the forward and reverse information of the sentence, the forward GRU calculates the hidden state of each time step from 1 to m, and in the reverse GRU, performs reverse calculation from m to 1. Finally, the bidirectional output is merged into the result of BiGRU, and the data is sent to the attention layer design module; The attention layer design module receives the data sent by the bidirectional GRU design module, performs weighted selection on the hidden states in the input sequence based on the attention mechanism, extracts feature information that fits the current task by calculating the attention weights, and sends the data to the SCL method design module; The SCL method design module receives data sent by the attention layer design module, maps the attention layer output to a feature vector and normalizes it based on data enhancement, an encoder network and a projection network, measures the similarity between different feature representations by an inner product metric, obtains a loss function equation by calculating the contrast loss, and sends the data to the determination model module; The model determination module receives the data sent by the SCL method design module, divides the space required by the model into subspaces, adjusts the position of the solution by different operations at different stages and uses the gradient ascent algorithm to find the local optimal parameters of each subspace, judges the global optimality according to the fitness value of the combined parameters, and then determines the final model, and regularly processes the Xinlianxinting information based on the model.

Images