Layer normalization scaling method and device based on large model, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a layer normalization scaling method and device based on a large model, equipment and a medium. Carrying out target data screening on the to-be-analyzed text data by utilizing a preset deep neural network model to obtain initial text data, carrying out layer normalization on the initial text data to obtain normalized data, determining a target layer position of the normalized data in the deep neural network model, performing inverse scaling on the normalized data by utilizing the layer number of the target layer position to obtain scaled data, if the target layer position is not corresponding to the preset output layer, activating the scaled data by utilizing a preset activation function to obtain target text data, and if the target layer position is corresponding to the preset output layer, activating the scaled data by utilizing a preset activation function to obtain target text data; and if yes, taking the zoomed data as target text data. According to the invention, the middle-layer normalization accuracy of the deep neural network model is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large-model-based layer normalization scaling method, device, equipment, and medium. Background Art

[0002] Layer normalization is a normalization technique commonly used in neural networks. It aims to help accelerate the training process and improve the stability of the model by normalizing the input of each layer. In large-scale deep learning models, the role of layer normalization is particularly critical because it can effectively reduce internal covariate shift and improve the model's performance during training. To cope with the computational complexity and resource requirements of large models, researchers have proposed scaling methods for layer normalization. However, in today's mainstream large models, deep normalization often fails to achieve the expected results for traditional layer normalization scaling methods, resulting in low accuracy and nearly half of the layers performing below expectations.

[0003] In healthcare, deep learning models are commonly used for disease diagnosis, personalized treatment recommendations, and predictive analysis. Due to the complexity and high dimensionality of medical data, efficient and accurate model training is crucial. However, as the number of model layers increases, the performance of deeper layers often falls short of expectations. Directly removing early layers can lead to a sharp drop in model performance, resulting in reduced accuracy.

[0004] In the fintech sector, deep learning models are widely used in tasks such as credit assessment, risk management, and stock market forecasting. Due to the volatility and complexity of financial data, model training and optimization also present significant challenges. However, removing deep layers or intermediate layers may have little impact on the model's overall performance. This forces large financial models to reassess the contribution of different layers to the task, resulting in reduced model accuracy.

[0005] In summary, in today's mainstream large models (such as Llama, Mistral, DeepSeek, and Qwen), deep model normalization often fails to achieve the expected results, with nearly half of the layers performing below expectations. While removing early layers can lead to a sharp drop in model performance, removing deep layers does not. Intermediate and deep layers are remarkably robust to perturbations such as layer swapping and layer dropping. This situation prevents these layers from performing meaningful transformations and effectively contributing to training and representation learning, leading to reduced model accuracy.

[0006] Therefore, the current technology has the problem of low accuracy of layer normalization in deep neural network models. Summary of the Invention

[0007] The present invention provides a large-model-based layer normalization scaling method, apparatus, device, and medium, the main purpose of which is to solve the problem of low accuracy of layer normalization in deep neural network models.

[0008] In a first aspect, to achieve the above-mentioned purpose, the present invention provides a layer normalization scaling method based on a large model, comprising:

[0009] Obtaining text data to be analyzed, and using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data;

[0010] Performing layer normalization processing on the initial text data to obtain normalized data;

[0011] Determining a target layer position of the normalized data in the deep neural network model;

[0012] Performing inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data, and determining whether the target layer position corresponds to a preset output layer of the deep neural network model;

[0013] If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data;

[0014] If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data.

[0015] In a second aspect, the present invention further provides a layer normalization scaling device based on a large model, comprising:

[0016] A data screening module is used to obtain text data to be analyzed, and use a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data;

[0017] A data normalization module, configured to perform layer normalization processing on the initial text data to obtain normalized data;

[0018] A position determination module, configured to determine a target layer position of the normalized data in the deep neural network model;

[0019] A data scaling module is used to inversely scale the normalized data using the number of layers at the target layer position to obtain scaled data, and to determine whether the target layer position corresponds to a preset output layer of the deep neural network model;

[0020] A first judgment result module is used to activate the scaled data using a preset activation function to obtain target text data if the target layer position does not correspond to the preset output layer of the deep neural network model;

[0021] The second judgment result module is used to use the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

[0022] In a third aspect, the present invention further provides an electronic device, comprising:

[0023] at least one processor; and,

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the above-mentioned large model-based layer normalization scaling method.

[0026] In a fourth aspect, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned large model-based layer normalization scaling method.

[0027] The present invention obtains text data to be analyzed, and uses a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data. This not only improves the automation and intelligence level of data screening, avoids the subjectivity and inefficiency problems caused by manual rule screening, but also effectively reduces the data scale of subsequent processing. The initial text data is layer-normalized to obtain normalized data, which not only speeds up the model convergence speed, but also improves the stability of the training process and the accuracy of the results. The target layer position of the normalized data in the deep neural network model is determined, so that the subsequent inverse scaling operation based on the number of layers has a clear and accurate positioning basis. The number of layers at the target layer position is used to inversely scale the normalized data. Scaling to obtain scaled data, dynamically adjusting the numerical range of the features according to the different levels of the features in the deep neural network, strengthening the importance of high-level features, and suppressing low-level redundant information, so that the scaled data is more in line with the model's requirements for distinguishing features at different semantic levels, and judging whether the target layer position corresponds to the preset output layer of the deep neural network model; if the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data; if the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data, which can effectively improve the accuracy of normalization of the layers in the deep neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 A schematic diagram of an application environment of a large model-based layer normalization scaling method according to an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a process flow of a layer normalization and scaling method based on a large model provided by one embodiment of the present invention;

[0031] Figure 3 A schematic flow chart of an inverse scaling process in a large model-based layer normalization scaling method provided by one embodiment of the present invention;

[0032] Figure 4 A schematic diagram of a module of a large model-based layer normalization and scaling device provided by one embodiment of the present invention;

[0033] Figure 5 A schematic structural diagram of an electronic device for implementing a large model-based layer normalization scaling method provided by one embodiment of the present invention;

[0034] Figure 6 Another structural diagram of an electronic device for implementing a large model-based layer normalization scaling method provided by an embodiment of the present invention.

[0035] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] The embodiment of the present application provides a layer normalization scaling method based on a large model, and the execution subject of the layer normalization scaling method based on a large model includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the device provided by the embodiment of the present application. In other words, the layer normalization scaling method based on a large model can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0039] The embodiment of the present invention provides a layer normalization scaling method based on a large model, which can be applied in the following situations: Figure 1In the application environment. Among them, the client communicates with the server through the network. The server can obtain the text data to be analyzed through the client, and use the preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data, which not only improves the automation and intelligence level of data screening, avoids the subjectivity and inefficiency problems brought by manual rule screening, but also can effectively reduce the scale of data for subsequent processing, and performs layer normalization on the initial text data to obtain normalized data, which not only speeds up the convergence speed of the model, but also improves the stability of the training process and the accuracy of the results, determines the target layer position of the normalized data in the deep neural network model, so that the subsequent inverse scaling operation based on the number of layers has a clear and accurate positioning basis, and uses the number of layers of the target layer position to perform inverse scaling on the normalized data to obtain the scaling data. According to the different levels of the features in the deep neural network, the numerical range of the features is dynamically adjusted, the importance of high-level features is strengthened, and low-level redundant information is suppressed, so that the scaled data is more in line with the model's requirements for distinguishing features at different semantic levels, and whether the target layer position corresponds to the preset output layer of the deep neural network model is judged. If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data, which can effectively improve the accuracy of normalization of the middle layer of the deep neural network model, and finally the target text data is output and fed back to the user client. Among them, the client can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0040] The following is an explanation of the specification of the present invention. The present invention uses the number of layers at the target layer position to inversely scale the normalized data. It can dynamically adjust the numerical range of the features according to the different levels of the features in the deep neural network, strengthen the importance of high-level features, and suppress low-level redundant information, so that the scaled data is more in line with the model's requirements for distinguishing features at different semantic levels. Since shallow features are directly derived from the input, the range of variation is small and the scaling factor is large, so more fine-grained information can be retained; while deep features have been abstracted many times, the number of layers is large, and the scaling factor is small, so that high-level semantic information occupies an appropriate proportion in the overall feature, preventing excessive expansion or distortion of the features. Ultimately, scaling data can improve the model's adaptability to complex structures and convergence speed, while enhancing the model's discrimination performance and accuracy.

[0041] Reference Figure 2FIG. 1 is a flow chart of a layer normalization scaling method based on a large model provided by an embodiment of the present invention. In this embodiment, the layer normalization scaling method based on a large model includes:

[0042] S1. Obtain text data to be analyzed, and use a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data.

[0043] In an embodiment of the present invention, the text data to be analyzed includes but is not limited to log records, user input, comment data, customer service conversations, emails, etc. By embedding word vectors in the text data to be analyzed, modeling contextual features, and calculating feature scoring functions, the model can identify and filter out semantic fragments or information units that are highly relevant to the target analysis task, thereby eliminating redundant content and irrelevant information, and finally extracting the initial text data for subsequent processing.

[0044] In healthcare scenarios, the text data to be analyzed typically includes medical records, doctors' diagnostic reports, patient health records, drug information, and academic literature. Deep learning models (such as BERT and LSTM) are used to filter key information from medical records, such as patient symptoms, diagnosis results, treatment plans, and drug prescriptions. This helps medical personnel automatically extract important information from large amounts of electronic medical records, reducing the workload of manual searches and improving patient diagnosis and treatment efficiency.

[0045] In specific FinTech scenarios, the text data to be analyzed may include news articles, financial reports, investment analysis, customer feedback, social media comments, etc. Using deep neural networks to filter and automatically analyze text data such as financial reports, quarterly financial statements, and annual reports, extracting key information such as financial status, profitability, and risk points, helps make more accurate investment decisions.

[0046] In an embodiment of the present invention, the method of using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data includes:

[0047] Standardizing the format of the text data to be analyzed to obtain standard text data;

[0048] Converting the standard text data into a text embedding vector using an embedding layer of a preset deep neural network model;

[0049] Encoding the text embedding vector to obtain a text embedding code;

[0050] Performing contextual semantic modeling on the text embedding code to obtain a semantic coding vector;

[0051] Obtaining a feature scoring function of the deep neural network model, performing feature analysis on the standard text data according to the semantic encoding vector and the feature scoring function to obtain a scoring vector;

[0052] Determine whether the scoring vector is greater than a preset score threshold, and filter out the scoring vectors that are greater than the score threshold;

[0053] The filtered scoring vectors are used as initial text data.

[0054] In detail, the text encoding is unified, for example, all texts are converted to UTF-8 format to avoid garbled characters caused by different encodings, special characters, HTML tags, control symbols or abnormal symbols are removed, meaningless content is cleaned up, punctuation formats are unified (such as interchangeability of Chinese and English symbols), and uppercase and lowercase, simplified and traditional Chinese characters are standardized to make the text structure more conducive to model understanding. Finally, the cleaned text is standardized and output according to the predetermined format as standard text data that can be directly processed in the subsequent modeling stage, thereby improving the overall data quality and algorithm performance.

[0055] The standard text data is segmented into words, subwords, or characters according to predefined segmentation rules. Each word is then converted into an index number through an encoding mechanism. This index number is then fed into the embedding layer, which maps each word into a text embedding vector representation by searching a pre-trained embedding matrix. The vectors typically have dimensions ranging from tens to hundreds. These vectors not only retain the basic information of the words but also learn the semantic connections between them during training, placing semantically similar words closer together in the vector space. This lays the foundation for subsequent semantic modeling and contextual analysis.

[0056] The initial text embedding vector is input into a preset encoding structure, such as a feedforward neural network, a convolutional network (CNN), or a simple linear transformation layer, and each vector is dimensionally transformed or feature mapped to generate an intermediate representation in a unified format with certain semantic expression capabilities, namely, text embedding encoding. Text embedding encoding not only inherits the basic semantic information in the original embedding, but also provides a clear structure and efficient processing input basis for the subsequent context modeling stage, making semantic reasoning more accurate and stable.

[0057] In an embodiment of the present invention, performing contextual semantic modeling on the text embedding code to obtain a semantic coding vector includes:

[0058] Embedding the text into a code sequence as an input text sequence;

[0059] Performing position encoding on the input text sequence to obtain a text update sequence;

[0060] Capturing the contextual dependencies of the text update sequence to obtain semantic relationships;

[0061] A semantic encoding vector of the text embedding encoding is generated according to the semantic relationship.

[0062] Specifically, the text embedding generated in the previous step is fed into the encoding module as the input text sequence. To preserve the positional information of the words in the original sentence, a positional encoding is first added to the input text sequence. By fusing the positional information with the original embedding vector, a text update sequence with positional information is obtained. Subsequently, context modeling is performed on the text update sequence, and the model's internal structures (such as self-attention mechanisms and bidirectional recurrent neural networks) are used to capture the dependencies between the context of each word in the sequence, identifying the semantic associations and mutual influences of each word in the entire sentence. Finally, based on the captured semantic relationships, the embedding representation of each word is semantically enhanced to generate a semantic encoding vector for the text embedding code with contextual semantic understanding capabilities, thus laying the foundation for subsequent feature extraction and text analysis.

[0063] Specifically, the preset feature scoring function is called from the model. The feature scoring function is usually a weight calculation layer. The semantic encoding vector obtained in the previous step is input into the feature scoring function. The calculation formula is as follows:

[0064] s i =f s (h i )=w T *tanh(Wh i +b)

[0065] Among them, h i represents the i-th semantic encoding vector, f s represents the feature scoring function, s i represents the i-th scoring vector, w T represents the transposed vector of the scoring weight vector, W represents the weight matrix, b represents the bias vector, and tanh represents the tangent function.

[0066] After obtaining the scoring vector corresponding to each text unit, the system sets a preset score threshold to distinguish which text units meet the importance requirements. If the scoring vector is greater than the score threshold, the text unit is considered to have a higher semantic value or feature representativeness in the overall text and is retained; if the scoring vector is less than or equal to the score threshold, it is not adopted, and the original content corresponding to all text units with score vectors greater than the threshold is extracted as the initial text data for subsequent analysis or processing.

[0067] By utilizing a pre-set deep neural network model to target text data for analysis, it is possible to systematically unify formats, extract semantic features, and assess importance within large amounts of raw data, thereby accurately selecting initial text data with high semantic value and representativeness. This approach not only enhances the automation and intelligence of data screening, avoiding the subjectivity and inefficiency inherent in manual rule-based screening, but also effectively reduces the scale of subsequent data processing, enhances the relevance and accuracy of model training or application, and significantly improves the quality and efficiency of data preprocessing and feature extraction overall.

[0068] S2. Perform layer normalization processing on the initial text data to obtain normalized data.

[0069] In an embodiment of the present invention, for the initial text data that has been screened, in order to further improve the consistency of feature distribution and the stability of model processing, the layer normalization technology is used to standardize the initial text data. First, the dimension-level mean and variance of the representation vector of each initial text data in the deep neural network are calculated, and then the mean is subtracted from each eigenvalue and divided by the standard deviation to ensure that all feature components are distributed on the same scale, thereby eliminating the influence of internal covariate offset on subsequent processing processes, and finally obtaining normalized data with numerical stability and structural balance.

[0070] In healthcare scenarios, electronic medical records (EMRs) present patient symptoms, medical histories, and test results as textual data. By applying layer normalization to this data, we can ensure that neural networks can robustly handle different types of symptom and diagnosis data during training.

[0071] In FinTech scenarios, financial market data analysis often involves large amounts of news articles, social media content, and investor commentary. Applying layer-by-layer normalization to this text data can reduce fluctuations caused by varying sources and formats, making sentiment analysis models more stable and accurate.

[0072] In an embodiment of the present invention, performing layer normalization processing on the initial text data to obtain normalized data includes:

[0073] Acquiring feature dimensions of the initial text data;

[0074] Generating the mean and standard deviation of the initial text data according to the feature dimension;

[0075] Performing normal distribution transformation on the initial text data using the mean and the standard deviation to obtain normalized features;

[0076] All the normalized features are aggregated into normalized data.

[0077] Specifically, the initial text data is transformed into a set of vectors with fixed feature representations through deep neural network processing. Each sample can be considered a multidimensional vector. The feature dimension refers to the number of elements contained in each vector. By examining the shape of the initial text data matrix (for example, the number of rows represents the number of samples, and the number of columns represents the number of features), the number of columns is extracted as the feature dimension.

[0078] For each feature dimension, the mean and standard deviation are calculated based on the values of all samples in the initial text data on that feature. The calculation formula is as follows:

[0079]

[0080] Among them, μ n represents the mean, N represents the total number of samples of the initial text data, x m,n Represents the value of the mth initial text data in the nth feature dimension, σ n Represents standard deviation.

[0081] For any feature value in the initial text data, first subtract the mean of the corresponding feature to eliminate the position deviation caused by the different value ranges between different features; then divide it by the standard deviation of the feature to unify the scale of each feature so that each feature conforms to the standard normal distribution with a mean of 0 and a standard deviation of 1 after processing. The normalized features of all the initial text data in each feature dimension are summarized according to the order of the initial text data and the feature order to obtain a complete normalized data matrix. The calculation formula is as follows:

[0082]

[0083] Among them, x m,n Represents the value of the mth initial text data in the nth feature dimension, μ n represents the mean, σ n Represents standard deviation.

[0084] By performing layer-by-layer normalization on the initial text data, we can effectively eliminate numerical differences between different feature dimensions, allowing all features to be processed at a uniform scale. This avoids convergence difficulties or training instability caused by uneven feature distribution during model training. First, we obtain the feature dimensions, calculate the mean and standard deviation for each dimension, then perform a normal distribution transformation to ensure that each feature conforms to a standard normal distribution. Finally, we aggregate all normalized features into overall normalized data. This not only accelerates model convergence but also improves the stability of the training process and the accuracy of the results.

[0085] S3. Determine the target layer position of the normalized data in the deep neural network model.

[0086] In an embodiment of the present invention, the network layer position where the normalized data is currently located is identified based on the feature channel or representation path where the normalized data is located, combined with the model structure definition and the forward propagation tracking mechanism. For example, in the Transformer structure, this can be determined by marking the sub-layer index where the normalization operation is located, and in the CNN or RNN structure, this can be determined by recording the layer number or module node index corresponding to the current normalization operation. The layer position information will serve as the basis for subsequent scaling processing and whether to activate it, ensuring that the model has flexible intermediate feature extraction and adjustment capabilities in the multi-layer processing structure.

[0087] In specific medical and health scenarios, during the analysis of electronic medical records, the model is used to predict whether a patient has a certain disease. Normalized medical record information includes: the patient's condition, medication records, etc., to determine the target layer position of the normalized data, and provide data support for subsequent judgments on whether it is in the output layer and whether data diagnosis has been completed.

[0088] In specific FinTech scenarios, such as credit score prediction, models are used to predict credit scores or a customer's probability of default. Normalized data, including a customer's financial history and credit card spending records, determines the target layer location of the normalized data, providing data support for subsequent determination of whether it is in the output layer and whether the probability prediction has been completed.

[0089] In an embodiment of the present invention, determining the target layer position of the normalized data in the deep neural network model includes:

[0090] Indexing and marking each layer structure of the deep neural network model to obtain layer identification information of each layer structure;

[0091] Obtaining layer index information of the layer where the normalized data is located;

[0092] The layer index information is compared with the layer identification information one by one to obtain the target layer position of the normalized data.

[0093] Specifically, a unique identification number is assigned to each layer in the order of the layers in the deep neural network model, and basic information such as the corresponding layer type and parameter settings are recorded to form layer identification information. First, all layers of the deep neural network (such as embedding layer, encoding layer, attention layer, fully connected layer, etc.) are traversed, and an ordered index value is assigned to each layer according to its position in the model. At the same time, the structural properties of the layer, such as input and output dimensions, activation function type, parameter number, etc., are extracted as layer identification information. By establishing complete layer identification information, clear management and tracking of the network structure can be achieved.

[0094] Generating layer index information based on normalized data means inferring the index numbers of each layer in the deep neural network corresponding to these data based on the source and feature structure of the normalized data, thereby forming the layer index information corresponding to the normalized data.

[0095] These layer indexes are then compared one by one with the layer identification information previously obtained through index tagging. The comparison process primarily relies on key attributes such as index number, layer type, or feature morphology to confirm which layer in the network structure the normalized data originated from. This one-by-one comparison accurately locates the target layer corresponding to the normalized data, providing a basis for subsequent data processing, feature analysis, or model adjustments for specific layers. This ensures a clear and traceable correspondence between the normalized features and the neural network structure, improving the accuracy of data usage and the controllability of operations.

[0096] By indexing and marking each layer structure of the deep neural network model and comparing the normalized data, the target layer position corresponding to the normalized data can be accurately determined, so that the subsequent inverse scaling operation based on the number of layers can have a clear and accurate positioning basis. The determined target layer position provides data support for the subsequent inverse scaling of the normalized data using the number of layers.

[0097] S4. Perform inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data.

[0098] In an embodiment of the present invention, in order to further regulate the feature amplitude distribution of normalized data in different levels and enhance the model's adaptive expression ability of inter-layer differences, the normalized data is inversely scaled according to the layer number information of the determined target layer position, and the overall feature value is compressed to a numerical range that is inversely proportional to the depth of the layer in which it is located. This can not only effectively control the activation intensity of deep features and prevent gradient explosion or saturation, but also give the model higher structural perception ability, providing a stable input basis for subsequent activation judgment or output processing.

[0099] In healthcare scenarios, when processing medical images (such as MRI, CT, and X-ray images), image data needs to be normalized and scaled to adapt to different machine learning models. Inverse scaling helps adjust the size and scale of data to different target layers (for example, feature extraction networks at different levels) to adapt it to different deep learning models. Scaling ensures that features at different levels are appropriately weighted, helping to improve the model's predictive accuracy.

[0100] In FinTech scenarios, credit scoring systems assess multiple financial factors (such as credit history, income level, and loan history), often with varying ranges and importance. When building models, inverse scaling can help adjust different data layers so that deep learning models can accurately capture the influence of each feature. For example, if the ranges of income data and debt data differ significantly, inverse scaling can help balance the influence of these two data types.

[0101] Figure 3 A flowchart of the inverse scaling process in a large model-based layer normalization scaling method provided by one embodiment of the present invention.

[0102] In an embodiment of the present invention, the step of inversely scaling the normalized data using the number of layers at the target layer position to obtain scaled data includes:

[0103] generating an inverse scaling factor according to the number of layers at the target layer position;

[0104] Scaling the normalized data one by one using the inverse scaling factor to obtain a plurality of scaling feature results;

[0105] All of the scaling feature results are aggregated into scaling data.

[0106] Specifically, the inverse scaling factor generated by the number of layers at the target layer position is: the smaller the number of layers (i.e., closer to the network input), the larger the scaling factor, which helps preserve more details; the larger the number of layers (i.e., closer to the output), the smaller the scaling factor, which makes the feature amplitude converge appropriately and reduces the oscillation of high-level features. The inverse scaling factor is calculated as follows:

[0107]

[0108] Where D represents the layer number of the target layer position.

[0109] The normalized eigenvalue is multiplied by the inverse scaling factor to obtain the scaled eigenvalue. In this way, the amplitude of the normalized features can be adjusted one by one to form several new scaled feature results. The amplitudes of features at different layer positions are adjusted in a targeted manner according to their importance and hierarchical depth, which helps the features to more reasonably express hierarchical semantic information in subsequent modeling and improve the accuracy of layer normalization in deep neural network models.

[0110] By inversely scaling the normalized data using the number of layers at the target layer position, the numerical range of the features can be dynamically adjusted according to the different levels of the features in the deep neural network, strengthening the importance of high-level features and suppressing redundant information at low levels, so that the scaled data is more in line with the model's requirements for distinguishing features at different semantic levels. Shallow features, because they come directly from the input, have a smaller range of variation and a larger scaling factor, and can retain more fine-grained information; while deep features have undergone multiple abstractions, a large number of layers, and a smaller scaling factor, so that high-level semantic information occupies an appropriate proportion of the overall features, preventing excessive expansion or distortion of the features. Ultimately, scaling data can improve the model's adaptability and convergence speed to complex structures, while enhancing the model's discriminative performance and robustness.

[0111] S5. Determine whether the target layer position is the preset output layer of the deep neural network model.

[0112] In this embodiment of the present invention, to ensure the accuracy and consistency of the model output, it is necessary to determine whether the target layer position of the current normalized data is the preset output layer. Based on the layer identification information in the model structure, it is checked whether the current target layer is consistent with the output layer of the model. If the layer identification information of the target layer matches the preset output layer identification, it means that the current normalized data is already in the output layer position; otherwise, the normalized data is still in the hidden layer and needs to be further activated before it can be used as output. This judgment process ensures that the model can flexibly adjust feature expression and avoid error accumulation during feature processing and output at different levels.

[0113] In healthcare scenarios, deep neural network models are often used for disease prediction, image analysis, or processing patient health data. For example, when diagnosing cancer, a neural network model might be used to extract features from medical images (such as X-rays or MRI scans) to predict whether a patient has cancer. For such tasks, determining whether the current target layer is the output layer is crucial, as the model's final diagnosis (e.g., the presence or absence of cancer) can only be accurately determined at the output layer.

[0114] In specific FinTech scenarios, such as credit risk assessment, deep neural network models are trained based on historical customer credit data (such as credit card repayment records and loan history). The feature extraction process in the hidden layer affects the intermediate results, but the model can only produce a final credit risk score after the data passes through the output layer. Outputting the results at the end of the intermediate layer can lead to erroneous credit assessments.

[0115] If the target layer position does not correspond to the preset output layer of the deep neural network model, then S6, use a preset activation function to activate the scaled data to obtain target text data.

[0116] In an embodiment of the present invention, if the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data needs to be further activated to convert it into an activation state suitable for the next layer of processing. In this process, the system will use a preset activation function (such as ReLU, Sigmoid, Tanh, etc.) to perform a nonlinear transformation on the scaled data. The activation function adjusts its numerical range and distribution according to the input scaled data so that the data can better adapt to the nonlinear processing requirements of the neural network, thereby enhancing the learning ability and expression ability of the model and improving the accuracy of layer normalization in the deep neural network model.

[0117] In healthcare scenarios, such as cancer detection, after passing through various intermediate layers but before entering the final output layer, activation functions (such as ReLU or Sigmoid) can be used to activate the data, enhancing the model's ability to nonlinearly map features. This helps the model better capture disease-related features and ensures accurate disease predictions. Especially when processing complex medical images, activation functions can help highlight key disease features.

[0118] In the specific context of fintech, such as financial fraud detection, neural networks extract features from transaction data through multiple layers. If the data has not yet reached the output layer after passing through the intermediate layers, activation functions (such as ReLU or Sigmoid) are used to convert the data into a format suitable for subsequent processing, ensuring that the final output can accurately determine whether the transaction is fraudulent. This activated data provides the nonlinear features required by the model, enabling it to better identify complex fraud patterns.

[0119] In an embodiment of the present invention, the step of activating the scaled data using a preset activation function to obtain target text data includes:

[0120] Performing weighted summation on the scaled data to obtain summed data;

[0121] Activating the summed data using a preset activation function to obtain an activation value of the summed data;

[0122] Determining whether the activation value is greater than a preset activation threshold;

[0123] If the activation value is greater than the activation threshold, retaining the summed data corresponding to the activation value greater than the activation threshold;

[0124] If the activation value is less than or equal to the activation threshold, deleting the summed data corresponding to the activation value that is less than or equal to the activation threshold;

[0125] Perform a reverse mapping operation on the retained activation values to obtain the target text data.

[0126] Specifically, the scaled data is weighted and summed, assigning a weight coefficient to each feature in the scaled data. Each feature is multiplied by its corresponding weight and then added together to produce a comprehensive sum. This weighted summation process maintains the overall feature distribution trend while highlighting features that have a greater impact on the final target text data generation.

[0127] The summed data is input into the preset activation function for nonlinear transformation. The activation function can be a common function such as ReLU, Sigmoid, Tanh, etc. The purpose is to introduce nonlinear factors so that the output features can better express complex semantic relationships, thereby obtaining the activation value corresponding to the target layer. The calculation formula is as follows:

[0128] A=ReLU(B)=max(0,B)

[0129] Among them, A represents the activation value, B represents the sum data, and ReLU represents the activation function that takes the maximum value of the input and 0 to ensure that the activation value is non-negative.

[0130] Based on the activation values obtained, a threshold is set to filter and extract or reconstruct the final target text data. A vector de-mapping operation is then performed on the filtered set of activated features. De-mapping involves converting the selected activated features back into the corresponding target text data using the embedding layer or decoder stored in the deep neural network. For example, a decoder network (such as a Transformer Decoder or RNN decoder) can be used to convert features into specific words, phrases, or sentences.

[0131] By using a preset activation function to activate the scaled data and combining it with activation threshold screening, we can effectively highlight the data parts with strong expressive ability or key semantic features in the deep neural network, avoid the interference of irrelevant or weak response information on the generation of target text, and further improve the accuracy, representativeness and semantic coherence of the generated text. At the same time, by reverse mapping the retained high-response activation values, we can also restore the target text that meets the expected semantics to the greatest extent while ensuring the refinement of information, realize efficient extraction and accurate reconstruction of the initial data, thereby greatly improving the reliability and effectiveness of the overall data processing and analysis.

[0132] If the target layer position corresponds to the preset output layer of the deep neural network model, then S7, the scaled data is used as the target text data.

[0133] In this embodiment of the present invention, if the target layer corresponds to the preset output layer of a deep neural network model, the scaled data is connected to the model's output layer to obtain the target text data. The scaling operation converts the data into a format that the model can efficiently process and understand, ensuring the accuracy and usability of the target text data. This process helps optimize subsequent output results, making model generation more accurate and effective.

[0134] In specific medical and health scenarios, by processing patients' health data, these data are scaled through a deep neural network model, and the scaled data can be used as target text data to generate disease prediction reports or personalized health management recommendations.

[0135] In the specific scenario of financial technology, in the application of intelligent investment advisors, investors' market behavior data, asset allocation, risk preference and other information are input into the deep neural network model and scaled. The output target text data may be an investment advice report. The investment advice report can provide personalized investment portfolio recommendations or adjustment strategies based on current market dynamics to help investors make decisions.

[0136] Using the scaled data as target text data can effectively improve the training efficiency and accuracy of deep neural network models. By normalizing or scaling the input data to a specific range, the model can converge faster, thereby improving the stability of learning. At the same time, scaling the data helps reduce the differences between features, making the model more consistent when processing different types of input. This data preprocessing method can not only improve the accuracy of layer normalization in deep neural network models, but also enhance their robustness in practical applications, facilitating better prediction and classification of target tasks.

[0137] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0138] like Figure 4 , which is a functional module diagram of a layer normalization and scaling device based on a large model provided by one embodiment of the present invention.

[0139] In the embodiment of the present disclosure, a layer normalization scaling device based on a large model is provided, and the layer normalization scaling device based on a large model corresponds one-to-one to the layer normalization scaling method based on a large model in the above embodiment. Figure 4As shown, the layer normalization and scaling device 100 based on a large model can be installed in an electronic device. According to the functions to be implemented, the layer normalization and scaling device 100 based on a large model includes a data screening module 101, a data normalization module 102, a position determination module 103, a data scaling module 104, a first judgment result module 105, and a second judgment result module 106. The functional modules are described in detail as follows:

[0140] The data screening module 101 is used to obtain text data to be analyzed, and use a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data;

[0141] A data normalization module 102 is used to perform layer normalization processing on the initial text data to obtain normalized data;

[0142] A position determination module 103 is used to determine the target layer position of the normalized data in the deep neural network model;

[0143] A data scaling module 104 is configured to inversely scale the normalized data using the number of layers at the target layer position to obtain scaled data, and determine whether the target layer position corresponds to a preset output layer of the deep neural network model;

[0144] A first judgment result module 105 is configured to activate the scaled data using a preset activation function to obtain target text data if the target layer position does not correspond to the preset output layer of the deep neural network model;

[0145] The second judgment result module 106 is used to use the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

[0146] In one embodiment, when the data screening module 101 performs target data screening on the text data to be analyzed using a preset deep neural network model to obtain initial text data, it is configured to:

[0147] Standardizing the format of the text data to be analyzed to obtain standard text data;

[0148] Converting the standard text data into a text embedding vector using an embedding layer of a preset deep neural network model;

[0149] Encoding the text embedding vector to obtain a text embedding code;

[0150] Performing contextual semantic modeling on the text embedding code to obtain a semantic coding vector;

[0151] Obtaining a feature scoring function of the deep neural network model, performing feature analysis on the standard text data according to the semantic encoding vector and the feature scoring function to obtain a scoring vector;

[0152] Determine whether the scoring vector is greater than a preset score threshold, and filter out the scoring vectors that are greater than the score threshold;

[0153] The filtered scoring vectors are used as initial text data.

[0154] In one embodiment, when the data screening module 101 performs target data screening on the text data to be analyzed using a preset deep neural network model to obtain initial text data, it is configured to:

[0155] Embedding the text into a code sequence as an input text sequence;

[0156] Performing position encoding on the input text sequence to obtain a text update sequence;

[0157] Capturing the contextual dependencies of the text update sequence to obtain semantic relationships;

[0158] A semantic encoding vector of the text embedding encoding is generated according to the semantic relationship.

[0159] In one embodiment, when performing layer normalization processing on the initial text data to obtain normalized data, the data normalization module 102 is configured to:

[0160] Acquiring feature dimensions of the initial text data;

[0161] Generating the mean and standard deviation of the initial text data according to the feature dimension;

[0162] Performing normal distribution transformation on the initial text data using the mean and the standard deviation to obtain normalized features;

[0163] All the normalized features are aggregated into normalized data.

[0164] In one embodiment, when determining the target layer position of the normalized data in the deep neural network model, the position determination module 103 is configured to:

[0165] Indexing and marking each layer structure of the deep neural network model to obtain layer identification information of each layer structure;

[0166] Obtaining layer index information of the layer where the normalized data is located;

[0167] The layer index information is compared with the layer identification information one by one to obtain the target layer position of the normalized data.

[0168] In one embodiment, when the data scaling module 104 performs inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data, it is configured to:

[0169] generating an inverse scaling factor according to the number of layers at the target layer position;

[0170] Scaling the normalized data one by one using the inverse scaling factor to obtain a plurality of scaling feature results;

[0171] All of the scaling feature results are aggregated into scaling data.

[0172] In one embodiment, when the first determination result module 105 activates the zoomed data using a preset activation function to obtain the target text data, it is configured to:

[0173] Performing weighted summation on the scaled data to obtain summed data;

[0174] Activating the summed data using a preset activation function to obtain an activation value of the summed data;

[0175] Determining whether the activation value is greater than a preset activation threshold;

[0176] If the activation value is greater than the activation threshold, retaining the summed data corresponding to the activation value greater than the activation threshold;

[0177] If the activation value is less than or equal to the activation threshold, deleting the summed data corresponding to the activation value that is less than or equal to the activation threshold;

[0178] Perform a reverse mapping operation on the retained activation values to obtain the target text data.

[0179] In the present invention, a layer normalization scaling device based on a large model is provided. First, the present invention obtains text data to be analyzed, and uses a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data. This not only improves the automation and intelligence level of data screening, avoids the subjectivity and inefficiency problems caused by manual rule screening, but also can effectively reduce the scale of data for subsequent processing. The initial text data is layer normalized to obtain normalized data, which not only speeds up the model convergence speed, but also improves the stability of the training process and the accuracy of the results. The target layer position of the normalized data in the deep neural network model is determined, so that the subsequent inverse scaling operation based on the number of layers has a clear and accurate positioning basis. The layer normalization of the target layer position is used to determine the target layer position. The normalized data is inversely scaled to obtain scaled data. According to the different levels of the features in the deep neural network, the numerical range of the features is dynamically adjusted to strengthen the importance of high-level features and suppress low-level redundant information, so that the scaled data is more in line with the model's requirements for distinguishing features at different semantic levels. It is determined whether the target layer position corresponds to the preset output layer of the deep neural network model. If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data, which can effectively improve the accuracy of layer normalization in the deep neural network model. For the specific definition of a layer normalization scaling device based on a large model, please refer to the definition of a layer normalization scaling method based on a large model above, which will not be repeated here. Each module in the above-mentioned layer normalization scaling device based on a large model can be implemented in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in or independent of the processor in the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0180] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a layer normalization scaling method based on a large model.

[0181] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a layer normalization scaling method based on a large model.

[0182] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0183] Obtaining text data to be analyzed, and using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data;

[0184] Performing layer normalization processing on the initial text data to obtain normalized data;

[0185] Determining a target layer position of the normalized data in the deep neural network model;

[0186] Performing inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data, and determining whether the target layer position corresponds to a preset output layer of the deep neural network model;

[0187] If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data;

[0188] If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data.

[0189] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and actual implementation may employ other division methods.

[0190] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0191] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0192] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0193] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0194] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can implement:

[0195] Obtaining text data to be analyzed, and using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data;

[0196] Performing layer normalization processing on the initial text data to obtain normalized data;

[0197] Determining a target layer position of the normalized data in the deep neural network model;

[0198] Performing inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data, and determining whether the target layer position corresponds to a preset output layer of the deep neural network model;

[0199] If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data;

[0200] If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data.

[0201] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0202] The computer-readable storage medium may also store at least one computer-executable program / instruction, such as a computer-readable instruction. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0203] In addition, the computer device may also include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.).

[0204] The processor can communicate with external devices via an I / O bus via a wired or wireless network.

[0205] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0206] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0207] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0208] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the above-mentioned module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0209] It should be noted that, in this disclosure, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element limited by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0210] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

[0211] It should be noted that if software tools or components other than those of our company appear in the embodiments of this application, they are only used for illustration and do not represent actual use.

Claims

1. A layer normalization scaling method based on a large model, characterized in that: The method comprises: Obtaining text data to be analyzed, and using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data; Performing layer normalization processing on the initial text data to obtain normalized data; Determining a target layer position of the normalized data in the deep neural network model; Performing inverse scaling on the normalized data using the number of layers at the target layer position to obtain scaled data, and determining whether the target layer position corresponds to a preset output layer of the deep neural network model; If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data; If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data.

2. The layer normalization scaling method based on a large model according to claim 1, characterized in that The method of using a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data includes: Standardizing the format of the text data to be analyzed to obtain standard text data; Converting the standard text data into a text embedding vector using an embedding layer of a preset deep neural network model; Encoding the text embedding vector to obtain a text embedding code; Performing contextual semantic modeling on the text embedding code to obtain a semantic coding vector; Obtaining a feature scoring function of the deep neural network model, performing feature analysis on the standard text data according to the semantic encoding vector and the feature scoring function to obtain a scoring vector; Determine whether the scoring vector is greater than a preset score threshold, and filter out the scoring vectors that are greater than the score threshold; The filtered scoring vectors are used as initial text data.

3. The layer normalization scaling method based on a large model according to claim 2, characterized in that The performing contextual semantic modeling on the text embedding code to obtain a semantic coding vector includes: Embedding the text into a code sequence as an input text sequence; Performing position encoding on the input text sequence to obtain a text update sequence; Capturing the contextual dependencies of the text update sequence to obtain semantic relationships; A semantic encoding vector of the text embedding encoding is generated according to the semantic relationship.

4. The large model-based layer normalization scaling method according to claim 1, wherein: The performing layer normalization processing on the initial text data to obtain normalized data includes: Acquiring feature dimensions of the initial text data; Generating the mean and standard deviation of the initial text data according to the feature dimension; Performing normal distribution transformation on the initial text data using the mean and the standard deviation to obtain normalized features; All the normalized features are aggregated into normalized data.

5. The layer normalization scaling method based on a large model according to claim 1, characterized in that Determining a target layer position of the normalized data in the deep neural network model includes: Indexing and marking each layer structure of the deep neural network model to obtain layer identification information of each layer structure; Obtaining layer index information of the layer where the normalized data is located; The layer index information is compared with the layer identification information one by one to obtain the target layer position of the normalized data.

6. The layer normalization scaling method based on a large model according to claim 1, characterized in that The step of inversely scaling the normalized data using the number of layers at the target layer position to obtain scaled data includes: generating an inverse scaling factor according to the number of layers at the target layer position; Scaling the normalized data one by one using the inverse scaling factor to obtain a plurality of scaling feature results; All of the scaling feature results are aggregated into scaling data.

7. The large model-based layer normalization scaling method according to claim 1, wherein: The step of activating the scaled data using a preset activation function to obtain target text data includes: Performing weighted summation on the scaled data to obtain summed data; Activating the summed data using a preset activation function to obtain an activation value of the summed data; Determining whether the activation value is greater than a preset activation threshold; If the activation value is greater than the activation threshold, retaining the summed data corresponding to the activation value greater than the activation threshold; If the activation value is less than or equal to the activation threshold, deleting the summed data corresponding to the activation value that is less than or equal to the activation threshold; Perform a reverse mapping operation on the retained activation values to obtain the target text data.

8. A layer normalization and scaling device based on a large model, characterized in that: The device comprises: A data screening module is used to obtain text data to be analyzed, and use a preset deep neural network model to perform target data screening on the text data to be analyzed to obtain initial text data; A data normalization module, configured to perform layer normalization processing on the initial text data to obtain normalized data; A position determination module, configured to determine a target layer position of the normalized data in the deep neural network model; A data scaling module is used to inversely scale the normalized data using the number of layers at the target layer position to obtain scaled data, and to determine whether the target layer position corresponds to a preset output layer of the deep neural network model; A first judgment result module is used to activate the scaled data using a preset activation function to obtain target text data if the target layer position does not correspond to the preset output layer of the deep neural network model; The second judgment result module is used to use the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform a large model-based layer normalization scaling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a large model-based layer normalization scaling method as described in any one of claims 1 to 7.

Images