Large model-based layer normalization scaling method, device, equipment and medium

Abstract

The application relates to the technical field of artificial intelligence, can be applied to a financial technology, medical health and other business system platform, and discloses a layer normalization scaling method, device, equipment and medium based on a large model, the method comprises the following steps: obtaining text data to be analyzed, using a preset deep neural network model to perform target data screening on the text data to be analyzed, obtaining initial text data, performing layer normalization on the initial text data, obtaining normalized data, determining a target layer position of the normalized data in the deep neural network model, using the number of layers of the target layer position to perform inverse proportional scaling on the normalized data, obtaining scaling data, if the target layer position does not correspond to a preset output layer, using a preset activation function to activate the scaling data, obtaining target text data, and if the target layer position corresponds to the preset output layer, taking the scaling data as the target text data. The application improves the layer normalization accuracy in the deep neural network model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a layer normalization scaling method, apparatus, device, and medium based on a large model. Background Technology

[0002] Layer normalization is a common normalization technique used in neural networks. It aims to accelerate the training process and improve model stability by normalizing the input of each layer. Layer normalization is particularly crucial in large-scale deep learning models because it effectively reduces internal covariate bias, improving model performance during training. To address the computational complexity and resource demands of large models, researchers have proposed scaling methods for layer normalization. However, in today's mainstream large models, deep layer normalization often fails to achieve the desired results with traditional scaling methods, exhibiting low accuracy and with nearly half of the layers performing worse than expected.

[0003] In the healthcare field, deep learning models are commonly used for disease diagnosis, personalized treatment recommendations, and predictive analytics. Due to the complexity and high dimensionality of medical data, the training efficiency and accuracy of these models are crucial. However, as the number of layers in a model increases, the performance of deeper layers often falls short of expectations. Directly removing earlier layers can lead to a sharp decline in model performance, resulting in reduced accuracy.

[0004] In the fintech sector, deep learning models are widely used in tasks such as credit assessment, risk management, and stock market forecasting. However, due to the volatility and complexity of financial data, model training and optimization also face significant challenges. Removing deep or intermediate layers may not significantly impact the overall model performance, forcing large financial models to reassess the contribution of different layers to the task, leading to reduced model accuracy.

[0005] In summary, in today's mainstream large-scale models (such as Llama, Mistral, DeepSeek, and Qwen), deep normalization often fails to achieve the expected results, with nearly half of the layers performing worse than anticipated. Removing early layers leads to a sharp drop in model performance, while removing deep layers does not. Intermediate and deep layers exhibit significant robustness to perturbations such as layer swapping and dropping. This prevents these layers from performing meaningful transformations and effectively contributing to training and representation learning, ultimately leading to reduced model accuracy.

[0006] Therefore, current technology suffers from low accuracy in layer normalization within deep neural network models. Summary of the Invention

[0007] This invention provides a layer normalization scaling method, apparatus, device, and medium based on large models, the main purpose of which is to solve the problem of low layer normalization accuracy in deep neural network models.

[0008] Firstly, to achieve the above objectives, the present invention provides a layer normalization scaling method based on a large model, comprising:

[0009] The text data to be analyzed is obtained, and the target data is filtered using a preset deep neural network model to obtain the initial text data.

[0010] The initial text data is subjected to layer normalization to obtain normalized data;

[0011] Determine the target layer position of the normalized data in the deep neural network model;

[0012] The normalized data is inversely scaled using the number of layers at the target layer position to obtain scaled data, and it is determined whether the target layer position corresponds to the preset output layer of the deep neural network model.

[0013] If the target layer position does not correspond to the preset output layer of the deep neural network model, then the scaled data is activated using a preset activation function to obtain the target text data.

[0014] If the target layer position corresponds to the preset output layer of the deep neural network model, then the scaled data is used as the target text data.

[0015] Secondly, the present invention also provides a layer normalization scaling device based on a large model, comprising:

[0016] The data filtering module is used to acquire the text data to be analyzed, and to use a preset deep neural network model to filter the text data to be analyzed to obtain the initial text data.

[0017] The data normalization module is used to perform layer normalization processing on the initial text data to obtain normalized data;

[0018] A position determination module is used to determine the target layer position of the normalized data in the deep neural network model;

[0019] The data scaling module is used to inversely scale the normalized data using the number of layers at the target layer position to obtain scaled data, and to determine whether the target layer position corresponds to the preset output layer of the deep neural network model.

[0020] The first judgment result module is used to activate the scaled data using a preset activation function if the target layer position does not correspond to the preset output layer of the deep neural network model, so as to obtain the target text data.

[0021] The second judgment result module is used to treat the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

[0022] Thirdly, the present invention also provides an electronic device, the electronic device comprising:

[0023] At least one processor; and,

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the layer normalization scaling method based on a large model as described above.

[0026] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described layer normalization scaling method based on a large model.

[0027] This invention acquires text data to be analyzed and uses a preset deep neural network model to filter the target data, obtaining initial text data. This not only improves the automation and intelligence of data filtering, avoiding the subjectivity and inefficiency of manual rule-based filtering, but also effectively reduces the data scale of subsequent processing. Layer normalization is performed on the initial text data to obtain normalized data, which not only accelerates model convergence but also improves the stability of the training process and the accuracy of the results. The target layer position of the normalized data in the deep neural network model is determined, providing a clear and precise positioning basis for subsequent inverse scaling operations based on the layer number. The normalized data is then inversely scaled using the layer number of the target layer position. Scaling is performed to obtain scaled data. Based on the different levels of features in the deep neural network, the numerical range of the features is dynamically adjusted to enhance the importance of high-level features and suppress redundant information in low-level layers. This makes the scaled data more consistent with the model's requirements for distinguishing features at different semantic levels. It is then determined whether the target layer position corresponds to the preset output layer of the deep neural network model. If the target layer position does not correspond to the preset output layer of the deep neural network model, the scaled data is activated using a preset activation function to obtain target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data. This can effectively improve the accuracy of layer normalization in the deep neural network model. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of an application environment for a layer normalization scaling method based on a large model, according to an embodiment of the present invention.

[0030] Figure 2 A flowchart illustrating a layer normalization scaling method based on a large model, provided in an embodiment of the present invention;

[0031] Figure 3 This is a flowchart illustrating the inverse scaling process in a layer normalization scaling method based on a large model, provided as an embodiment of the present invention.

[0032] Figure 4 A schematic diagram of a layer normalization scaling device based on a large model provided in an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of the structure of an electronic device that implements a layer normalization scaling method based on a large model, according to an embodiment of the present invention.

[0034] Figure 6 This is another structural schematic diagram of an electronic device that implements a layer normalization scaling method based on a large model, according to an embodiment of the present invention.

[0035] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0038] This application provides a layer normalization scaling method based on a large model. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the device provided in this application: a server, a terminal, etc. In other words, the layer normalization scaling method based on a large model can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0039] This invention provides a layer normalization scaling method based on a large model, which can be applied to applications such as... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain the text data to be analyzed through the client, and use a preset deep neural network model to filter the target data of the text data to obtain initial text data. This not only improves the automation and intelligence of data filtering and avoids the subjectivity and inefficiency caused by manual rule filtering, but also effectively reduces the data scale of subsequent processing. Layer normalization is performed on the initial text data to obtain normalized data, which not only speeds up model convergence but also improves the stability of the training process and the accuracy of the results. The target layer position of the normalized data in the deep neural network model is determined, providing a clear and accurate positioning basis for subsequent inverse scaling operations based on the layer number. The normalized data is then inversely scaled using the layer number of the target layer position to obtain the scaling factor. According to the invention, based on the different levels of features in the deep neural network, the numerical range of features is dynamically adjusted to enhance the importance of high-level features and suppress redundant information in low-level layers. This makes the scaled data more consistent with the model's requirements for distinguishing features at different semantic levels. The invention determines whether the target layer position corresponds to the preset output layer of the deep neural network model. If the target layer position does not correspond to the preset output layer, a preset activation function is used to activate the scaled data to obtain target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data. This effectively improves the accuracy of layer normalization in the deep neural network model. Finally, the target text data is output and fed back to the user client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0040] The following explanation of this invention relates to the present invention. This invention uses the number of layers at the target layer to inversely scale normalized data. This dynamically adjusts the numerical range of features based on their location within the deep neural network, enhancing the importance of higher-level features and suppressing redundant information in lower layers. This results in scaled data that better meets the model's requirements for distinguishing features at different semantic levels. Shallow features, directly derived from the input, have a smaller range of variation and a larger scaling factor, thus retaining more fine-grained information. Deep features, having undergone multiple abstractions and a larger number of layers, have a smaller scaling factor, ensuring that high-level semantic information occupies an appropriate proportion in the overall feature set, preventing excessive feature inflation or distortion. Ultimately, scaling the data improves the model's adaptability to complex structures and convergence speed, while also enhancing the model's discriminative performance and accuracy.

[0041] Reference Figure 2The diagram shown is a flowchart illustrating a layer normalization scaling method based on a large model according to an embodiment of the present invention. In this embodiment, the layer normalization scaling method based on a large model includes:

[0042] S1. Obtain the text data to be analyzed, and use a preset deep neural network model to filter the target data of the text data to be analyzed to obtain the initial text data.

[0043] In this embodiment of the invention, the text data to be analyzed includes, but is not limited to, log records, user input, comment data, customer service dialogues, emails, etc. By embedding word vectors, modeling contextual features, and calculating feature scoring functions in the text data to be analyzed, the model can identify and filter out semantic fragments or information units that are highly relevant to the target analysis task, thereby eliminating redundant content and irrelevant information, and finally extracting the initial text data for subsequent processing.

[0044] In specific healthcare scenarios, the textual data to be analyzed typically includes medical records, doctors' diagnostic reports, patient health records, drug information, academic literature, etc. Using deep learning models (such as BERT, LSTM, etc.) to filter key information in medical records, such as patient symptoms, diagnostic results, treatment plans, and drug prescriptions, helps medical personnel automatically extract important information from large amounts of electronic medical records, reducing the workload of manual searching and improving patient diagnosis and treatment efficiency.

[0045] In specific fintech scenarios, the text data to be analyzed may include news articles, financial reports, investment analyses, customer feedback, and social media comments. Using deep neural networks to filter and automatically analyze text data such as financial reports, quarterly financial statements, and company annual reports can extract key information such as financial condition, profitability, and risk points, helping to make more accurate investment decisions.

[0046] In this embodiment of the invention, the step of using a preset deep neural network model to filter the text data to be analyzed to obtain initial text data includes:

[0047] The text data to be analyzed is formatted to obtain standard text data;

[0048] The standard text data is converted into text embedding vectors using the embedding layer of a pre-defined deep neural network model.

[0049] The text embedding vector is encoded to obtain the text embedding code;

[0050] Contextual semantic modeling is performed on the text embedding encoding to obtain a semantic encoding vector;

[0051] Obtain the feature scoring function of the deep neural network model, and perform feature analysis on the standard text data based on the semantic encoding vector and the feature scoring function to obtain the scoring vector;

[0052] Determine whether the scoring vector is greater than a preset score threshold, and filter out scoring vectors that are greater than the score threshold;

[0053] The selected scoring vectors are used as the initial text data.

[0054] In detail, the text is encoded uniformly, for example, all text is converted to UTF-8 format to avoid garbled characters caused by different encodings, special characters, HTML tags, control characters or abnormal symbols are removed, meaningless content is cleaned up, punctuation marks are standardized (such as Chinese and English symbols are interchanged), and uppercase and lowercase, simplified and traditional Chinese characters are standardized to make the text structure more conducive to model understanding. Finally, the cleaned text is output in a standardized format according to a predetermined format as standard text data that can be directly processed in the subsequent modeling stage, thereby improving the overall data quality and algorithm performance.

[0055] The standard text data is segmented into words, subwords, or characters according to a set segmentation rule. Then, each word is converted into a corresponding index number through an encoding mechanism. These index numbers are input into an embedding layer, which maps each word to a text embedding vector representation by looking up a pre-trained embedding matrix. The vector dimension is typically between tens and hundreds. These vectors not only preserve the basic information of the words but also learn the semantic relationships between words during training. This makes semantically similar words closer together in the vector space, thus laying the foundation for subsequent semantic modeling and contextual analysis.

[0056] The initial text embedding vectors are input into a pre-defined encoding structure, such as a feedforward neural network, a convolutional network (CNN), or a simple linear transformation layer. Each vector undergoes dimensionality transformation or feature mapping to generate an intermediate representation with a unified format and certain semantic expressive power, i.e., text embedding encoding. Text embedding encoding not only inherits the basic semantic information in the original embedding, but also provides a clear and efficient input foundation for the subsequent context modeling stage, making semantic reasoning more accurate and stable.

[0057] In this embodiment of the invention, the step of performing contextual semantic modeling on the text embedding encoding to obtain a semantic encoding vector includes:

[0058] The text embedding encoding is used as the input text sequence;

[0059] The input text sequence is positionally encoded to obtain the text update sequence;

[0060] The contextual dependencies of the text update sequence are captured to obtain semantic relationships;

[0061] Generate the semantic encoding vector of the text embedding encoding based on the semantic relationship.

[0062] In detail, the text embedding encoding generated in the previous step is fed into the encoding module as the input text sequence. To preserve the positional information of words in the original sentence, positional encoding is first added to the input text sequence. By fusing the positional information with the original embedding vector, a text update sequence with positional information is obtained. Subsequently, contextual modeling is performed on the text update sequence. The internal structure of the model (such as self-attention mechanism, bidirectional recurrent neural network, etc.) is used to capture the dependencies between the preceding and following contexts of each word in the sequence, identifying the semantic relationships and mutual influences of each word in the whole sentence. Finally, based on the captured semantic relationships, semantic enhancement is performed on the embedding representation of each word, generating a semantic encoding vector of the text embedding encoding with contextual semantic understanding capabilities, thus laying the foundation for subsequent feature extraction and text analysis.

[0063] In detail, a pre-defined feature scoring function is called from the model. The feature scoring function is usually a weight calculation layer. The semantic encoding vector obtained in the previous step is input into the feature scoring function, and the calculation formula is as follows:

[0064] s i =f s (h i ) = w T *tanh(Wh i +b)

[0065] Among them, h i Let f represent the i-th semantic encoding vector. s S represents the feature scoring function. i Let w represent the i-th scoring vector. T Let W represent the transpose of the scoring weight vector, W represent the weight matrix, b represent the bias vector, and tanh represent the tangent function.

[0066] After obtaining the scoring vector corresponding to each text unit, the system sets a preset score threshold to distinguish which text units meet the importance requirements. If the scoring vector is greater than the score threshold, the text unit is considered to have high semantic value or feature representativeness in the overall text and is retained. If the scoring vector is less than or equal to the score threshold, it is not adopted. The original content corresponding to all text units with scoring vectors greater than the threshold is extracted and used as the initial text data for subsequent analysis or processing.

[0067] By utilizing a pre-defined deep neural network model to filter target data from the text data to be analyzed, this method can systematically complete format unification, semantic feature extraction, and importance assessment from large-scale raw data, thereby accurately selecting initial text data with high semantic value and representativeness. This approach not only improves the automation and intelligence of data filtering, avoiding the subjectivity and inefficiency of manual rule-based filtering, but also effectively reduces the scale of subsequent data processing, enhances the relevance and accuracy of model training or application, and significantly optimizes the overall quality and efficiency of data preprocessing and feature extraction.

[0068] S2. Perform layer normalization on the initial text data to obtain normalized data.

[0069] In this embodiment of the invention, for the initial text data that has been screened, in order to further improve the consistency of feature distribution and the stability of model processing, layer normalization technology is used to standardize the initial text data. First, the mean and variance of the representation vector of each initial text data in the deep neural network are calculated at the dimension level. Then, the mean is subtracted from each feature value and divided by the standard deviation to ensure that all feature components are distributed on the same scale, thereby eliminating the influence of internal covariate bias on the subsequent processing, and finally obtaining normalized data with stable values ​​and balanced structure.

[0070] In specific healthcare scenarios, within electronic medical records (EMRs), patient symptoms, historical medical records, and examination results are presented as text data. By performing layer normalization on this data, it can be ensured that the neural network can stably process different types of symptom and diagnostic data during training.

[0071] In specific fintech scenarios, data analysis in financial markets typically involves a large amount of news articles, social media content, and investor comments. By performing layer-normalization on this textual data, fluctuations caused by different sources or formats can be reduced, making sentiment analysis models more stable and accurate.

[0072] In this embodiment of the invention, the step of performing layer normalization processing on the initial text data to obtain normalized data includes:

[0073] Obtain the feature dimensions of the initial text data;

[0074] Generate the mean and standard deviation of the initial text data based on the aforementioned feature dimensions;

[0075] The initial text data is transformed into a normal distribution using the mean and the standard deviation to obtain normalized features;

[0076] All the normalized features are aggregated into normalized data.

[0077] In detail, the initial text data is processed by a deep neural network and transformed into a set of vectors with fixed feature representations. Each sample can be viewed as a multi-dimensional vector. The feature dimension refers to the number of elements contained in each vector. By detecting the shape of the initial text data matrix (e.g., the number of rows represents the number of samples, and the number of columns represents the number of features), the column number information is extracted as the feature dimension.

[0078] For each feature dimension, the mean and standard deviation are calculated based on the values ​​of all samples in the initial text data for that feature, as shown in the following formulas:

[0079]

[0080] Where, μ n Let x represent the mean, N represent the total number of samples in the initial text data, and x represent the mean. m,n σ represents the value of the m-th initial text data in the n-th feature dimension. n It represents the standard deviation.

[0081] For any feature value in the initial text data, first subtract the mean of the corresponding feature to eliminate positional bias caused by different value ranges between features; then divide by the standard deviation of the feature to unify the scale of each feature, so that each feature conforms to a standard normal distribution with a mean of 0 and a standard deviation of 1 after processing. Summarize the normalized features of all initial text data in each feature dimension according to the order of the initial text data and the feature order to obtain the complete normalized data matrix. The calculation formula is as follows:

[0082]

[0083] Where, x m,n μ represents the value of the m-th initial text data in the n-th feature dimension. n σ represents the mean. n It represents the standard deviation.

[0084] By performing layer-level normalization on the initial text data, numerical differences between different feature dimensions can be effectively eliminated, ensuring that each feature is processed on a uniform scale. This avoids convergence difficulties or training instability caused by imbalanced feature distribution during model training. First, the feature dimensions are obtained, and the mean and standard deviation are calculated for each feature dimension. Then, a normal distribution transformation is performed to ensure that each feature conforms to a standard normal distribution. Finally, all normalized features are aggregated into a unified normalized dataset. This not only accelerates model convergence but also improves the stability of the training process and the accuracy of the results.

[0085] S3. Determine the target layer position of the normalized data in the deep neural network model.

[0086] In this embodiment of the invention, the current network layer position of the data is identified based on the feature channel or representation path where the normalized data is located, combined with the model structure definition and the forward propagation tracking mechanism. For example, in a Transformer structure, the sublayer index where the normalization operation is located can be marked; in a CNN or RNN structure, it can be determined by recording the layer number or module node index corresponding to the current normalization operation. The layer position information will serve as the basis for subsequent scaling processing and activation determination, ensuring that the model has flexible intermediate feature extraction and adjustment capabilities in multi-layer processing structures.

[0087] In specific healthcare scenarios, during the analysis of electronic medical records, the model is used to predict whether a patient has a certain disease. Normalized medical record information includes: the patient's condition, medication records, etc. Determining the target layer position of the normalized data provides data support for subsequent judgments on whether it is located in the output layer and whether data diagnosis has been completed.

[0088] In specific fintech scenarios, such as credit score prediction, models are used to predict credit scores or the probability of a customer's default. Normalized data includes a customer's financial history, credit card spending records, etc. Determining the target layer position of the normalized data provides data support for subsequent judgments on whether it is located in the output layer and whether probability prediction has been completed.

[0089] In this embodiment of the invention, determining the target layer position of the normalized data in the deep neural network model includes:

[0090] Each layer of the deep neural network model is indexed and labeled to obtain the layer identifier information of each layer.

[0091] Obtain the layer index information of the layer where the normalized data is located;

[0092] The layer index information is compared with the layer identifier information one by one to obtain the target layer position of the normalized data.

[0093] In detail, following the order of layers in the deep neural network model, each layer is assigned a unique identifier number, and its corresponding layer type, parameter settings, and other basic information are recorded to form layer identification information. First, all layers of the deep neural network (such as embedding layers, encoding layers, attention layers, fully connected layers, etc.) are traversed. Based on their position in the model, each layer is assigned an ordered index value. Simultaneously, the structural attributes of the layer, such as input / output dimensions, activation function type, and number of parameters, are extracted as layer identification information. By establishing complete layer identification information, clear management and tracking of the network structure can be achieved.

[0094] Generating layer index information from normalized data refers to inferring the index numbers of each layer in a deep neural network based on the source and feature structure of the normalized data, thereby forming the layer index information corresponding to the normalized data.

[0095] These layer indexes are compared one by one with the layer identifiers obtained through indexing. The comparison primarily relies on key attributes such as index number, layer type, or feature morphology to confirm which layer of the network structure the normalized data originates from. This step-by-step comparison accurately locates the target layer corresponding to the normalized data, providing a basis for subsequent data processing, feature analysis, or model adjustment for specific layers. This ensures a clear and traceable correspondence between normalized features and the neural network structure, improving the accuracy of data use and the controllability of operations.

[0096] By indexing and labeling each layer of the deep neural network model and comparing it with normalized data, the target layer position corresponding to the normalized data can be accurately determined. This provides a clear and precise basis for subsequent inverse scaling operations based on the number of layers. The determined target layer position provides data support for subsequent inverse scaling of normalized data using the number of layers.

[0097] S4. Scale the normalized data inversely using the number of layers at the target layer position to obtain scaled data.

[0098] In this embodiment of the invention, in order to further regulate the feature amplitude distribution of normalized data in different layers and enhance the model's adaptive expression ability to inter-layer differences, the normalized data is subjected to inverse scaling based on the layer number information of the determined target layer position. The feature values ​​are compressed as a whole to a numerical range inversely proportional to the depth of the layer in which they are located. This not only effectively controls the activation intensity of deep features and prevents gradient explosion or saturation, but also gives the model a higher structural awareness ability, providing a stable input basis for subsequent activation judgment or output processing.

[0099] In specific healthcare scenarios, the processing of medical images (such as MRI, CT, and X-ray images) requires normalization and scaling to adapt to different machine learning models. In this process, inverse scaling helps adjust the size and scale of the data according to different target layers (e.g., different levels of feature extraction networks), making it suitable for different deep learning models. Scaling ensures that features at different levels receive appropriate weights, contributing to improved model prediction accuracy.

[0100] In specific fintech scenarios, credit scoring systems need to assess multiple financial factors (such as credit history, income level, and loan records), which typically have different numerical ranges and levels of importance. When building a model, inverse scaling can help adjust different data layers so that the deep learning model can accurately capture the influence of each feature. For example, the numerical ranges of income data and debt data differ significantly; inverse scaling helps balance the influence of these data.

[0101] Figure 3 This is a flowchart illustrating the inverse scaling process in a layer normalization scaling method based on a large model, provided as an embodiment of the present invention.

[0102] In this embodiment of the invention, the step of inversely scaling the normalized data using the number of layers at the target layer position to obtain scaled data includes:

[0103] Generate an inverse scaling factor based on the layer number at the target layer location;

[0104] The normalized data is scaled one by one using the inverse scaling factor to obtain several scaled feature results;

[0105] All the scaling feature results are aggregated into scaling data.

[0106] In detail, the inverse scaling factor generated by the layer number at the target layer location is as follows: the smaller the layer number (i.e., closer to the network input), the larger the scaling factor, which helps to retain more details; the larger the layer number (i.e., closer to the output), the smaller the scaling factor, which allows the feature amplitude to converge moderately and reduces oscillations in higher-level features. The inverse scaling factor is calculated as follows:

[0107]

[0108] Where D represents the layer number of the target layer location.

[0109] Multiplying the normalized feature value by an inverse scaling factor yields the scaled feature value. In this way, the amplitude of the normalized feature can be adjusted one by one to form several new scaled feature results. Features at different layer positions are adjusted in a targeted manner according to their importance and layer depth, which helps the features to more reasonably express hierarchical semantic information in subsequent modeling and improves the accuracy of layer normalization in deep neural network models.

[0110] By inversely scaling normalized data according to the layer number of the target layer, the numerical range of features can be dynamically adjusted based on their position within the deep neural network. This strengthens the importance of higher-level features and suppresses redundant information in lower layers, making the scaled data more suitable for the model's ability to distinguish features at different semantic levels. Shallow features, directly derived from the input, have a smaller range of variation and a larger scaling factor, thus retaining more fine-grained information. Deep features, having undergone multiple abstractions and a larger number of layers, have a smaller scaling factor, ensuring that high-level semantic information occupies an appropriate proportion in the overall feature set, preventing feature over-inflation or distortion. Ultimately, scaling the data improves the model's adaptability to complex structures and convergence speed, while also enhancing its discriminative performance and robustness.

[0111] S5. Determine whether the target layer position is the preset output layer of the deep neural network model.

[0112] In this embodiment of the invention, to ensure the accuracy and consistency of the model output, it is necessary to determine whether the target layer position of the current normalized data is a preset output layer. Based on the layer identifier information in the model structure, it is checked whether the current target layer is consistent with the model's output layer. If the layer identifier information of the target layer matches the preset output layer identifier, it indicates that the current normalized data is already in the output layer position; otherwise, the normalized data is still in a hidden layer and needs further activation processing before it can be used as output. This judgment process ensures that the model can flexibly adjust feature representation and avoid error accumulation during feature processing and output at different levels.

[0113] In specific healthcare scenarios, deep neural network models are frequently used for disease prediction, image analysis, or processing of patient health data. For example, in diagnosing cancer, a neural network model might be used to extract features from medical images (such as X-rays or MRI scans) to predict whether a patient has cancer. For such tasks, determining whether the current target layer is an output layer is crucial, because only when it is an output layer can the model accurately derive its final diagnostic result (such as the presence or absence of cancer).

[0114] In specific fintech scenarios, particularly in credit risk assessment, deep neural network models are trained based on a customer's historical credit data (such as credit card repayment records and loan history). The feature extraction process in the hidden layers affects intermediate results, but the model can only provide a final credit risk score when the data passes through the output layer. Outputting results at the end of the intermediate layers could lead to incorrect credit assessments.

[0115] If the target layer position does not correspond to the preset output layer of the deep neural network model, then S6, activate the scaled data using a preset activation function to obtain the target text data.

[0116] In this embodiment of the invention, if the target layer position does not correspond to the preset output layer of the deep neural network model, further activation processing is required on the scaled data to transform it into an activation state suitable for the next layer. During this process, the system uses preset activation functions (such as ReLU, Sigmoid, Tanh, etc.) to perform a non-linear transformation on the scaled data. The activation function adjusts its numerical range and distribution according to the input scaled data, enabling the data to better adapt to the non-linear processing requirements of the neural network, thereby enhancing the model's learning and expressive abilities and improving the normalization accuracy of layers in the deep neural network model.

[0117] In specific healthcare scenarios, such as cancer detection, after passing through various intermediate layers but before reaching the final output layer, activation functions (such as ReLU or Sigmoid) can be used to activate the data, enhancing the model's ability to non-linearly map features. This helps the model better capture disease-related features, ensuring accurate disease predictions. Especially when processing complex medical images, activation functions can help highlight key disease features.

[0118] In specific fintech scenarios, such as financial fraud detection, neural networks extract features from transaction data through multiple layers. If the data hasn't reached the output layer after passing through intermediate layers, activation functions (such as ReLU or Sigmoid) are used to transform the data into a format suitable for subsequent processing, ensuring that the final output can accurately determine whether a transaction is fraudulent. The activated data provides the non-linear features required by the model, enabling it to better identify complex fraud patterns.

[0119] In this embodiment of the invention, the step of activating the scaled data using a preset activation function to obtain target text data includes:

[0120] The scaled data is weighted and summed to obtain the summed data;

[0121] The summed data is activated using a preset activation function to obtain the activation value of the summed data;

[0122] Determine whether the activation value is greater than a preset activation threshold;

[0123] If the activation value is greater than the activation threshold, then the summation data corresponding to the activation values ​​that are greater than the activation threshold will be retained;

[0124] If the activation value is less than or equal to the activation threshold, then the summation data corresponding to the activation values ​​that are less than or equal to the activation threshold will be deleted.

[0125] The retained activation values ​​are reverse-mapped to obtain the target text data.

[0126] In detail, a weighted summation process is performed on the scaled data. A corresponding weight coefficient is assigned to each feature in the scaled data, and the products of each feature and its corresponding weight are multiplied and summed to obtain a comprehensive sum. This weighted summation process can highlight features that have a more significant impact on the generation of the final target text data while maintaining the overall feature distribution trend.

[0127] The summed data is input into a preset activation function for nonlinear transformation. The activation function can be a common function such as ReLU, Sigmoid, or Tanh. The purpose is to introduce nonlinear factors so that the output features can better express complex semantic relationships, thereby obtaining the activation value corresponding to the target layer. The calculation formula is as follows:

[0128] A = ReLU(B) = max(0,B)

[0129] Where A represents the activation value, B represents the sum of data, and ReLU represents the activation function that takes the maximum value of the input and 0, ensuring that the activation value is non-negative.

[0130] Based on the obtained activation values, the final target text data is extracted or reconstructed by setting a threshold for filtering. Then, a vector inverse mapping operation is performed on the filtered set of activation features. Inverse mapping refers to using the embedding layers or decoders stored in a deep neural network to transform the selected activation features back into the corresponding target text data. For example, a decoder network (such as a Transformer Decoder, RNN decoder, etc.) can be used to restore the features to specific words, phrases, or sentences.

[0131] By using a preset activation function to activate scaled data and combining it with activation threshold filtering, the data parts in the deep neural network with strong expressive power or key semantic features can be effectively highlighted, avoiding interference from irrelevant or weak response information in the generation of target text. This further improves the accuracy, representativeness, and semantic coherence of the generated text. At the same time, by inverse mapping the retained high-response activation values, the target text that conforms to the expected semantics can be restored to the greatest extent while ensuring information refinement. This achieves efficient extraction and accurate reconstruction of the initial data, thereby significantly improving the reliability and effectiveness of the overall data processing and analysis.

[0132] If the target layer position corresponds to the preset output layer of the deep neural network model, then S7, the scaled data is used as the target text data.

[0133] In this embodiment of the invention, if the target layer corresponds to a preset output layer of a deep neural network model, the scaled data is interfaced with the model's output layer to obtain the target text data. The scaling operation transforms the data into a format that the model can efficiently process and understand, ensuring the accuracy and usability of the target text data. This process helps optimize subsequent output results, making the model's generation more accurate and effective.

[0134] In specific healthcare scenarios, by processing patients' health data, which is then scaled using deep neural network models, the scaled data can be used as target text data to generate disease prediction reports or personalized health management recommendations.

[0135] In specific fintech scenarios, such as the application of robo-advisors, investors' market behavior data, asset allocation, risk preferences, and other information are input into a deep neural network model. After scaling, the output target text data may be an investment advice report. This report can provide personalized portfolio recommendations or adjust strategies based on current market dynamics to help investors make decisions.

[0136] Using scaled data as target text data can effectively improve the training efficiency and accuracy of deep neural network models. By normalizing or scaling the input data to a specific range, the model can converge faster, thereby improving learning stability. Simultaneously, scaling the data helps reduce differences between features, making the model more consistent in handling different types of inputs. This data preprocessing method not only improves the normalization accuracy of layers in deep neural network models but also enhances their robustness in practical applications, facilitating better prediction and classification of target tasks.

[0137] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0138] like Figure 4 The diagram shown is a functional block diagram of a layer normalization scaling device based on a large model provided in an embodiment of the present invention.

[0139] In this embodiment of the disclosure, a layer normalization scaling device based on a large model is provided, which corresponds one-to-one with the layer normalization scaling method based on a large model in the above embodiments. For example... Figure 4As shown, this layer normalization and scaling device 100 based on a large model can be installed in an electronic device. According to its functions, the layer normalization and scaling device 100 based on a large model includes a data filtering module 101, a data normalization module 102, a position determination module 103, a data scaling module 104, a first judgment result module 105, and a second judgment result module 106. Detailed descriptions of each functional module are as follows:

[0140] The data filtering module 101 is used to acquire the text data to be analyzed and to perform target data filtering on the text data to be analyzed using a preset deep neural network model to obtain initial text data.

[0141] The data normalization module 102 is used to perform layer normalization processing on the initial text data to obtain normalized data;

[0142] The position determination module 103 is used to determine the target layer position of the normalized data in the deep neural network model;

[0143] The data scaling module 104 is used to scale the normalized data inversely using the number of layers at the target layer position to obtain scaled data, and to determine whether the target layer position corresponds to the preset output layer of the deep neural network model.

[0144] The first judgment result module 105 is used to activate the scaled data using a preset activation function if the target layer position does not correspond to the preset output layer of the deep neural network model, so as to obtain the target text data.

[0145] The second judgment result module 106 is used to treat the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

[0146] In one embodiment, when the data filtering module 101 performs target data filtering on the text data to be analyzed using a preset deep neural network model to obtain initial text data, it is used to:

[0147] The text data to be analyzed is formatted to obtain standard text data;

[0148] The standard text data is converted into text embedding vectors using the embedding layer of a pre-defined deep neural network model.

[0149] The text embedding vector is encoded to obtain the text embedding code;

[0150] Contextual semantic modeling is performed on the text embedding encoding to obtain a semantic encoding vector;

[0151] Obtain the feature scoring function of the deep neural network model, and perform feature analysis on the standard text data based on the semantic encoding vector and the feature scoring function to obtain the scoring vector;

[0152] Determine whether the scoring vector is greater than a preset score threshold, and filter out scoring vectors that are greater than the score threshold;

[0153] The selected scoring vectors are used as the initial text data.

[0154] In one embodiment, when the data filtering module 101 performs target data filtering on the text data to be analyzed using a preset deep neural network model to obtain initial text data, it is used to:

[0155] The text embedding encoding is used as the input text sequence;

[0156] The input text sequence is positionally encoded to obtain the text update sequence;

[0157] The contextual dependencies of the text update sequence are captured to obtain semantic relationships;

[0158] Generate the semantic encoding vector of the text embedding encoding based on the semantic relationship.

[0159] In one embodiment, when the data normalization module 102 performs layer normalization processing on the initial text data to obtain normalized data, it is used to:

[0160] Obtain the feature dimensions of the initial text data;

[0161] Generate the mean and standard deviation of the initial text data based on the aforementioned feature dimensions;

[0162] The initial text data is transformed into a normal distribution using the mean and the standard deviation to obtain normalized features;

[0163] All the normalized features are aggregated into normalized data.

[0164] In one embodiment, when the position determination module 103 performs the task of determining the target layer position of the normalized data in the deep neural network model, it is used to:

[0165] Each layer of the deep neural network model is indexed and labeled to obtain the layer identifier information of each layer.

[0166] Obtain the layer index information of the layer where the normalized data is located;

[0167] The layer index information is compared with the layer identifier information one by one to obtain the target layer position of the normalized data.

[0168] In one embodiment, when the data scaling module 104 performs inverse scaling of the normalized data using the number of layers at the target layer position to obtain scaled data, it is used to:

[0169] Generate an inverse scaling factor based on the layer number at the target layer location;

[0170] The normalized data is scaled one by one using the inverse scaling factor to obtain several scaled feature results;

[0171] All the scaling feature results are aggregated into scaling data.

[0172] In one embodiment, when the first determination result module 105 performs the activation of the scaled data using a preset activation function to obtain the target text data, it is used to:

[0173] The scaled data is weighted and summed to obtain the summed data;

[0174] The summed data is activated using a preset activation function to obtain the activation value of the summed data;

[0175] Determine whether the activation value is greater than a preset activation threshold;

[0176] If the activation value is greater than the activation threshold, then the summation data corresponding to the activation values ​​that are greater than the activation threshold will be retained;

[0177] If the activation value is less than or equal to the activation threshold, then the summation data corresponding to the activation values ​​that are less than or equal to the activation threshold will be deleted.

[0178] The retained activation values ​​are reverse-mapped to obtain the target text data.

[0179] In this invention, a layer normalization scaling device based on a large model is first described. The invention acquires text data to be analyzed and uses a preset deep neural network model to perform target data filtering on the text data to obtain initial text data. This not only improves the automation and intelligence of data filtering, avoiding the subjectivity and inefficiency of manual rule-based filtering, but also effectively reduces the data scale of subsequent processing. Layer normalization processing is then performed on the initial text data to obtain normalized data, which not only accelerates model convergence but also improves the stability of the training process and the accuracy of the results. The target layer position of the normalized data in the deep neural network model is determined, providing a clear and accurate positioning basis for subsequent inverse scaling operations based on the number of layers. The layer normalization process is then used to determine the target layer position. The normalized data is inversely scaled to obtain scaled data. Based on the different levels of the features in the deep neural network, the numerical range of the features is dynamically adjusted to enhance the importance of higher-level features and suppress redundant information in lower levels. This makes the scaled data more consistent with the model's requirements for distinguishing features at different semantic levels. It is determined whether the target layer position corresponds to the preset output layer of the deep neural network model. If the target layer position does not correspond to the preset output layer of the deep neural network model, a preset activation function is used to activate the scaled data to obtain target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, the scaled data is used as the target text data. This effectively improves the accuracy of layer normalization in the deep neural network model. Specific limitations of a layer normalization scaling device based on a large model can be found in the limitations of a layer normalization scaling method based on a large model described above, and will not be repeated here. Each module in the above-described layer normalization scaling device based on a large model can be implemented entirely or partially through software, hardware, or a combination thereof. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or they can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above modules.

[0180] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements server-side functions or steps of a layer normalization scaling method based on a large model.

[0181] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of a layer normalization scaling method based on a large model.

[0182] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0183] The text data to be analyzed is obtained, and the target data is filtered using a preset deep neural network model to obtain the initial text data.

[0184] The initial text data is subjected to layer normalization to obtain normalized data;

[0185] Determine the target layer position of the normalized data in the deep neural network model;

[0186] The normalized data is inversely scaled using the number of layers at the target layer position to obtain scaled data, and it is determined whether the target layer position corresponds to the preset output layer of the deep neural network model.

[0187] If the target layer position does not correspond to the preset output layer of the deep neural network model, then the scaled data is activated using a preset activation function to obtain the target text data.

[0188] If the target layer position corresponds to the preset output layer of the deep neural network model, then the scaled data is used as the target text data.

[0189] In the several embodiments provided by this invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0190] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0191] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0192] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0193] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.

[0194] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0195] The text data to be analyzed is obtained, and the target data is filtered using a preset deep neural network model to obtain the initial text data.

[0196] The initial text data is subjected to layer normalization to obtain normalized data;

[0197] Determine the target layer position of the normalized data in the deep neural network model;

[0198] The normalized data is inversely scaled using the number of layers at the target layer position to obtain scaled data, and it is determined whether the target layer position corresponds to the preset output layer of the deep neural network model.

[0199] If the target layer position does not correspond to the preset output layer of the deep neural network model, then the scaled data is activated using a preset activation function to obtain the target text data.

[0200] If the target layer position corresponds to the preset output layer of the deep neural network model, then the scaled data is used as the target text data.

[0201] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0202] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0203] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0204] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0205] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0206] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0207] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0208] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0209] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0210] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0211] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

Claims

1. A layer normalization scaling method based on a large model, characterized in that, The method includes: The text data to be analyzed is obtained, and the target data is filtered using a preset deep neural network model to obtain the initial text data. The initial text data is subjected to layer normalization to obtain normalized data; Determine the target layer position of the normalized data in the deep neural network model; The normalized data is inversely scaled using the number of layers at the target layer position to obtain scaled data, and it is determined whether the target layer position corresponds to the preset output layer of the deep neural network model. If the target layer position does not correspond to the preset output layer of the deep neural network model, then the scaled data is activated using a preset activation function to obtain the target text data. If the target layer position corresponds to the preset output layer of the deep neural network model, then the scaled data is used as the target text data; The step of using a preset deep neural network model to filter target data of the text data to be analyzed to obtain initial text data includes: unifying the format of the text data to be analyzed to obtain standard text data; converting the standard text data into text embedding vectors using the embedding layer of the preset deep neural network model; encoding the text embedding vectors to obtain text embedding codes; performing contextual semantic modeling on the text embedding codes to obtain semantic encoding vectors; obtaining the feature scoring function of the deep neural network model; performing feature analysis on the standard text data based on the semantic encoding vectors and the feature scoring function to obtain scoring vectors; determining whether the scoring vectors are greater than a preset score threshold, and filtering out scoring vectors greater than the score threshold; and using the filtered scoring vectors as initial text data. Determining the target layer position of the normalized data in the deep neural network model includes: indexing and labeling each layer structure of the deep neural network model to obtain layer identifier information for each layer structure; obtaining layer index information of the layer where the normalized data is located; and comparing the layer index information with the layer identifier information one by one to obtain the target layer position of the normalized data.

2. The layer normalization scaling method based on a large model as described in claim 1, characterized in that, The step of performing contextual semantic modeling on the text embedding encoding to obtain a semantic encoding vector includes: The text embedding encoding is used as the input text sequence; The input text sequence is positionally encoded to obtain the text update sequence; The contextual dependencies of the text update sequence are captured to obtain semantic relationships; Generate the semantic encoding vector of the text embedding encoding based on the semantic relationship.

3. The layer normalization scaling method based on a large model as described in claim 1, characterized in that, The step of performing layer normalization on the initial text data to obtain normalized data includes: Obtain the feature dimensions of the initial text data; Generate the mean and standard deviation of the initial text data based on the aforementioned feature dimensions; The initial text data is transformed into a normal distribution using the mean and the standard deviation to obtain normalized features; All the normalized features are aggregated into normalized data.

4. The layer normalization scaling method based on a large model as described in claim 1, characterized in that, The step of inversely scaling the normalized data using the layer number at the target layer position to obtain scaled data includes: Generate an inverse scaling factor based on the layer number at the target layer location; The normalized data is scaled one by one using the inverse scaling factor to obtain several scaled feature results; All the scaling feature results are aggregated into scaling data.

5. The layer normalization scaling method based on a large model as described in claim 1, characterized in that, The step of activating the scaled data using a preset activation function to obtain target text data includes: The scaled data is weighted and summed to obtain the summed data; The summed data is activated using a preset activation function to obtain the activation value of the summed data; Determine whether the activation value is greater than a preset activation threshold; If the activation value is greater than the activation threshold, then the summation data corresponding to the activation values ​​that are greater than the activation threshold will be retained; If the activation value is less than or equal to the activation threshold, then the summation data corresponding to the activation values ​​that are less than or equal to the activation threshold will be deleted. The retained activation values ​​are reverse-mapped to obtain the target text data.

6. A layer normalization scaling device based on a large model, used to implement the layer normalization scaling method based on a large model as described in any one of claims 1 to 5, characterized in that, The device includes: The data filtering module is used to acquire the text data to be analyzed, and to use a preset deep neural network model to filter the text data to be analyzed to obtain the initial text data. The data normalization module is used to perform layer normalization processing on the initial text data to obtain normalized data; A position determination module is used to determine the target layer position of the normalized data in the deep neural network model; The data scaling module is used to inversely scale the normalized data using the number of layers at the target layer position to obtain scaled data, and to determine whether the target layer position corresponds to the preset output layer of the deep neural network model. The first judgment result module is used to activate the scaled data using a preset activation function if the target layer position does not correspond to the preset output layer of the deep neural network model, so as to obtain the target text data. The second judgment result module is used to treat the scaled data as target text data if the target layer position corresponds to the preset output layer of the deep neural network model.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a layer normalization scaling method based on a large model as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a layer normalization scaling method based on a large model as described in any one of claims 1 to 5.

Images