Data processing method and device, equipment, storage medium and program product

Abstract

The invention relates to a data processing method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence, the method comprises the steps that input text information is processed through a pre-trained large language model, output text information is obtained, a gating control layer in the pre-trained large language model comprises a routing module, and the routing module is used for receiving the output text information; the routing module selects the first feed-forward neural network or the second feed-forward neural network in the gating control layer to carry out data processing, and in the same large language model, the data is processed through different feed-forward neural networks, so that the advantages of different feed-forward neural networks are fully combined, and the data processing efficiency is improved. The problem that a single feedforward neural network has performance defects in the aspect of data processing is avoided, the context understanding ability and embedding ability of the large language model are improved, the performance of the large language model is improved, and the quality of text information generated by the large language model is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, device, storage medium, and program product. Background Art

[0002] With the development of the times, the application of intelligent models is becoming increasingly widespread. Due to the outstanding performance of the unidirectional decoder architecture in natural language processing and sequence generation tasks, large language models based on this architecture are gradually gaining popularity. The application of large language models based on this architecture has brought about a qualitative leap in the development of intelligent systems across various industries. However, large unidirectional language models have weak text understanding capabilities, which can lead to model hallucinations. Improving the text understanding capabilities of large unidirectional language models is an urgent problem in this field.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a data processing method, a device vehicle and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a data processing method, including:

[0006] Get input text information;

[0007] The input text information is processed by a pre-trained large language model to obtain output text information; wherein the pre-trained large language model includes at least a gating control layer, the gating control layer includes a routing module, a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

[0008] In one embodiment of the present disclosure, input text information is processed by a pre-trained large language model to obtain output text information, including:

[0009] Perform feature extraction on the input text information to obtain a first feature for inputting into the gating control layer;

[0010] Selecting the first feedforward neural network or the second feedforward neural network to process the first feature through a routing module in the gating control layer to obtain a processed second feature;

[0011] Output text information is generated based on the first feature and the second feature.

[0012] In one embodiment of the present disclosure, feature extraction is performed on input text information to obtain a first feature for inputting a gating control layer, including:

[0013] The input text information is processed through a one-way self-attention layer to obtain the processed third feature;

[0014] The input text information and the third feature are processed through the first residual connection and the normalization layer to obtain the processed first feature.

[0015] In one embodiment of the present disclosure, generating output text information based on the first feature and the second feature includes:

[0016] The first feature and the second feature are processed through the second residual connection and the normalization layer to obtain output text information.

[0017] In one embodiment of the present disclosure, the first feedforward neural network is trained based on a unidirectional language model;

[0018] The second feedforward neural network is trained based on a bidirectional language model.

[0019] In one embodiment of the present disclosure, the method further includes:

[0020] Obtain a first training sample;

[0021] Inputting the first training sample into the large language model to be trained to obtain a first training result;

[0022] Determine a first loss function value based on the first training result;

[0023] The routing module is trained according to the first loss function value, and in response to the first loss function value converging, a pre-trained large language model is obtained.

[0024] In one embodiment of the present disclosure, the method further includes:

[0025] The first loss function is constructed by cross entropy loss and gate regularization loss.

[0026] In one embodiment of the present disclosure, the method further includes:

[0027] Obtain a second training sample;

[0028] Inputting the second training sample into the bidirectional language model to obtain a second training result;

[0029] Determine a second loss function value based on the second training result;

[0030] The second feedforward neural network to be trained is trained according to the second loss function value, and in response to the second loss function value converging, a trained second feedforward neural network is obtained.

[0031] In one embodiment of the present disclosure, inputting the second training sample into the bidirectional language model to obtain a second training result includes:

[0032] The second training sample is processed through a bidirectional self-attention layer to obtain the fourth feature;

[0033] The second training sample and the fourth feature are processed through a third residual connection and a normalization layer to obtain a fifth feature;

[0034] The fifth feature is processed by a second feedforward neural network to obtain a sixth feature;

[0035] The fifth feature and the sixth feature are processed through the fourth residual connection and the normalization layer to obtain a second training result.

[0036] In one embodiment of the present disclosure, the second training sample includes a target text, a forward text, and a reverse text;

[0037] The second training result includes the seventh feature corresponding to the target text, the eighth feature corresponding to the forward text, and the ninth feature corresponding to the reverse text.

[0038] In one embodiment of the present disclosure, the method further includes:

[0039] determining a first similarity between the seventh feature and the eighth feature;

[0040] determining a second similarity between the eighth feature and the ninth feature;

[0041] A second loss function value is determined according to the first similarity and the second similarity.

[0042] According to a second aspect of an embodiment of the present disclosure, there is provided a data processing apparatus, including:

[0043] A first acquisition module is used to acquire input text information;

[0044] A generation module is used to process input text information through a pre-trained large language model to obtain output text information; wherein the pre-trained large language model includes at least a gating control layer, the gating control layer includes a routing module, a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

[0045] In one embodiment of the present disclosure, the generation module includes:

[0046] An extraction unit, configured to extract features from the input text information to obtain a first feature for input into the gating control layer;

[0047] A first processing unit is configured to select the first feedforward neural network or the second feedforward neural network to process the first feature through a routing module in the gating control layer to obtain a processed second feature;

[0048] A generating unit is configured to generate output text information based on the first feature and the second feature.

[0049] In one embodiment of the present disclosure, the extraction unit includes:

[0050] A first processing subunit is configured to process the input text information through a unidirectional self-attention layer to obtain a processed third feature;

[0051] The second processing subunit is used to process the input text information and the third feature through the first residual connection and the normalization layer to obtain the processed first feature.

[0052] In one embodiment of the present disclosure, the generating unit includes:

[0053] The third processing subunit is used to process the first feature and the second feature through the second residual connection and the normalization layer to obtain output text information.

[0054] In one embodiment of the present disclosure, the first feedforward neural network is trained based on a unidirectional language model;

[0055] The second feedforward neural network is trained based on a bidirectional language model.

[0056] In one embodiment of the present disclosure, the apparatus further comprises:

[0057] A second acquisition module is used to acquire a first training sample;

[0058] A first input module, configured to input a first training sample into the large language model to be trained to obtain a first training result;

[0059] A first determining module, configured to determine a first loss function value based on a first training result;

[0060] The first training module is used to train the routing module according to the first loss function value, and obtain a pre-trained large language model in response to the convergence of the first loss function value.

[0061] In one embodiment of the present disclosure, the apparatus further comprises:

[0062] A building module for constructing the first loss function through cross entropy loss and gated regularization loss.

[0063] In one embodiment of the present disclosure, the apparatus further comprises:

[0064] A third acquisition module is used to acquire a second training sample;

[0065] A second input module, configured to input a second training sample into the bidirectional language model to obtain a second training result;

[0066] A second determining module, configured to determine a second loss function value based on a second training result;

[0067] The second training module is used to train the second feedforward neural network to be trained according to the second loss function value, and obtain the trained second feedforward neural network in response to the convergence of the second loss function value.

[0068] In one embodiment of the present disclosure, the second input module includes:

[0069] A second processing unit is configured to process the second training sample through a bidirectional self-attention layer to obtain a fourth feature;

[0070] a third processing unit, configured to process the second training sample and the fourth feature through a third residual connection and a normalization layer to obtain a fifth feature;

[0071] a fourth processing unit, configured to process the fifth feature through a second feedforward neural network to obtain a sixth feature;

[0072] The fifth processing unit is used to process the fifth feature and the sixth feature through a fourth residual connection and a normalization layer to obtain a second training result.

[0073] In one embodiment of the present disclosure, the second training sample includes a target text, a forward text, and a reverse text;

[0074] The second training result includes the seventh feature corresponding to the target text, the eighth feature corresponding to the forward text, and the ninth feature corresponding to the reverse text.

[0075] In one embodiment of the present disclosure, the apparatus further comprises:

[0076] a third determining module, configured to determine a first similarity between the seventh feature and the eighth feature;

[0077] a fourth determining module, configured to determine a second similarity between the eighth feature and the ninth feature;

[0078] The fifth determining module is configured to determine a second loss function value based on the first similarity and the second similarity.

[0079] According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0080] processor;

[0081] a memory for storing processor-executable instructions;

[0082] The processor is configured to implement the steps of any one of the data processing methods of the first aspect above.

[0083] According to a fifth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, which, when instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute any one of the data processing methods of the first aspect above.

[0084] According to the sixth aspect of an embodiment of the present disclosure, a computer program product is provided. The computer program product includes a computer program or computer instructions, which are loaded and executed by a processor to enable a computer to perform any one of the data processing methods of the first aspect above.

[0085] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0086] The present disclosure obtains input text information, processes the input text information through a pre-trained large language model, and obtains output text information. The gating control layer in the pre-trained large language model includes a routing module, and the routing module selects the first feedforward neural network or the second feedforward neural network in the gating control layer to process data. In the same large language model, different feedforward neural networks are used to process data respectively, fully combining the advantages of different feedforward neural networks, avoiding the problem of performance defects of a single feedforward neural network in data processing, improving the context understanding and embedding capabilities of the large language model, enhancing the performance of the large language model, and improving the quality of text information generated by the large language model.

[0087] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0089] Figure 1 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 1 .

[0090] Figure 2 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 2 .

[0091] Figure 3 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 3 .

[0092] Figure 4 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 4 .

[0093] Figure 5 2 is a structural diagram of a pre-trained large language model according to an exemplary embodiment of the present disclosure.

[0094] Figure 6 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 5 .

[0095] Figure 7 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 6 .

[0096] Figure 8 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 7 .

[0097] Figure 9 4 is a structural diagram of a bidirectional language model according to an exemplary embodiment of the present disclosure.

[0098] Figure 10 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 8 .

[0099] Figure 11 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 9 .

[0100] Figure 12 It is a block diagram of a data processing device according to an exemplary embodiment of the present disclosure.

[0101] Figure 13 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0102] Some exemplary embodiments of the present disclosure will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications and equivalents of the methods, devices and / or systems described herein will become apparent after understanding the present disclosure. For example, the order of operations described herein is merely an example and is not limited to those orders set forth herein, but may be changed as becomes apparent after understanding the present disclosure, except for operations that must be performed in a specific order. In addition, descriptions of features known in the art may be omitted for clarity and brevity.

[0103] The following exemplary embodiments of the present disclosure do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0104] The data processing method disclosed herein can be applied to various electronic devices, including but not limited to terminals such as smartphones, computers, and smart tablets, vehicles, and servers such as independent physical servers, server clusters composed of multiple physical servers, distributed systems, and cloud servers. The large language model disclosed herein can be configured in these electronic devices, resolving the potential for hallucinations in large language models within these devices, enhancing the large language model's text comprehension capabilities, and improving the quality of text information generated by these electronic devices.

[0105] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0106] Figure 1 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 1 .

[0107] like Figure 1 As shown, the following steps are included.

[0108] S110: Obtain input text information.

[0109] In some embodiments, the input text information may include text information for input to the pre-trained large language model of the present disclosure. The text information may include text information corresponding to natural language, text information corresponding to images, and text information corresponding to mathematical symbols. In the embodiments of the present disclosure, the text information may be obtained through user input or automatically obtained by the pre-trained large language model.

[0110] S120, processing the input text information through a pre-trained large language model to obtain output text information; wherein the pre-trained large language model (Large Language Model) includes at least a gate control layer (gate control layer), the gate control layer includes a routing module (Router), a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

[0111] In some embodiments, the large language model may include an intelligent model comprising multiple feed-forward neural networks (FFNs), wherein the multiple FFNs are connected in parallel. The output text information may be data output by the large language model, and the output text information may include text information corresponding to natural language, text information corresponding to images, and text information corresponding to mathematical symbols.

[0112] In some embodiments, the gate control layer may further include a feedforward neural network other than the first feedforward neural network and the second feedforward neural network. The routing module in the gate control layer may select one or more feedforward neural networks from the plurality of feedforward neural networks for data processing.

[0113] In some embodiments, the first feedforward neural network may be a unidirectional feedforward neural network, and the second feedforward neural network may be a bidirectional feedforward neural network. The unidirectional feedforward neural network may be the same as the feedforward neural network in the unidirectional language model. The bidirectional feedforward neural network may be the same as the feedforward neural network in the bidirectional language model. It should be noted that a unidirectional language model refers to a model that can only utilize the contextual information of the current position and the previous position when predicting the next word in a sequence, and cannot access future information. A bidirectional language model refers to a model that can simultaneously utilize the contextual information on both sides of the word when predicting a word in a sequence. The unidirectional language model implements text generation tasks through a causal self-attention mechanism and performs well in multi-task scenarios. However, since it can only obtain sequence information in one direction, its performance in text embedding tasks is limited. By enabling the bidirectional attention mechanism and combining it with contrastive learning training, its embedding ability can be significantly improved, but the native architecture still has the problem of incomplete information acquisition. The bidirectional language model optimizes the text embedding task through the full-sequence self-attention mechanism, which can effectively capture global context dependencies, but cannot be directly used for generation tasks. Directly applying bidirectional training to generation tasks results in severe performance loss, which becomes a key bottleneck in unifying embedding and generation capabilities.

[0114] In order to avoid the performance degradation of the language model, the present disclosure discovered a negative correlation between backward dependency and model generation capability. The feedforward neural network layer has little effect on backward dependency while having a good bidirectional modeling effect.

[0115] For example, the present disclosure first defines the single-layer attention dependency. For the k-th layer in the n-th layer of self-attention, its attention weight is w=softmax(QK T )∈R b×l×l Where K is the key, Q is the query, b is the number of attention heads, and l is the length of the token sequence. The attention matrices of all attention heads in this layer are summed to obtain the attention score matrix A between each token pair in this layer. k ∈R l×l Then, the global attention weight A of the model is obtained by averaging the Ak of ​​all n layers of self-attention. Assume A i，j Indicates the degree of dependence of the i-th token on the j-th token, then the average dependence of the i-th token on its previous i-1 tokens is and the average dependence on the seven subsequent 1-i tokens Can be expressed as:

[0116]

[0117] Among them, A i，i Indicates the degree of dependence of the i-th token on itself, recorded as Experiments based on the above formula show that fine-tuning only the feedforward neural network layer consistently exhibits the lowest backward dependency across models of all sizes. Therefore, in this disclosure, only the feedforward neural network layer is fine-tuned.

[0118] It should be noted that the feedforward neural network layer is one of the most basic neural network structures. Its core feature is that information flows in one direction, that is, from the input layer to the hidden layer and finally to the output layer, without feedback or loop connections.

[0119] It can be seen from the above embodiments that unidirectional feedforward neural networks are deficient in text embedding, and bidirectional feedforward neural networks are deficient in text generation capabilities. The large language model in the present disclosure connects a unidirectional feedforward neural network with a bidirectional feedforward neural network in parallel, and selects the unidirectional feedforward neural network or the bidirectional feedforward neural network through a routing module. While retaining the context understanding capability of the bidirectional feedforward neural network, it minimizes the dependence on backward text information during prediction, thereby retaining the text generation capability of the unidirectional feedforward neural network and improving the performance of the large language model.

[0120] The present disclosure obtains input text information, processes the input text information through a pre-trained large language model, and obtains output text information. The gating control layer in the pre-trained large language model includes a routing module, and the routing module selects the first feedforward neural network or the second feedforward neural network in the gating control layer to process data. In the same large language model, different feedforward neural networks are used to process data respectively, fully combining the advantages of different feedforward neural networks, avoiding the problem of performance defects of a single feedforward neural network in data processing, improving the context understanding and embedding capabilities of the large language model, enhancing the performance of the large language model, and improving the quality of text information generated by the large language model.

[0121] Figure 2 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 2 . Figure 2 Step S210 and Figure 1 The step S110 in FIG. 1 corresponds to the step S110 in FIG. 1 and will not be repeated here. Figure 2 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0122] S220 , performing feature extraction on the input text information to obtain a first feature for inputting into a gating control layer.

[0123] In some embodiments, feature extraction may be performed on the input text information based on a processing layer in a large language model. The embodiments of the present disclosure do not limit the processing layer in the applied large language model.

[0124] S230 , selecting the first feedforward neural network or the second feedforward neural network to process the first feature through the routing module in the gating control layer to obtain a processed second feature.

[0125] In some embodiments, the gating control layer is a neural network component that dynamically regulates the flow of information through a learnable gating mechanism. The gating control layer may include a routing module, a first feedforward neural network, and a second feedforward neural network. The routing module is connected to the first feedforward neural network and the second feedforward neural network, respectively. The first feedforward neural network may be a unidirectional feedforward neural network, and the second feedforward neural network may be a bidirectional feedforward neural network. In the gating control layer, the routing module may select a unidirectional feedforward neural network or a bidirectional feedforward neural network to process the third feature. Selecting a unidirectional feedforward neural network or a bidirectional feedforward neural network to process the third feature by the routing module can fully utilize the advantages of different feedforward neural networks.

[0126] S240: Generate output text information based on the first feature and the second feature.

[0127] In some embodiments, the first feature and the second feature may be processed based on a processing layer in a large language model. The embodiments of the present disclosure do not limit the processing layer in the applied large language model.

[0128] In the disclosed embodiment, feature extraction is performed on input text information to obtain a first feature for input into a gating control layer, a first feedforward neural network or a second feedforward neural network is selected through a routing module in the gating control layer to process the first feature to obtain a second feature, and output text information is generated based on the first feature and the second feature. This fully combines the advantages of different feedforward neural networks, avoids the problem that a single feedforward neural network has performance defects in data processing, improves the context understanding and embedding capabilities of the large language model, enhances the performance of the large language model, and improves the quality of text information generated by the large language model.

[0129] Figure 3 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 3 . Figure 3 Steps S310, S340 and S350 are Figure 2 The steps S210, S230 and S240 in FIG. 2 correspond to each other and are not repeated here. Figure 3 As shown, in Figure 3 The implementation process shown above also includes the following steps:

[0130] S320: Process the input text information through a unidirectional self-attention layer to obtain a processed third feature.

[0131] In some embodiments, the unidirectional self-attention layer (Unidirectional Self-Attention Layer) can be a self-attention layer in a unidirectional language model, wherein the self-attention mechanism (Self-Attention Mechanism) is a core component of the Transformer architecture, which is used to capture the dependencies between different positions in the input sequence without relying on a loop or convolutional structure. The self-attention layer processes the input text information and may include obtaining the dependencies between texts at different positions of the input text information. It should be noted that the dependencies between texts at different positions of the input text information can be represented by features. It should be noted that when calculating the attention weight, the unidirectional self-attention layer only allows the current position to pay attention to itself and the previous position, while shielding future position information. Its core feature is the use of causal masking to ensure that the information flow is unidirectional, which meets the requirements of autoregressive generation.

[0132] S330: Process the input text information and the third feature through a first residual connection and a normalization layer to obtain a processed first feature.

[0133] In some embodiments, residual connection and normalization layer (Add&Layer Norm) Residual connection and normalization layer are key components of modern deep neural networks, such as Transformer. It should be noted that residual connection refers to adding the input directly to the output in a certain layer of the network. Normalization is used to stabilize the distribution of input data and make model training more efficient. The residual connection and normalization layer significantly improve the training stability and performance of the model. By processing the input text information and the first feature through the first residual connection and the normalization layer, it is possible to avoid the loss of features in the input text information after processing by the self-attention layer, and it is also possible to avoid accelerating feature convergence and reduce dependence on initialization.

[0134] In the disclosed embodiment, the input text information is processed by a unidirectional self-attention layer to obtain a processed third feature, and the input text information and the third feature are processed by a first residual connection and a normalization layer to obtain a processed first feature. By processing the text information to obtain the first feature, the subsequent gating control layer can process the first feature.

[0135] Figure 4 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 4 . Figure 4 Steps S410 to S440 are Figure 3 The steps S310 to S340 in FIG. 3 correspond to each other and are not repeated here. Figure 4 As shown, in Figure 4 The implementation process shown above also includes the following steps:

[0136] S450: Process the first feature and the second feature through a second residual connection and a normalization layer to obtain output text information.

[0137] In some embodiments, the second residual connection and normalization layer may be the same as the first residual connection and normalization layer, and will not be described again here.

[0138] It should be noted that the pre-trained large language model in the present disclosure may also include other layers such as input layer and output layer, which are not limited in the present disclosure.

[0139] To explain the present disclosure in detail, the present disclosure provides a structural diagram of a pre-trained large language model. Figure 5 This is a structural diagram of a pre-trained large language model according to an exemplary embodiment of the present disclosure. Figure 5As shown, the pre-trained large language model includes a forward independent power layer, a first residual connection and normalization layer, a gated control layer, a second residual connection and normalization layer connected in sequence, and the routing module in the gated control layer is connected to the first feedforward neural network and the second feedforward neural network respectively.

[0140] Figure 6 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 5 . Figure 6 Steps S650 and S660 are Figure 1 The steps S110 and S120 in FIG. 1 correspond to each other and are not repeated here. Figure 6 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0141] S610: Obtain a first training sample.

[0142] In some embodiments, the first training sample may include input data, label data, and auxiliary information. The input data is the original information for model learning and can be structured or unstructured data in various forms. The label data can be the target variable that the model needs to predict. The auxiliary information can be information used to assist in training.

[0143] In some embodiments, the first training sample may be customized by a user, or may be obtained through network resources. The present disclosure does not limit the method for obtaining the first training sample.

[0144] S620: Input the first training sample into the large language model to be trained to obtain a first training result.

[0145] In some embodiments, the method for processing the first training sample with the trained large language model can be the same as the method for processing input text information with the pre-trained large language model, which is not repeated here.

[0146] S630: Determine a first loss function value based on the first training result.

[0147] In some embodiments, determining the first loss function value through the first training result may include determining the first loss function value based on a predetermined first loss function, the first training result, input data, label data, etc.

[0148] S640: Train the routing module according to the first loss function value, and obtain a pre-trained large language model in response to the first loss function value converging.

[0149] In some embodiments, training the routing module according to the first loss function value may include: in response to the first loss function value not converging while other parameters in the large language model remain unchanged, adjusting parameters of the routing module.

[0150] Exemplarily, a first training sample is input into the large language model to be trained to obtain a first training result, a first loss function value is determined based on the first training result, and whether the first loss function value converges is determined based on the first training results obtained multiple times in a row. If the first loss function value does not converge, the parameters of the routing module are adjusted, and training is continued based on the adjusted routing module until the first loss function value converges to obtain an adjusted routing module.

[0151] In some embodiments, by training the routing module, the routing module in the pre-trained large language model can better select the first feedforward neural network or the second feedforward neural network based on features, thereby fully utilizing the advantages of different feedforward neural networks.

[0152] In some embodiments, before training the routing module, the method may further include: training the large language model to be trained based on the loss function value, and obtaining a trained large language model in response to convergence of the loss function value.

[0153] In the disclosed embodiment, the routing module is trained using the first loss function value, and when the first loss function value converges, a pre-trained large language model is obtained. This enables the routing module in the pre-trained large language model to better select the first feedforward neural network or the second feedforward neural network based on features, fully utilizing the advantages of different feedforward neural networks and improving the performance of the pre-trained large language model.

[0154] Figure 7 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 6 . Figure 7 Steps S710, S720, and S740 to S770 are Figure 6 The steps S610 and S620 and S630 to S660 in FIG. 1 correspond to each other and are not repeated here. Figure 7 As shown, in Figure 6 The implementation process shown above also includes the following steps:

[0155] S730: Construct a first loss function through cross entropy loss and gated regularization loss.

[0156] In some embodiments, a cross-entropy loss is used to measure the difference between the model's predicted probability distribution and the true distribution. A gated regularization loss is used to constrain the routing module's selection of the first feedforward neural network or the second feedforward neural network. The cross-entropy loss may include a loss determined by the first training sample and the first training result. The gated regularization loss may include a loss determined by the number of feedforward neural networks in the gated control layer and the probability of selecting each feedforward neural network.

[0157] In some embodiments, before S530 , the method may further include: determining a cross entropy loss based on the label data in the first training sample and data corresponding to the label data in the first training result;

[0158] The gated regularization loss is determined based on the number of feedforward neural networks in the gated control layer and the probability of selecting each feedforward neural network.

[0159] In some embodiments, a first loss function is constructed by a cross entropy loss and a gated regularization loss. This may include performing a mathematical operation on the cross entropy loss and the gated regularization loss to obtain the first loss function. For example, the first loss function may be determined based on the sum of the cross entropy loss and the gated regularization loss. For example:

[0160] L=L CE +λL Gating

[0161] Among them, L CE is the cross entropy loss, L Gating is the gate regularization loss, L is the first loss function value, and λ is a hyperparameter.

[0162] In the embodiment of the present disclosure, the first loss function is determined by cross entropy loss and gated regularization loss, the first loss function value is determined based on the first loss function, and then the routing module is trained based on the first loss function value, which enables the routing module to be more accurate in selecting the first feedforward neural network or the second feedforward neural network.

[0163] Figure 8 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 7 . Figure 8 Steps S850 and S860 are Figure 1 The steps S110 and S120 in FIG. 1 correspond to each other and are not repeated here. Figure 8 As shown, in Figure 1 The implementation process shown above also includes the following steps:

[0164] S810: Obtain a second training sample.

[0165] In some embodiments, the second training sample includes target text, forward text, and reverse text.

[0166] The target text can be a standard text, the forward text can be a text with the same or similar meaning as the target text, and the reverse text can be a text with a meaning opposite to or significantly different from the target text. For example, the target text can be a wild goose, the forward text can be a swan, and the reverse text can be a car. The second training sample can be user-defined or obtained from general network resources. This disclosure does not limit the method for obtaining the second training sample.

[0167] S820: Input the second training sample into the bidirectional language model to obtain a second training result.

[0168] In some embodiments, the second training result includes the seventh feature corresponding to the target text, the eighth feature corresponding to the forward text, and the ninth feature corresponding to the reverse text. To illustrate the bidirectional language model, Figure 9 FIG. 1 is a structural diagram of a bidirectional language model according to an exemplary embodiment of the present disclosure. Figure 9 As shown, the bidirectional language model may include a bidirectional self-attention layer, a third residual connection and normalization layer, a second feedforward neural network, and a fourth residual connection and normalization layer.

[0169] In some embodiments, training can be performed based on contrastive learning, which is a self-supervised learning method that allows the model to learn the essential feature representation of the data by comparing similar samples (positive text) and dissimilar samples (reverse text).

[0170] S830: Determine a second loss function value based on the second training result.

[0171] In some embodiments, since the present disclosure performs training based on contrastive learning, the second loss function may be determined only through the second training result.

[0172] S840: Train the second feedforward neural network to be trained according to the second loss function value, and obtain a trained second feedforward neural network in response to the second loss function value converging.

[0173] In some embodiments, training the second feedforward neural network to be trained based on the second loss function value may include: in response to the second loss function failing to converge, adjusting parameters in the second feedforward neural network while keeping other parameters of the bidirectional language model unchanged. In the present disclosure, by training the second feedforward neural network, the embedding capability of the word vectors of the second feedforward neural network can be enhanced.

[0174] In some embodiments, after the second feedforward neural network is trained, a pre-trained large language model can be constructed based on the trained second feedforward neural network.

[0175] In the embodiment of the present disclosure, by training the trained second feedforward neural network based on the second loss function value, a trained second feedforward neural network is obtained, which can enhance the word embedding ability of the second feedforward neural network and further enhance the performance of the pre-trained large language model.

[0176] Figure 10 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 8 . Figure 10 Steps S1010, S1060 to S1090 and Figure 8 The steps S810, S830 to S860 in FIG. 8 correspond to each other and are not repeated here. Figure 10 As shown, in Figure 8 The implementation process shown above also includes the following steps:

[0177] S1020: Process the second training sample through a bidirectional self-attention layer to obtain a fourth feature.

[0178] In some embodiments, the bidirectional self-attention layer simultaneously utilizes contextual information at all positions in the sequence when calculating attention weights, rather than only accessing historical information like the unidirectional self-attention layer. The bidirectional self-attention layer has no causal masking, allowing the current position to pay attention to any position in the sequence, thereby achieving global context modeling. The bidirectional self-attention layer can be the self-attention layer in a bidirectional language model. The bidirectional self-attention layer is also used to process input text information, including obtaining dependencies between text at different positions in the input text information.

[0179] S1030: Process the second training sample and the fourth feature through a third residual connection and a normalization layer to obtain a fifth feature.

[0180] In some embodiments, the third residual connection and normalization layer may be the same as the first residual connection and normalization layer and the second residual connection and normalization layer, and will not be described again here.

[0181] S1040: Process the fifth feature through a second feedforward neural network to obtain a sixth feature.

[0182] In some embodiments, the second feedforward neural network may be a bidirectional feedforward neural network. The bidirectional feedforward neural network has been described in the above embodiments and will not be described again here.

[0183] S1050: Process the fifth feature and the sixth feature through a fourth residual connection and a normalization layer to obtain a second training result.

[0184] In some embodiments, the fourth residual connection and normalization layer may be the same as the third residual connection and normalization layer, and will not be described again here.

[0185] In the disclosed embodiment, by training the bidirectional feedforward neural network, the word embedding capability of the bidirectional feedforward neural network is enhanced, and further, the performance of the pre-trained large language model is enhanced.

[0186] Figure 11 This is a flow chart of a data processing method according to an exemplary embodiment of the present disclosure. Figure 9 . Figure 11 Steps S1110, S1120, S1160 to S1190 and Figure 8 The steps S810, S820, S830 to S860 in FIG. 8 correspond to each other and are not repeated here. Figure 11 As shown, in Figure 11 The implementation process shown above also includes the following steps:

[0187] S1130 , determining a first similarity between the seventh feature and the eighth feature.

[0188] In some embodiments, the similarity of the seventh feature or the eighth feature may be determined based on algorithms such as cosine similarity, Euclidean distance, dot product, Manhattan distance, Pearson correlation coefficient, and Hamming distance.

[0189] S1140 , determining a second similarity between the eighth feature and the ninth feature.

[0190] In some embodiments, the similarity between the eighth feature and the ninth feature may be determined based on a method for determining the similarity between the seventh feature and the eighth feature, which will not be described in detail here.

[0191] S1150: Determine a second loss function value based on the first similarity and the second similarity.

[0192] In some embodiments, the second loss function may be determined based on the first similarity compared to a ratio of the first similarity to the sum of the second similarity.

[0193] Exemplarily, the second loss function may be:

[0194]

[0195] Where N is the size of the training sample, i is the i-th group of samples, and each group of samples includes the input text x i The vector h obtained by the model outputi , and x i Corresponding positive text The vector obtained by the model output with x i Corresponding reverse text The vector obtained by the model output j is the jth group of samples, and τ is the temperature hyperparameter.

[0196] In the embodiment of the present disclosure, the second loss function value is determined by the first similarity between the seventh feature and the eighth feature and the second similarity between the eighth feature and the ninth feature, and the second feedforward neural network is trained based on the second loss function value corresponding to the second loss function, which can improve the embedding ability of the word vector of the trained second feedforward neural network.

[0197] Figure 12 FIG. 1 is a block diagram of a data processing device according to an exemplary embodiment of the present disclosure. Figure 12 , the apparatus 1200 comprises:

[0198] A first acquisition module 1210 is used to acquire input text information;

[0199] The generation module 1220 is used to process the input text information through a pre-trained large language model to obtain output text information; wherein the pre-trained large language model includes at least a gating control layer, the gating control layer includes a routing module, a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

[0200] In one embodiment of the present disclosure, the generating module 1220 includes:

[0201] A first processing unit is configured to process the input text information through a unidirectional self-attention layer to obtain a processed first feature;

[0202] a second processing unit, configured to process the input text information and the first feature through a first residual connection and a normalization layer to obtain a processed second feature;

[0203] a third processing unit, configured to select the first feedforward neural network or the second feedforward neural network to process the second feature through the routing module in the gate control layer, to obtain a processed third feature;

[0204] The fourth processing unit is used to process the second feature and the third feature through a second residual connection and a normalization layer to obtain output text information.

[0205] In one embodiment of the present disclosure, the first feedforward neural network is trained based on a unidirectional language model;

[0206] The second feedforward neural network is trained based on a bidirectional language model.

[0207] In one embodiment of the present disclosure, the apparatus further comprises:

[0208] A second acquisition module is used to acquire a first training sample;

[0209] A first input module, configured to input a first training sample into the large language model to be trained to obtain a first training result;

[0210] A first determining module, configured to determine a first loss function value based on a first training result;

[0211] The first training module is used to train the routing module according to the first loss function value, and obtain a pre-trained large language model in response to the convergence of the first loss function value.

[0212] In one embodiment of the present disclosure, the apparatus further comprises:

[0213] A building module for constructing the first loss function through cross entropy loss and gated regularization loss.

[0214] In one embodiment of the present disclosure, the apparatus further comprises:

[0215] A third acquisition module is used to acquire a second training sample;

[0216] A second input module, configured to input a second training sample into the bidirectional language model to obtain a second training result;

[0217] A second determining module, configured to determine a second loss function value based on a second training result;

[0218] The second training module is used to train the second feedforward neural network to be trained according to the second loss function value, and obtain the trained second feedforward neural network in response to the convergence of the second loss function value.

[0219] In one embodiment of the present disclosure, the second input module includes:

[0220] a fifth processing unit, configured to process the second training sample through a bidirectional self-attention layer to obtain a fourth feature;

[0221] a sixth processing unit, configured to process the second training sample and the fourth feature through a third residual connection and a normalization layer to obtain a fifth feature;

[0222] a seventh processing unit, configured to process the fifth feature through a second feedforward neural network to obtain a sixth feature;

[0223] The eighth processing unit is configured to process the fifth feature and the sixth feature through a fourth residual connection and normalization layer to obtain a second training result.

[0224] In an embodiment of the present disclosure, the second training sample includes a target text, a forward text, and a reverse text.

[0225] The second training result includes a seventh feature corresponding to the target text, an eighth feature corresponding to the forward text, and a ninth feature corresponding to the reverse text.

[0226] In an embodiment of the present disclosure, the apparatus further includes:

[0227] The third determination module is configured to determine a first similarity between the seventh feature and the eighth feature.

[0228] The fourth determination module is configured to determine a second similarity between the eighth feature and the ninth feature.

[0229] The fifth determination module is configured to determine a second loss function value based on the first similarity and the second similarity.

[0230] The present disclosure obtains input text information, processes the input text information through a pre-trained large language model, and obtains output text information. The gating control layer in the pre-trained large language model includes a routing module. The routing module selects a first feedforward neural network or a second feedforward neural network in the gating control layer to process data. In the same large language model, different feedforward neural networks are used for processing, which improves the context understanding ability and embedding ability of the large language model and improves the performance of the large language model.

[0231] Figure 13 is a block diagram of an electronic device according to an example embodiment of the present disclosure. For example, the apparatus 1300 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0232] Referring to Figure 13 , the apparatus 1300 can include one or more of the following components: a processing component 1302, a memory 1304, a power supply component 1306, a multimedia component 1308, an audio component 1310, an input / output (I / O) interface 1312, a sensor component 1314, and a communication component 1316.

[0233] Processing component 1302 generally controls the overall operation of device 1300, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. Processing component 1302 may include one or more processors 1320 to execute instructions to perform all or part of the steps of the aforementioned methods. In addition, processing component 1302 may include one or more modules to facilitate interaction between processing component 1302 and other components. For example, processing component 1302 may include a multimedia module to facilitate interaction between multimedia component 1308 and processing component 1302.

[0234] The memory 1304 is configured to store various types of data to support the operations of the device 1300. Examples of such data include instructions for any application or method operating on the device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0235] The power supply component 1306 provides power to the various components of the device 1300. The power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1300.

[0236] The multimedia component 1308 includes a screen that provides an output interface between the device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0237] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC) that is configured to receive external audio signals when the device 1300 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 1304 or transmitted via the communication component 1316. In some embodiments, the audio component 1310 further includes a speaker for outputting audio signals.

[0238] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0239] Sensor assembly 1314 includes one or more sensors for providing various aspects of the status assessment of device 1300. For example, sensor assembly 1314 can detect the open / closed state of device 1300, the relative positioning of components, such as the display and keypad of device 1300. Sensor assembly 1314 can also detect changes in the position of device 1300 or a component of device 1300, the presence or absence of user contact with device 1300, the orientation or acceleration / deceleration of device 1300, and changes in the temperature of device 1300. Sensor assembly 1314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1314 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0240] The communication component 1316 is configured to facilitate wired or wireless communication between the apparatus 1300 and other devices. The apparatus 1300 can access a wireless network based on a communication standard, such as Wi-Fi, 3G, 4G, 5G, other communication standards, or a combination thereof. In some embodiments of the present disclosure, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of the present disclosure, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0241] In some embodiments of the present disclosure, the device 1300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0242] In some embodiments of the present disclosure, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 1304 including instructions, and the instructions can be executed by the processor 1320 of the apparatus 1300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0243] In some embodiments of the present disclosure, a non-transitory computer-readable storage medium enables a terminal to perform a data processing method when instructions in the storage medium are executed by a processor of a terminal.

[0244] In some embodiments of the present disclosure, a computer program product is further provided, including a computer program / instruction, which implements a data processing method when the computer program / instruction is executed by a processor.

[0245] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0246] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can use various methods to implement the functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0247] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims. It should be understood that the present disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: include: Get input text information; The input text information is processed by a pre-trained large language model to obtain output text information; wherein the pre-trained large language model includes at least a gating control layer, the gating control layer includes a routing module, a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

2. The method according to claim 1, characterized in that The processing of the input text information by the pre-trained large language model to obtain output text information includes: Performing feature extraction on the input text information to obtain a first feature for inputting into the gate control layer; Selecting the first feedforward neural network or the second feedforward neural network to process the first feature through the routing module in the gating control layer to obtain a processed second feature; The output text information is generated based on the first feature and the second feature.

3. The method according to claim 2, characterized in that The extracting features of the input text information to obtain a first feature for inputting into the gate control layer includes: Processing the input text information through a unidirectional self-attention layer to obtain a processed third feature; The input text information and the third feature are processed through a first residual connection and a normalization layer to obtain the processed first feature.

4. The method according to claim 2, characterized in that The generating the output text information based on the first feature and the second feature includes: The first feature and the second feature are processed through a second residual connection and a normalization layer to obtain the output text information.

5. The method according to claim 1, wherein The first feedforward neural network is obtained based on unidirectional language model training; The second feedforward neural network is obtained by training based on a bidirectional language model.

6. The method according to claim 1, characterized in that The method further comprises: Obtain a first training sample; Inputting the first training sample into the large language model to be trained to obtain a first training result; Determine a first loss function value based on the first training result; The routing module is trained according to the first loss function value, and in response to the first loss function value converging, the pre-trained large language model is obtained.

7. The method according to claim 4, characterized in that The method further comprises: The first loss function is constructed by cross entropy loss and gate regularization loss.

8. The method according to claim 1 or 5, characterized in that The method further comprises: Obtain a second training sample; Inputting the second training sample into the bidirectional language model to obtain a second training result; Determine a second loss function value according to the second training result; The second feedforward neural network to be trained is trained according to the second loss function value, and in response to the second loss function value converging, the trained second feedforward neural network is obtained.

9. The method according to claim 8, characterized in that Inputting the second training sample into the bidirectional language model to obtain a second training result includes: Processing the second training sample through a bidirectional self-attention layer to obtain a fourth feature; Processing the second training sample and the fourth feature through a third residual connection and a normalization layer to obtain a fifth feature; Processing the fifth feature through a second feedforward neural network to obtain a sixth feature; The fifth feature and the sixth feature are processed through a fourth residual connection and a normalization layer to obtain a second training result.

10. The method according to claim 8, characterized in that The second training sample includes a target text, a forward text, and a reverse text; The second training result includes the seventh feature corresponding to the target text, the eighth feature corresponding to the forward text, and the ninth feature corresponding to the reverse text.

11. The method according to claim 10, characterized in that The method further comprises: determining a first similarity between the seventh feature and the eighth feature; determining a second similarity between the eighth feature and the ninth feature; The second loss function value is determined according to the first similarity and the second similarity.

12. A data processing device, characterized in that: include: A first acquisition module is used to acquire input text information; A generation module is used to process the input text information through a pre-trained large language model to obtain output text information; wherein the pre-trained large language model includes at least a gating control layer, the gating control layer includes a routing module, a first feedforward neural network and a second feedforward neural network, and the routing module is used to select the first feedforward neural network or the second feedforward neural network.

13. The device according to claim 12, characterized in that The device further comprises: A second acquisition module is used to acquire a first training sample; A first input module, configured to input the first training sample into the large language model to be trained to obtain a first training result; A first determining module, configured to determine a first loss function value based on the first training result; A first training module is configured to train the routing module according to the first loss function value, and obtain the pre-trained large language model in response to convergence of the first loss function value.

14. The device according to claim 12, characterized in that The device further comprises: A third acquisition module is used to acquire a second training sample; A second input module, configured to input the second training sample into the bidirectional language model to obtain a second training result; A second determining module, configured to determine a second loss function value based on the second training result; The second training module is used to train the second feedforward neural network to be trained according to the second loss function value, and obtain the trained second feedforward neural network in response to the convergence of the second loss function value.

15. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the data processing method according to any one of claims 1 to 11. 16 . A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to perform the steps of the data processing method according to claim 1 .

17. A computer program product comprising a computer program or computer instructions, characterized in that: The computer program or the computer instructions are loaded and executed by a processor to enable the computer to implement the steps of the data processing method according to any one of claims 1 to 11.

Images