Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium, and relates to the field of data processing, in particular to the fields of deep learning, NLP technologies and the like. According to the specific implementation scheme, a training data set is obtained, the training data set comprises at least one sub-data set, and each sub-data set corresponds to one data type; independently training the language model according to each sub-data set, and in the training process, performing back propagation for each piece of data to obtain gradient information of each level parameter in the language model; according to gradient information of each level parameter in the language model, performing interpretability analysis on a plurality of levels in the language model to obtain a mapping relationship between data of different data types in the training data set and different level parameter gradients in the language model; and generating a customized data screening tool based on the mapping relation. According to the method, the influence of the data on the parameters of each layer of the model can be evaluated on a finer granularity, so that the interpretability analysis of the model is realized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, in particular to the fields of deep learning and NLP technology, and more particularly to a data processing method, device, electronic device and storage medium. Background Art

[0002] Training data largely determines the performance of language models. Conducting research on the correlation between data and model capabilities can effectively guide the mining and collection of high-value data. In related technologies, the evaluation of data types and data values ​​is mainly verified through the following two methods: the first method is to directly analyze the data type and make value judgments based on artificial prior knowledge; the second method is to verify through model training results, that is, to incorporate the target data into the model training process and infer the distribution or quality of the data by observing the changes in relevant indicators.

[0003] However, in the evaluation of data types and data value, although the above two methods have been widely used in practice, they each have certain limitations. For example, the first method has the problems of high subjectivity and lack of systematicity; the second method has the problems of indirectness and lack of interpretability of the evaluation, and lack of fine-grained analysis. Summary of the invention

[0004] The present disclosure provides a data processing method, device, electronic device and storage medium.

[0005] According to a first aspect of the present disclosure, there is provided a data processing method, comprising:

[0006] Acquire a training data set, wherein the training data set includes at least one sub-data set, and each sub-data set corresponds to a data type;

[0007] The language model is trained independently according to each of the sub-data sets, and during the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model;

[0008] According to the gradient information of the parameters at each level in the language model, an interpretability analysis is performed on the multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of the parameters at different levels in the language model;

[0009] Based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model, a customized data screening tool is generated.

[0010] According to a second aspect of the present disclosure, there is provided a data processing device, comprising:

[0011] An acquisition module, used to acquire a training data set, wherein the training data set includes at least one sub-data set, and each sub-data set corresponds to a data type;

[0012] A training module, used to independently train the language model according to each of the sub-data sets, and during the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model;

[0013] An analysis module, configured to perform interpretability analysis on multiple levels in the language model according to gradient information of parameters at each level in the language model, and obtain a mapping relationship between data of different data types in the training data set and gradients of parameters at different levels in the language model;

[0014] A generation module is used to generate a customized data screening tool based on the mapping relationship between data of different data types in the training data set and parameter gradients of different levels in the language model.

[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the steps of the method described in the first aspect when executed by a processor.

[0021] According to the technology disclosed in this disclosure, the impact of data on the parameters of each layer of the model can be evaluated at a finer granularity, thereby realizing the interpretability analysis of the model. Through the mapping relationship between data of different data types and parameter gradients at different levels in the language model, this disclosure can develop a customized data screening tool that can be used for the precise selection of different types of data, that is, it can more accurately identify and classify data of different quality levels, thereby optimizing the pre-training stage of the model.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

[0025] Figure 2 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

[0026] Figure 3 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

[0027] Figure 4 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

[0028] Figure 5 is an example diagram of a data processing method provided by an embodiment of the present disclosure;

[0029] Figure 6 is a block diagram of a data processing device provided by an embodiment of the present disclosure;

[0030] Figure 7 is a block diagram of a data processing device provided by an embodiment of the present disclosure;

[0031] Figure 8 is a block diagram of a data processing device provided by an embodiment of the present disclosure;

[0032] Fig. 9 is a block diagram of a data processing device provided by an embodiment of the present disclosure;

[0033] Fig.10 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0036] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0037] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.

[0038] It is worth noting that the training data largely determines the performance of the language model. Conducting research on the correlation between data and model capabilities can effectively guide the mining and collection of high-value data. However, current research on the relationship between data and model capabilities is still at a shallow and preliminary stage, lacking a detailed description of the deep relationship between different modules or parameters of the model and different data types or data values. Therefore, establishing a direct analytical bridge between data and model parameters to deeply understand and reveal these complex relationships has important academic value and practical significance.

[0039] In related technologies, the evaluation of data types and data values ​​is mainly verified through the following two methods: the first method is to directly analyze the data type and make value judgments based on artificial prior knowledge; the second method is to verify through model training results, that is, to incorporate the target data into the model training process, and infer the distribution or quality of the data by observing the changes in relevant indicators. For example, in the verification process of the open source model (LLama3.1 model), the researchers mixed the target data with the original data in a ratio of 3:7 and performed annealing training, and evaluated the quality of the data by observing the performance changes of the model in the benchmark. This method provides an indirect data evaluation method based on actual model performance. However, there are still challenges in deeply understanding the specific impact of different data types on model performance.

[0040] However, in the data type and data value assessment, although the above two methods have been widely used in practice, they each have certain limitations. For example, the first method has the following problems:

[0041] (1) High Subjectivity: This method relies on the experience and prior knowledge of experts and is highly subjective. This reliance may lead to bias in the assessment of data types or values, which in turn may lead to misjudgment;

[0042] (2) Lack of systematicity: Analysis based on prior knowledge is often discrete and fragmented, making it difficult to form a systematic global perspective, which may ignore the underlying complex relationships between data. Especially when dealing with complex and diverse data sets, manual methods may find it difficult to capture subtle patterns or anomalies in the data.

[0043] The second method mentioned above has the following problems:

[0044] (1) Indirectness and lack of interpretability of evaluation: Although data quality can be inferred by observing changes in the performance of the model in the benchmark test, this method is indirect and the evaluation results may be affected by the limitations of the model itself or other training parameters. Therefore, they may not be accurate enough and have poor interpretability.

[0045] (2) Lack of fine-grained analysis: This method usually focuses on the changes in the overall performance of the model, and it is difficult to conduct fine-grained analysis on the impact of specific data types or specific data subsets. At the same time, it is also impossible to effectively determine which data has made a key contribution to the optimization of specific modules or parameters of the model.

[0046] To this end, the embodiments of the present disclosure provide a data processing method that can evaluate the impact of data on the parameters of each layer of the model at a finer granularity, thereby realizing the interpretability analysis of the model. The present disclosure can develop a customized data screening tool through the mapping relationship between data of different data types and parameter gradients at different levels in the language model, which can be used for the precise selection of different types of data, that is, it can more accurately identify and classify data of different quality levels, thereby optimizing the pre-training stage of the model.

[0047] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the data processing method can be applied to electronic devices, and the method may include but is not limited to the following steps.

[0048] In step 101, a training data set is obtained.

[0049] In some embodiments, the training data set may include but is not limited to one or more sub-data sets. Exemplarily, the training data set may include a sub-data set, and the data in the sub-data set are all of the same data type. Exemplarily, the training data set may include multiple sub-data sets, and the multiple sub-data sets correspond to different data types. Exemplarily, each sub-data set in the multiple sub-data sets corresponds to a data type. For example, the training data set may include 4 sub-data sets (such as sub-data set A, sub-data set B, sub-data set C, and sub-data set D), wherein the data type of the data in sub-data set A is data type a, the data type of the data in sub-data set B is data type b, the data type of the data in sub-data set C is data type c, and the data type of the data in sub-data set D is data type d.

[0050] In some embodiments, the training data set can be pre-constructed in the following manner: Based on the conversational language model and prompt engineering technology, a training data set based on the knowledge management method is constructed through the human-in-the-Loop method. In other words, based on the conversational language model and prompt engineering technology, the classification accuracy of the data in the training data set can be iteratively improved through the human-in-the-Loop method, and finally a high-quality training data set based on the knowledge management method can be constructed.

[0051] In some embodiments, prompt word engineering technology combined with knowledge management methods can be used to process open source data sets to generate guidance instructions for the data in the open source data sets; the data in the open source data sets are labeled based on the guidance instructions and conversational language model of the data in the open source data sets; the annotations of the data in the open source data sets are reviewed and optimized using a human-computer loop method; and finally the labeled open source data sets are used as training data sets.

[0052] In some embodiments, the conversational language model may include but is not limited to a pre-trained language model based on a Transformer architecture. Exemplarily, the conversational language model may be a GPT (Generative Pre-trained Transformer) model, such as a GPT-4 model.

[0053] In some embodiments, the training data set may be a data set constructed based on a knowledge management method. Exemplarily, the knowledge management method may be a DIKW knowledge management model, wherein the DIKW knowledge management model divides knowledge into four levels: data, information, knowledge, and wisdom, which form a progressive relationship. The training data set may be a data set constructed based on the DIKW knowledge management model, and may also be referred to as a DIKW data set. That is: the data set constructed based on the DIKW knowledge management model may include four data types, namely, a Data data type, an Information data type, a Knowledge data type, and a Wisdom data type, and each data type corresponds to a sub-data set.

[0054] Exemplarily, the Prompt engineering technology can be combined with the DIKW knowledge management model to process the open source data set and generate guidance instructions for the data in the open source data set. For example, taking data A as an example, the guidance instructions for the data A can be generated by combining the Prompt engineering technology with the DIKW knowledge management model. For example, the guidance instructions can be "analyze which level (i.e., which category) the data A belongs to in the DIKW knowledge management model". The data in the open source data set and its guidance instructions are input into the GPT-4 model for processing. The GPT-4 model outputs the labeling results of the data. For example, the labeling result of the data A is label I (information), that is, the data A belongs to I (information) in the DIKW knowledge management model. In the process of constructing the training data set, the labeling of the data in the open source data set can be reviewed and optimized through a human-computer loop, and the classification accuracy of the DIKW data can be iteratively improved, and finally a set of DIKW training data sets can be constructed.

[0055] In step 102, the language model is trained independently according to each sub-data set. During the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model.

[0056] In some embodiments, the language model may be a large language model (LLM) or a small language model (SLM). Exemplarily, in the training phase of the language model, independent training may be performed for each sub-data set. When each sub-data set is trained independently, back propagation is performed for each piece of data during the training process, and the gradient values ​​of the parameters at each level of the language model are recorded and stored. For example, the gradient values ​​of the parameters at each level of the model corresponding to each piece of data are recorded in a gradient database, so that the gradient information of the parameters at each level in the language model can be obtained (such as the gradient values ​​of the parameters at each level of the model corresponding to each piece of data) for subsequent analysis and optimization.

[0057] Exemplarily, the above-mentioned "back propagation" means inputting data into the language model to obtain the predicted label corresponding to the data, and obtaining the loss function according to the predicted label and the real label of the data, and then each weight (parameter) matrix can be learned and updated according to the gradient back-propagated from the loss function, so as to make the loss function of the next round smaller. When performing back propagation, the gradient values ​​of the parameters at each level of the language model are recorded and stored.

[0058] In step 103, based on the gradient information of the parameters at each level in the language model, an interpretability analysis is performed on multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of parameters at different levels in the language model.

[0059] In some embodiments, the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model can be understood as: there is a mapping relationship between data of different data types and parameter gradients at different levels of the language model, and the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model can be determined based on the training contribution of different data types to the parameters at different levels of the language model. Exemplarily, if the training contribution of a data type to a parameter at a certain level in the language model is greater than its training contribution to parameters at other levels, then there is a mapping relationship between the data type and the parameter gradient at that certain level.

[0060] It is worth noting that different data types contribute differently to the training of parameters at different levels of the model. For example, samples with a single format and simple content have a greater impact on the shallow parameters of the model, which is manifested as a larger training gradient for the shallow parameters; while complex samples tend to produce larger training gradients for the deep parameters of the model. Based on this, in some embodiments of the present disclosure, according to the gradient information of the parameters at each level in the language model, the impact of different data types on the parameters at each level in the language model can be evaluated at a fine-grained level, and based on the impact of data of different data types in the training data set on the parameters at each level in the language model, the mapping relationship between the data of the different data types and the gradients of the parameters at different levels in the language model is determined, thereby achieving interpretable decomposition of the language model.

[0061] In step 104, a customized data screening tool is generated based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model.

[0062] For example, based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model, a customized data screening tool can be developed for accurate selection of different types of data.

[0063] In some embodiments, each data type in different data types may correspond to a data screening tool for accurate screening of corresponding type data. In some embodiments, the data screening tool may include, but is not limited to, a language model, and a mapping relationship between the corresponding data type and the hierarchical parameter gradient in the language model. For example, taking the training data set including data type a, data type b, and data type c as an example, assuming that the language model includes level 1, level 2, and level 3, data type a has a mapping relationship with level 1 parameter gradient, data type b has a mapping relationship with level 1 parameter gradient and level 2 parameter gradient respectively, and data type c has a mapping relationship with level 3 parameter gradient. According to the mapping relationship between data type a and level 1 parameter gradient, a corresponding data screening tool can be generated for accurate selection of data of data type a. According to the mapping relationship between data type b and level 1 parameter gradient and level 2 parameter gradient, a corresponding data screening tool can be generated for accurate selection of data of data type b. According to the mapping relationship between data type c and level 3 parameter gradient, a corresponding data screening tool can be generated for accurate selection of data of data type c.

[0064] By implementing the embodiments of the present disclosure, the impact of data on the parameters of each layer of the model can be evaluated at a finer granularity, thereby achieving interpretability analysis of the model. The embodiments of the present disclosure can develop customized data screening tools through the mapping relationship between data of different data types and parameter gradients at different levels in the language model, which can be used for the precise selection of different types of data, that is, it can more accurately identify and classify data of different quality levels, thereby optimizing the pre-training stage of the model.

[0065] Figure 2 is a flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the data processing method can be applied to electronic devices, and the method may include but is not limited to the following steps.

[0066] In step 201, a training data set is obtained.

[0067] In some embodiments, the training data set may include, but is not limited to, one or more sub-data sets, each of which corresponds to a data type.

[0068] Optional implementations of step 201 can be found in Figure 1 Optional implementations of step 101, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0069] In step 202, the language model is trained independently according to each sub-data set. During the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model.

[0070] Optional implementations of step 202 can be found in Figure 1 Optional implementations of step 102, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0071] In step 203, based on the gradient information of the parameters at each level in the language model, a fine-grained influence function is used to perform interpretability analysis on multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of parameters at different levels in the language model.

[0072] In some embodiments, a fine-grained influence function can be used to analyze the mapping relationship between data and gradients at different levels of the model, so as to more intuitively describe the impact of data of different levels or qualities on the model. It is worth noting that the fine-grained influence function involved in the present disclosure is an extension of the traditional influence function method. The traditional influence function is mainly used to evaluate the impact of specific samples in the training data on the final model. Its core is to measure the contribution of a certain data point to the overall parameters of the model, prediction results or loss function. The fine-grained influence function decomposes the language model into multiple levels, evaluates the impact of data on the parameters of each level at a finer granularity, and thus realizes the interpretable decomposition of the model.

[0073] In some embodiments, Figure 3 As shown, step 203 may include but is not limited to the following steps.

[0074] In step 301, based on the gradient information of the parameters at each level in the language model, a fine-grained influence function is used to evaluate the influence of data on the parameters at each level, and the training contribution of data of different data types in the training data set to the parameters at different levels in the language model is obtained.

[0075] In some embodiments, based on the gradient information of the parameters at each level corresponding to each data in the training data set, a fine-grained influence function is used to determine the gradient change information of the parameters at each level during the training process; based on the gradient change information of the parameters at each level during the training process, the training contribution of data of different data types in the training data set to the parameters at different levels in the language model is determined.

[0076] Exemplarily, the gradient change information of the above-mentioned parameters at each level during the training process may refer to: the gradient of the level parameter during the current round of model training, compared with the change of the gradient during the previous round of model training. For example, taking a certain level parameter in the language model as an example, assuming that the level parameter is a level 1 parameter, the gradient of the level 1 parameter during the current round of model training is gradient grad 1, and the gradient of the level 1 parameter during the previous round of model training is gradient grad 2, then the gradient change information of the level 1 parameter during the training process may be the difference between gradient grad 2 and gradient grad 1.

[0077] Exemplarily, after obtaining the gradient change information of each layer parameter during the training process, the gradient change information can be analyzed to obtain the training contribution of data of different data types in the training data set to the training of different layer parameters in the language model. It is worth noting that for a certain data type, if the gradient change of the layer parameter is larger, it can be said that the data of the data type has a greater contribution to the training of the layer parameter; if the gradient change of the layer parameter is smaller, it can be said that the data of the data type has a smaller contribution to the training of the layer parameter.

[0078] It should be noted that the optional implementation method of the gradient change information of each layer parameter during the training process given above is an example of an optional implementation method given for the convenience of understanding the present disclosure, that is, other means may also be used, such as the gradient of the layer parameter during the current round of model training, compared with the change of the initial gradient of the layer parameter, that is, the gradient change information of the layer parameter during the training process is obtained. The present disclosure does not limit this and will not be repeated here.

[0079] In step 302, the mapping relationship between the data of different data types in the training data set and the gradients of the parameters at different levels in the language model is determined according to the training contribution of the data of different data types in the training data set to the parameters at different levels in the language model.

[0080] Exemplarily, for a certain data type, if the gradient change of the level parameter is larger, it can be said that the data of the data type has a greater contribution to the training of the level parameter; if the gradient change of the level parameter is smaller, it can be said that the data of the data type has a smaller contribution to the training of the level parameter. Exemplarily, if the training contribution of a data type to a certain level parameter in the language model is larger than the training contribution to other level parameters, then there is a mapping relationship between the data type and the gradient of the level parameter.

[0081] For example, taking the training data set including data type a, data type b and data type c as an example, assuming that the language model includes level 1, level 2 and level 3, if the training contribution of data type a to level 1 parameters is greater than the training contribution to level 2 parameters and level 3 parameters, it can be said that there is a mapping relationship between data type a and level 1 parameter gradient. If the training contribution of data type b to level 1 parameters and level 2 parameters is much greater than the training contribution to level 3 parameters, it can be said that there is a mapping relationship between data type b and level 1 parameter gradient and level 2 parameter gradient respectively. If the training contribution of data type c to level 3 parameters is much greater than the training contribution to level 1 parameters and level 2 parameters, it can be said that there is a mapping relationship between data type c and level 3 parameter gradient.

[0082] In step 204, a customized data screening tool is generated based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model.

[0083] Optional implementations of step 204 can be found in Figure 1 Optional implementations of step 104, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0084] It is worth noting that the present disclosure can be used as a key tool to explore the correlation between large language models and data quality, and has a wide range of application potential, especially in the fields of classification and quality improvement of pre-training data. By applying the present disclosure to the processing of pre-training data, it is possible to more accurately identify and classify data of different quality levels, thereby optimizing the pre-training stage of the model. Specific application areas may include, but are not limited to: classification of pre-training data, evaluation and improvement of data quality, and targeted enhancement of model capabilities.

[0085] Optionally, in some embodiments, Figure 4 As shown, the data processing method may further include: step 405, classifying and identifying the data to be selected based on the data screening tool to obtain the data type of the data to be selected. Figure 4 The optional implementation of steps 401 to 404 in the above Figure 1 The optional implementation methods of steps 101 to 104 in the above description will not be repeated here.

[0086] It should be noted that each of the different data types corresponds to a data screening tool, and the data screening tool includes a language model, and a mapping relationship between the corresponding data type and the gradient of parameters at different levels in the language model. In some embodiments, the optional implementation method of classifying and identifying the data to be selected based on the data screening tool to obtain the data type of the data to be selected is as follows: input the data to be selected into the language model to obtain the predicted type of the data to be selected; for each of the different data types, according to the loss function between the predicted type of the data to be selected and the data type, determine the gradient information of the parameters of each level of the model of the data to be selected under the different data types; according to the gradient information of the parameters of each level of the model of the data to be selected under the different data types, and the mapping relationship between the data of different data types in the training data set and the gradient of parameters at different levels in the language model, determine the data type of the data to be selected.

[0087] For example, taking data types including data type a, data type b and data type c as an example, assuming that data type a has a mapping relationship with the level 1 parameter gradient, data type b has a mapping relationship with the level 1 parameter gradient and the level 2 parameter gradient respectively, and data type c has a mapping relationship with the level 3 parameter gradient, the data to be selected can be input into the language model to obtain the predicted type of the data to be selected. For each data type among data types a, data type b and data type c, according to the predicted type of the data to be selected and the loss function between the data type, the gradient information of the model parameters at each level of the data to be selected under the data type is determined, and according to the gradient information, the data type corresponding to the level parameter gradient closest to the gradient information can be determined from the mapping relationship between the data type and the level parameter gradient, and the data type is used as the data type of the data to be selected. For example, when selecting data of data type a, an indicator associated with a high sensitivity parameter gradient can be used as a scoring standard, that is, the data is input into the language model to obtain a high sensitivity parameter gradient that has a mapping relationship with the data, and according to the high sensitivity parameter gradient, the data type corresponding to the level model parameter gradient closest to the high sensitivity parameter gradient can be determined from the mapping relationship that has been obtained as data type a. This achieves more accurate data optimization and selection.

[0088] Optionally, in some embodiments, the data processing method may also include: evaluating and improving data quality based on a data screening tool; and / or, directional enhancement of model capabilities based on a data screening tool. Exemplarily, based on the mapping relationship between different data types and parameter gradients at each level of the model, a customized data screening tool is developed, and the data screening tool is used to explore and analyze existing data, discover potential features and patterns, and effectively integrate them. This step may include data cleaning, removing noise and bias, and converting it into an input form suitable for deep learning models. Divide the data set and prepare it for training and testing the model. This process is the basis for data quality assessment, because the quality of the data directly affects the training effect of the model and the final prediction accuracy.

[0089] Exemplarily, the directional enhancement of model capabilities based on data screening tools mainly involves two aspects: data analysis and model design. Through the optimization of these two aspects, the performance and generalization ability of the model can be effectively improved. Among them, the optional implementation methods of data analysis are as follows: Use the data screening tool to explore and analyze the existing data, discover potential features and patterns, and effectively integrate them, clearly process the data, remove noise and bias, and convert them into input forms suitable for deep learning models. Divide the data set and prepare for training and testing models. The optional implementation methods of model design are as follows: model selection. When selecting a model, you can comprehensively consider the requirements, efficiency, accuracy, ease of use and complexity of different tasks, as well as the mapping relationship between different data types and different levels of parameter gradients in the language model. Combined with business scenarios, customized model design is carried out according to the characteristics of the scenario to make the model more in line with actual needs.

[0090] In order to facilitate those skilled in the art to understand the present disclosure, Figure 5 A detailed description is given, wherein the data of the DIKW knowledge management model is taken as an example for description.

[0091] like Figure 5 As shown, the disclosure as a whole can be divided into four parts. First, DIKW data is constructed as the data basis for model analysis. Second, DIKW data is input into the language model for training. Then, fine-grained interpretability analysis of data-parameters is performed through fine-grained influence functions. Finally, a customized data screening tool is formed through the mapping relationship between DIKW data and model parameters. The specific implementation process can be as follows:

[0092] 1. DIKW Data Construction

[0093] The DIKW Pyramid (Data-Information-Knowledge-Wisdom Pyramid) is a hierarchical model that describes human understanding, reasoning, and interpretation, and is widely used in the fields of information science and knowledge management. The pyramid divides data, information, knowledge, and wisdom into four levels to express the gradual deepening of human understanding of information. In the process of constructing model training data, hierarchical processing can be performed according to the DIKW framework:

[0094] Data: This is the bottom layer of the pyramid, which contains raw data that has not been processed and organized. This type of data is usually unstructured and lacks semantic meaning, requiring further processing and analysis to reveal its potential value.

[0095] Information: The information level involves the initial processing and understanding of data, which can answer basic questions. The data at this stage has a certain structure and semantics, providing context and background information.

[0096] Knowledge: Knowledge is a systematic understanding formed through in-depth analysis and application of information. It is usually based on experience and reasoning, and can provide guidance for decision-making in specific situations.

[0097] Wisdom: As the highest level, wisdom represents the abstraction and high generalization of knowledge, and has wide applicability and universality. Wisdom is not only the expansion of knowledge, but also includes a comprehensive understanding and insight into complex situations.

[0098] For example, based on the GPT-4 model and Prompt engineering technology, the classification accuracy of DIKW data can be iteratively improved through the human-in-the-Loop approach, and finally a high-quality DIKW training dataset can be constructed.

[0099] 2. DIKW model training

[0100] In the model training phase, independent training is performed on four sub-datasets: D (data), I (information), K (knowledge), and W (wisdom). During the training process, back propagation is performed on each piece of data, and the gradient values ​​of the parameters at each level of the language model are recorded in the gradient database for subsequent analysis and optimization.

[0101] For example, Figure 5As shown, the language model may include an input embedding layer, multiple Transformer block layers, and the language model may also include an RMS Norm (mean square error layer normalization) layer, a Linear layer, and a Softmax layer. Among them, the Transformer block layer may include an attention unit, a feed forward, and the like.

[0102] 3. Fine-grained interpretability analysis of data-parameters

[0103] Different data types contribute differently to the training of model parameters at different levels. For example, samples with a single format and simple content have a greater impact on the shallow parameters of the model, which is manifested as a larger training gradient for the shallow parameters; while complex samples tend to produce a larger training gradient for the deep parameters of the model. Based on this, we can use fine-grained influence functions to analyze the mapping relationship between DIKW data and gradients of model parameters at different levels, so as to more intuitively describe the impact of data of different levels or qualities on the model.

[0104] Exemplarily, the fine-grained influence function is an extension of the traditional influence function. The traditional influence function is mainly used to evaluate the impact of a specific sample in the training data on the final model. Its core is to measure the contribution of a certain data point to the overall parameters of the model, the prediction results or the loss function. The fine-grained influence function decomposes the model into multiple levels and evaluates the impact of data on the parameters of each layer at a finer granularity, thereby achieving the interpretable decomposition of the model.

[0105] 4. DIKW customized data selection tool

[0106] Based on the mapping relationship between DIKW data and model parameters, customized data screening tools can be developed for accurate selection of different types of data. For example, when selecting Wisdom data, indicators associated with high-sensitivity parameter gradients can be used as scoring criteria to achieve more accurate data optimization and selection. Exemplarily, the data can be input into the language model to obtain a high-sensitivity parameter gradient that has a mapping relationship with the data. Based on the high-sensitivity parameter gradient, the data type corresponding to the hierarchical model parameter gradient that is closest to the high-sensitivity parameter gradient can be determined from the obtained mapping relationship as "Wisdom data".

[0107] Figure 6 is a block diagram of a data processing device provided by an embodiment of the present disclosure. Figure 6 As shown, the data processing device may include: an acquisition module 601, a training module 602, an analysis module 603 and a generation module 604.

[0108] The acquisition module 601 is used to acquire a training data set, where the training data set includes at least one sub-data set, and each sub-data set corresponds to a data type.

[0109] The training module 602 is used to independently train the language model according to each sub-data set. During the training process, back propagation is performed on each piece of data to obtain the gradient information of each level parameter in the language model.

[0110] The analysis module 603 is used to perform interpretability analysis on multiple levels in the language model according to the gradient information of the parameters at each level in the language model, and obtain the mapping relationship between data of different data types in the training data set and the gradients of parameters at different levels in the language model.

[0111] In some embodiments, the analysis module 603 is used to: perform interpretability analysis on multiple levels in the language model using fine-grained influence functions based on the gradient information of parameters at each level in the language model, and obtain a mapping relationship between data of different data types in the training data set and gradients of parameters at different levels in the language model.

[0112] In some embodiments, the analysis module 603 is used to: evaluate the influence of data on parameters at each level according to the gradient information of parameters at each level in the language model using a fine-grained influence function, and obtain the training contribution of data of different data types in the training data set to parameters at different levels in the language model; determine the mapping relationship between data of different data types in the training data set and gradients of parameters at different levels in the language model according to the training contribution of data of different data types in the training data set to parameters at different levels in the language model.

[0113] In some embodiments, the analysis module 603 is used to: determine the gradient change information of the parameters at each level during the training process using a fine-grained influence function based on the gradient information of the parameters at each level corresponding to each data in the training data set; determine the training contribution of data of different data types in the training data set to the parameters at different levels in the language model based on the gradient change information of the parameters at each level during the training process.

[0114] The generation module 604 is used to generate a customized data screening tool based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model.

[0115] In some embodiments, Figure 7 As shown, the data processing device may further include: a construction module 705. The construction module 705 is used to construct a training data set based on a knowledge management method through a human-computer loop method based on a conversational language model and prompt word engineering technology. The knowledge management method is a DIKW knowledge management model.

[0116] In some embodiments, the construction module 705 is used to: use prompt word engineering technology combined with knowledge management methods to process the open source data set to generate guidance instructions for the data in the open source data set; annotate the data in the open source data set according to the guidance instructions and conversational language model of the data in the open source data set; use the human-computer loopback method to review and optimize the annotations of the data in the open source data set; and use the annotated open source data set as a training data set. Figure 7 701-704 and Figure 6 601-604 have the same function and structure.

[0117] In some embodiments, Figure 8 As shown, the data processing device may also include: a classification identification module 806. The classification identification module 806 is used to classify and identify the data to be selected based on the data screening tool to obtain the data type of the data to be selected. Among them, each data type in the different data types corresponds to a data screening tool, and the data screening tool includes a language model, and a mapping relationship between the corresponding data type and the gradient of parameters at different levels in the language model; the classification identification module 806 is used to: input the data to be selected into the language model to obtain the predicted type of the data to be selected; for each data type in the different data types, according to the loss function between the predicted type of the data to be selected and the data type, determine the gradient information of the parameters at each level of the model of the data to be selected under the different data types; according to the gradient information of the parameters at each level of the model of the data to be selected under the different data types, and the mapping relationship between the data of different data types in the training data set and the gradient of parameters at different levels in the language model, determine the data type of the data to be selected. Among them, Figure 8 801-805 and Figure 7 701-705 have the same function and structure.

[0118] In some embodiments, Fig. 9 As shown, the data processing device may further include: an enhancement module 907. The enhancement module 907 is used to: evaluate and improve data quality based on the data screening tool; and / or, perform directional enhancement of model capabilities based on the data screening tool. Fig. 9 901-906 and Figure 8 801-806 have the same function and structure.

[0119] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0120] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0121] like Fig.10 , is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0122] like Fig.10 As shown, the electronic device includes: one or more processors 1001, a memory 1002, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 1001 is taken as an example.

[0123] The memory 1002 is a non-transient computer-readable storage medium provided by the present disclosure. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the data processing method provided by the present disclosure. The non-transient computer-readable storage medium of the present disclosure stores computer instructions, which are used to enable a computer to perform the data processing method provided by the present disclosure.

[0124] The memory 1002 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the data processing method in the embodiment of the present disclosure (for example, the attached Figure 6 The processor 1001 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 1002, that is, the data processing method in the above method embodiment is implemented.

[0125] The memory 1002 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1002 may optionally include a memory remotely arranged relative to the processor 1001, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The electronic device may further include: an input device 1003 and an output device 1004. The processor 1001, the memory 1002, the input device 1003 and the output device 1004 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0127] The input device 1003 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 1004 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0128] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0132] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0133] According to the technical solution of the embodiment of the present disclosure, the impact of data on the parameters of each layer of the model can be evaluated at a finer granularity, thereby realizing the interpretability analysis of the model. The embodiment of the present disclosure can develop a customized data screening tool through the mapping relationship between data of different data types and parameter gradients at different levels in the language model, which can be used for the precise selection of different types of data, that is, it can more accurately identify and classify data of different quality levels, thereby optimizing the pre-training stage of the model.

[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this disclosure can be achieved, and this document is not limited here.

[0135] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, comprising: Acquire a training data set, wherein the training data set includes at least one sub-data set, and each sub-data set corresponds to a data type; The language model is trained independently according to each of the sub-data sets, and during the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model; According to the gradient information of the parameters at each level in the language model, an interpretability analysis is performed on the multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of the parameters at different levels in the language model; Based on the mapping relationship between data of different data types in the training data set and parameter gradients at different levels in the language model, a customized data screening tool is generated.

2. The method according to claim 1, wherein: The training dataset is pre-constructed in the following way: Based on the conversational language model and prompt word engineering technology, a training dataset based on the knowledge management method is constructed through the human-computer loop method.

3. The method according to claim 2, wherein: The training data set based on the conversational language model and prompt word engineering technology is constructed through a human-computer loopback method based on a knowledge management method, including: The prompt word engineering technology is combined with the knowledge management method to process the open source data set to generate guidance instructions for the data in the open source data set; Annotating the data in the open source dataset according to the guidance instructions of the data in the open source dataset and the conversational language model; The human-machine loopback method is used to recheck and optimize the annotation of the data in the open source dataset; The annotated open source dataset is used as the training dataset.

4. The method according to claim 1, wherein: The interpretability analysis is performed on multiple levels in the language model according to the gradient information of the parameters at each level in the language model to obtain the mapping relationship between the data of different data types in the training data set and the gradients of the parameters at different levels in the language model, including: According to the gradient information of the parameters at each level in the language model, a fine-grained influence function is used to perform interpretability analysis on multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of parameters at different levels in the language model.

5. The method according to claim 4, wherein: The method of performing interpretability analysis on multiple levels in the language model using a fine-grained influence function according to the gradient information of the parameters at each level in the language model, and obtaining a mapping relationship between data of different data types in the training data set and the gradients of the parameters at different levels in the language model, includes: According to the gradient information of the parameters at each level in the language model, the fine-grained influence function is used to evaluate the influence of the data on the parameters at each level, so as to obtain the training contribution of the data of different data types in the training data set to the parameters at different levels in the language model; According to the training contribution of data of different data types in the training data set to the parameters of different levels in the language model, the mapping relationship between the data of different data types in the training data set and the gradients of the parameters of different levels in the language model is determined.

6. The method according to claim 5, wherein: The step of using the fine-grained influence function to evaluate the influence of the data on the parameters at each level according to the gradient information of the parameters at each level in the language model, and obtaining the training contribution of the data of different data types in the training data set to the parameters at different levels in the language model, includes: According to the gradient information of the parameters of each level corresponding to each data in the training data set, the fine-grained influence function is used to determine the gradient change information of the parameters of each level during the training process; According to the gradient change information of the parameters at each level during the training process, the training contribution of data of different data types in the training data set to the parameters at different levels in the language model is determined.

7. The method according to claim 1, further comprising: The data to be selected are classified and identified based on the data screening tool to obtain the data type of the data to be selected.

8. The method according to claim 7, wherein: Each of the different data types corresponds to a data screening tool, and the data screening tool includes the language model, and a mapping relationship between the corresponding data type and different levels of parameter gradients in the language model; The classifying and identifying the data to be selected based on the data screening tool to obtain the data type of the data to be selected includes: Inputting the data to be selected into the language model to obtain a predicted type of the data to be selected; For each of the different data types, according to the loss function between the prediction type of the to-be-selected data and the data type, determine the gradient information of the parameters of each level of the model of the to-be-selected data under the different data types; The data type of the data to be selected is determined according to the gradient information of the model parameters at each level of the data to be selected under the different data types, and the mapping relationship between the data of different data types in the training data set and the gradients of the parameters at different levels in the language model.

9. The method according to claim 1, further comprising: Evaluate and improve data quality based on the data screening tool; and / or, Directed enhancement of model capabilities is performed based on the data screening tool.

10. The method according to any one of claims 2 to 9, wherein: The knowledge management method is the DIKW knowledge management model.

11. A data processing device, comprising: An acquisition module, used to acquire a training data set, wherein the training data set includes at least one sub-data set, and each sub-data set corresponds to a data type; A training module, used to independently train the language model according to each of the sub-data sets, and during the training process, back propagation is performed on each piece of data to obtain the gradient information of the parameters at each level in the language model; An analysis module, configured to perform interpretability analysis on multiple levels in the language model according to gradient information of parameters at each level in the language model, and obtain a mapping relationship between data of different data types in the training data set and gradients of parameters at different levels in the language model; A generation module is used to generate a customized data screening tool based on the mapping relationship between data of different data types in the training data set and parameter gradients of different levels in the language model.

12. The apparatus according to claim 11, further comprising: A building module is used to build a training dataset based on a knowledge management approach through a human-computer loopback method based on a conversational language model and prompt word engineering technology.

13. The device according to claim 12, wherein: The building blocks are used to: The prompt word engineering technology is combined with the knowledge management method to process the open source data set to generate guidance instructions for the data in the open source data set; Annotating the data in the open source dataset according to the guidance instructions of the data in the open source dataset and the conversational language model; The human-machine loopback method is used to recheck and optimize the annotation of the data in the open source dataset; The annotated open source dataset is used as the training dataset.

14. The device according to claim 11, wherein: The analysis module is used to: According to the gradient information of the parameters at each level in the language model, a fine-grained influence function is used to perform interpretability analysis on multiple levels in the language model to obtain a mapping relationship between data of different data types in the training data set and the gradients of parameters at different levels in the language model.

15. The device according to claim 14, wherein: The analysis module is used to: According to the gradient information of the parameters at each level in the language model, the fine-grained influence function is used to evaluate the influence of the data on the parameters at each level, so as to obtain the training contribution of the data of different data types in the training data set to the parameters at different levels in the language model; According to the training contribution of data of different data types in the training data set to the parameters of different levels in the language model, the mapping relationship between the data of different data types in the training data set and the gradients of the parameters of different levels in the language model is determined.

16. The device according to claim 15, wherein: The analysis module is used to: According to the gradient information of the parameters of each level corresponding to each data in the training data set, the fine-grained influence function is used to determine the gradient change information of the parameters of each level during the training process; According to the gradient change information of the parameters at each level during the training process, the training contribution of data of different data types in the training data set to the parameters at different levels in the language model is determined.

17. The apparatus according to claim 11, further comprising: The classification and identification module is used to classify and identify the data to be selected based on the data screening tool to obtain the data type of the data to be selected.

18. The device according to claim 17, wherein: Each of the different data types corresponds to a data screening tool, and the data screening tool includes the language model and a mapping relationship between the corresponding data type and the parameter gradients of different levels in the language model; the classification recognition module is used to: Inputting the data to be selected into the language model to obtain a predicted type of the data to be selected; For each of the different data types, according to the loss function between the prediction type of the to-be-selected data and the data type, determine the gradient information of the parameters of each level of the model of the to-be-selected data under the different data types; The data type of the data to be selected is determined according to the gradient information of the model parameters at each level of the data to be selected under the different data types, and the mapping relationship between the data of different data types in the training data set and the gradients of the parameters at different levels in the language model.

19. The apparatus according to claim 11, further comprising an enhancement module, wherein the enhancement module is configured to: Assess and improve data quality based on the data screening tool; and / or, Directed enhancement of model capabilities is performed based on the data screening tool.

20. The device according to any one of claims 12 to 19, wherein: The knowledge management method is the DIKW knowledge management model.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

23. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10.

Images