Score prediction model training method and device and electronic equipment

Abstract

The invention provides a score prediction model training method and device and electronic equipment, and the method comprises the steps: obtaining a training sample set, and enabling each training sample in the training sample set to take the numerical values of a plurality of data of a predefined type of an enterprise as feature values, and taking the innovation evaluation score of the enterprise as a label value; respectively training a random forest sub-model, a linear regression sub-model and a gradient boosting sub-model by using the training sample set to obtain a trained random forest sub-model, a trained linear regression sub-model and a trained gradient boosting sub-model; circularly executing the following steps until a preset stop condition is met: acquiring a test sample set, and generating a new training sample set by utilizing the test sample set and an output result of the voting regression module; according to output results of the random forest sub-model, the linear regression sub-model and the gradient boosting sub-model, determining a target sub-model which needs to be continuously trained; and training the target sub-model by using the new training sample set.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method, apparatus, and electronic device for a score prediction model. Background Technology

[0002] Currently, there are many assessment platforms on the market. When companies need to evaluate a certain capability, they need to upload various relevant data to the assessment platform. The platform then evaluates the capabilities of each company based on the uploaded data. Current assessment platforms require manual intervention to convert the uploaded data into scores based on certain judgment principles, thus obtaining the evaluation results for each company. However, this method is labor-intensive and inefficient. Furthermore, human evaluation often introduces subjective factors, leading to inaccurate results.

[0003] How to quickly and accurately evaluate a company's capabilities is a technical problem that needs to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a training method, apparatus, and electronic device for a score prediction model.

[0005] According to one aspect of this application, a training method for a score prediction model is provided, the score prediction model including a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model, and a voting regression module; the voting regression module is used to synthesize the output results of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model as the output result of the score prediction model; therefore, the method includes:

[0006] Obtain a training sample set, wherein each training sample in the training sample set uses the numerical values ​​of various data of a predefined type of enterprise as feature values ​​and the enterprise's innovation evaluation score as label values;

[0007] Using the training sample set, a random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model are trained respectively to obtain the trained random forest sub-model, linear regression model, and gradient boosting model;

[0008] Repeat the following steps until the preset stopping condition is met:

[0009] Obtain a test sample set, and generate a new training sample set using the test sample set and the output of the voting regression module; determine the target sub-model that needs to be trained further based on the output of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model; train the target sub-model using the new training sample set.

[0010] According to a second aspect of this application, an innovation evaluation score prediction method based on the above-described score prediction model is provided, comprising:

[0011] Acquire various types of predefined data provided by the enterprise to be tested, and output the various types of data in parallel to the random forest sub-model, linear regression sub-model and gradient boosting sub-model of the pre-trained score prediction model;

[0012] Obtain the output of the voting regression module to get the innovation evaluation score of the enterprise under test.

[0013] According to a third aspect of this application, a training apparatus for a score prediction model is provided, the score prediction model including a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model, and a voting regression module; the voting regression module is used to synthesize the output results of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model as the output result of the score prediction model; therefore, the apparatus includes:

[0014] The sample acquisition module is used to acquire a training sample set, wherein each training sample in the training sample set uses the numerical values ​​of various data of a predefined type of enterprise as feature values ​​and the enterprise's innovation evaluation score as label values.

[0015] The model training module is used to train the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model respectively using the training sample set, so as to obtain the trained random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model.

[0016] Repeat the following steps until the preset stopping condition is met:

[0017] Obtain a test sample set, and generate a new training sample set using the test sample set and the output of the voting regression module; determine the target sub-model that needs to be trained further based on the output of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model; train the target sub-model using the new training sample set.

[0018] According to a fourth aspect of this application, an innovation evaluation score prediction device based on the above-described score prediction model is provided, comprising:

[0019] The data acquisition module is used to acquire various types of predefined data provided by the enterprise under test, and output the various types of data in parallel to the random forest sub-model, linear regression sub-model and gradient boosting sub-model of the pre-trained score prediction model;

[0020] The result acquisition module is used to acquire the output results of the voting regression module and obtain the innovation evaluation score of the enterprise under test.

[0021] According to a fifth aspect of this application, an electronic device is provided, including a processor and a memory, the memory storing a computer program, the processor executing the computer program stored in the memory to perform the steps of the method as described in any of the first or second aspects above.

[0022] According to a sixth aspect of this application, a computer program is provided, including computer instructions that, when executed by a processor, implement the steps of the method described in any of the first or second aspects above.

[0023] According to a seventh aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps of the method as described in any of the first or second aspects above. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a training method for a score prediction model provided in one embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a score prediction model provided in one embodiment of this application;

[0026] Figure 3 A schematic diagram of a random forest sub-model provided in one embodiment of this application;

[0027] Figure 4 A flowchart illustrating an innovative evaluation score prediction method provided in one embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the structure of a score prediction model training device provided in one embodiment of this application;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0030] Before introducing the embodiments of this application, it should be noted that:

[0031] Some embodiments of this application are described as processing flows. Although the various operation steps of the flow may be given sequential step numbers, the operation steps may be implemented in parallel, concurrently, or simultaneously.

[0032] The embodiments of this application may use terms such as "first," "second," etc., to describe various features, but these features should not be limited by these terms. These terms are used merely to distinguish one feature from another.

[0033] The term “and / or” may be used in the embodiments of this application, and “and / or” includes any and all combinations of one or more of the associated features listed.

[0034] It should be understood that when describing the connection or communication relationship between two components, unless it is explicitly stated that the two components are directly connected or communicate directly, the connection or communication between the two components can be understood as a direct connection or communication, or it can be understood as an indirect connection or communication through an intermediate component.

[0035] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0036] Currently, there are many assessment platforms on the market. When companies need to evaluate a certain capability, they need to upload their relevant data to the assessment platform. The assessment platform then evaluates the capabilities of each company based on the data uploaded.

[0037] Innovation is the driving force behind the development of enterprises and organizations; therefore, a company's innovation capability is a crucial indicator for evaluating its performance. Evaluating a company's innovation capability requires assessment based on various types of data. These include basic operational data (such as the number of employees, industry, and region), intellectual property data (number of valid patents, software copyrights, trademarks, and overseas patents), intellectual property operation data (number of valid invention patents related to the company's core technologies / products, transfer data, and licensing data), financial data (total R&D expenses over the years, total revenue, and the proportion of revenue from the main business), and data on the company's advancement (receiving Chinese patent awards, whether core products have achieved or are expected to achieve foreign substitution, and whether the company is a national high-tech enterprise). The types and volume of data involved are substantial. Current evaluation methods often rely on manual assessment, which is inefficient and subject to subjective human factors, resulting in unfair and inaccurate evaluations.

[0038] Based on the above problems, this application proposes a training score prediction model, which includes three different types of sub-models and a voting regression module. The voting regression module is used to integrate the output results of the three different types of sub-models to obtain the final output result of the score prediction model. Using this score prediction model, the innovation evaluation score of the enterprise to be tested can be predicted relatively accurately and quickly.

[0039] Specifically, such as Figure 1As shown, this application proposes a training method for a score prediction model, such as... Figure 2 The score prediction model shown includes a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model, and a voting regression module. The random forest sub-model, linear regression sub-model, and gradient boosting sub-model are used to input various types of predefined data from enterprises in parallel. The voting regression module is used to synthesize the output results of the random forest sub-model, linear regression sub-model, and gradient boosting sub-model as the output result of the score prediction model.

[0040] The training method for this score prediction model includes the following.

[0041] S101, Obtain the training sample set, wherein each training sample in the training sample set uses the numerical values ​​of various data of a predefined type for the enterprise as feature values ​​and the enterprise's innovation evaluation score as label value. The innovation evaluation score can be manually labeled data.

[0042] Table 1 shows a schematic representation of the training sample set, which includes N training samples, each representing data from a single enterprise. Specifically, each training sample contains feature values ​​X1, X2, ..., X... m， Each feature represents a predefined set of numerical values ​​for various data types, with the label value being a manually annotated innovation evaluation score for that enterprise. These feature values ​​can be discrete or continuous; for example, if the feature is "industry affiliation," the feature value is discrete, while if the feature is "number of valid patents held," the feature value is continuous. This application does not limit the type of feature values. Different encoding techniques can be used to encode discrete and continuous feature values; specific techniques can be referenced, and this application does not impose any limitations.

[0043] Training samples Eigenvalues Tag value 1 <![CDATA[X1、X2....X m ]]> <![CDATA[Innovation evaluation score Y1]]> 2 <![CDATA[X1、X2....X m ]]> <![CDATA[Innovation evaluation score Y2]]> ... ... ... N <![CDATA[X1、X2....X m ]]> <![CDATA[Innovation evaluation score Y N >

[0044] Table 1

[0045] S102, using the training sample set, train the random forest sub-model, linear regression sub-model, and gradient boosting sub-model respectively to obtain the trained random forest sub-model, linear regression model, and gradient boosting model;

[0046] During training, the same training sample set can be used to train the three sub-models in the score prediction model separately. Specific training methods are detailed below and will not be elaborated upon here.

[0047] After training the random forest sub-model, linear regression sub-model, and gradient boosting sub-model using the training sample set, the score prediction model already possesses the ability to predict the innovation evaluation score of the enterprise under test based on various predefined types of data. To further improve the model's accuracy, this application proposes performing further targeted training on each sub-model using steps S103-S106, thereby enhancing the overall accuracy of the score prediction model.

[0048] S103, determine whether the preset stopping condition has been met. If not, execute S104-S106 repeatedly until the preset stopping condition is met; if yes, the training ends.

[0049] S104, Obtain the test sample set, and use the test sample set and the output of the voting regression module to generate a new training sample set;

[0050] S105. Based on the output results of the random forest sub-model, linear regression sub-model, and gradient boosting sub-model, determine the target sub-model that needs to be trained further.

[0051] S106, using the new training sample set to train the target sub-model.

[0052] The steps S104-S106 above will be explained in detail below.

[0053] In S104, when obtaining the test sample set, it can be based on the range of innovation evaluation scores, that is, the test samples in the test sample set cover every innovation evaluation score within the range of innovation evaluation scores, or cover the innovation evaluation scores of each segment.

[0054] For example, if the innovation evaluation score is a positive integer from 0 to 100, and the test sample set includes 100 test samples, then each selected test sample can correspond to an innovation evaluation score.

[0055] For example, if the innovation evaluation score is 0-100 points, and the test sample set includes 100 test samples, the innovation evaluation score can be divided into several segments, such as 5 segments: 0-20, 21-40, 41-60, 61-80, and 81-100. The obtained test sample set must then cover test samples for each segment. For instance, 20 test samples could be obtained from each of the 5 segments to form the 100 test samples in the test sample set.

[0056] Obtaining a test sample set based on the range of innovation evaluation scores can ensure the completeness of the test and guarantee that the accuracy of various prediction results can be tested.

[0057] After obtaining the test sample set, a new training sample set can be generated in the following ways.

[0058] In one implementation, a test sample may be determined as a test sample to be improved if the difference between the actual innovation evaluation score in any test sample and the innovation evaluation score output by the voting regression module is greater than a first preset score.

[0059] For any test sample to be improved, obtain several data samples whose difference between the actual innovation evaluation score and the actual innovation evaluation score of the test sample to be improved is less than a second preset score. Use the obtained data samples and the test sample to be improved to form a new training sample set.

[0060] Taking a first preset score of 5 points and a second preset score of 3 points as an example, if the actual innovation evaluation score of a certain test sample is 50 points, and the innovation evaluation score obtained after inputting the feature value of this test sample into the score prediction model is 75 points, then the difference between the innovation evaluation score of 75 points output by the score prediction model (i.e., the voting regression module) and the actual innovation evaluation score of 50 points is 25 points, which exceeds the first preset score of 5 points. Therefore, this test sample is identified as a test sample to be improved. Simultaneously, data samples with innovation evaluation scores in the range [47, 53] from the test sample set or other datasets are obtained, and a new training sample set is formed using the obtained data samples and the test sample to be improved.

[0061] In another implementation, if the difference between the true innovation evaluation score in any test sample and the innovation evaluation score output by the voting regression module is greater than a first preset score, the test sample is determined to be a test sample to be improved; for any test sample to be improved, several data samples with the same true innovation evaluation score as the test sample to be improved are obtained, and a new training sample set is formed by using the obtained data samples and the test sample to be improved.

[0062] The first implementation method yields a larger number of training samples, resulting in a higher accuracy of the trained model. The second implementation method yields fewer training samples, but subsequent training is more efficient. Those skilled in the art can choose the method that best suits their needs; this application does not impose any limitations.

[0063] S105 will be explained below.

[0064] Since the output of the score prediction model is the combined result of the voting regression module's outputs from the three sub-models—specifically, the voting regression module's output is the average of the innovation evaluation scores output by the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model—if the voting regression module's output for the test sample to be improved differs significantly from the actual innovation evaluation score (i.e., greater than the first preset score), it indicates that at least one of the three sub-models outputs an innovation evaluation score that differs significantly from the actual innovation evaluation score.

[0065] Therefore, this application proposes, for any test sample to be improved, to identify the target sub-model among the random forest sub-model, linear regression sub-model, and gradient boosting sub-model whose output innovation evaluation score differs from the actual innovation evaluation score of the test sample to be improved by more than a third preset score.

[0066] For example, if the third preset score is 8 points, the voting regression module outputs 75 points for the test sample to be improved, the true innovation evaluation score is 50 points, the random forest sub-model outputs 100 points, the linear regression sub-model outputs 50 points, and the gradient boosting sub-model outputs 75 points, then the random forest sub-model and the gradient boosting sub-model are determined as the target sub-models.

[0067] After the target sub-model is determined, S106 can be executed.

[0068] Using the example above as an example, if the random forest sub-model and the gradient boosting sub-model are determined as the target sub-models, then the random forest sub-model and the gradient boosting sub-model can be trained respectively using a new training sample set (a new training sample set consisting of data samples with innovation evaluation scores in the range of [47,53] and test samples to be improved), so that the random forest sub-model and the gradient boosting sub-model can be specifically improved for the prediction ability of the random forest sub-model and the gradient boosting sub-model for samples with innovation evaluation scores in the range of [47,53].

[0069] Using the above method, a new training sample set is generated based on the overall prediction results of the score prediction model. The target sub-models that need to be further selected are then selected. The target sub-models are then trained in a targeted manner based on the new training sample set. On the one hand, all target sub-models can be trained. On the other hand, new training sample sets are generated only for scores or segments with poor prediction performance. Based on these two aspects, the prediction ability of the target sub-models can be improved in a targeted manner, while reducing the computing power required for training and improving training efficiency.

[0070] Understandably, in S104, when determining the test samples to be improved, multiple test samples can be determined simultaneously. For each test sample to be improved, a new training sample set is generated using the above method, and a target sub-model is determined. The target sub-model is then trained using the new training sample set. After the target sub-model has been trained for each test sample to be improved, S103 is executed, i.e., it is determined whether to continue the loop operation.

[0071] In one implementation, the preset stopping condition in S103 may include at least one of the following conditions: reaching a preset number of training iterations, the model prediction accuracy reaching a preset threshold, or no test samples to be improved being found for two consecutive times.

[0072] By training the sub-model multiple times using the above method, the overall accuracy of the score prediction model can be guaranteed to be strong.

[0073] The training methods for the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model are explained below.

[0074] The training process for a random forest sub-model may include the following steps.

[0075] Obtain an initial training sample set, in which each training sample in the initial training sample set uses the numerical values ​​of various data of a predefined type of enterprise as feature values ​​and the enterprise's innovation evaluation score as label values;

[0076] For the N regression trees to be generated, the initial training sample set is sampled with replacement to obtain N training sample sets corresponding to the N regression trees. Each training sample set includes several training samples. Sampling with replacement can increase the perturbation of the samples.

[0077] N regression trees are generated in parallel using N training sample sets;

[0078] The outputs of N regression trees are combined to obtain a random forest sub-model, where the output of the random forest sub-model can be the average of the innovation evaluation scores of the N regression trees after training.

[0079] Generating N regression trees in parallel using N training sample sets may include the following: For any training sample set, repeatedly perform the following steps to determine the attributes of each partition node of the regression tree until the regression tree reaches a preset depth:

[0080] For each type of data in a predefined dataset, each value in each data type is used as an attribute of a splitting node. The squared losses corresponding to the innovation evaluation scores of the two groups of data separated by each splitting node attribute are determined. Specifically, the scoring loss can be the variance of the innovation evaluation scores. For example, if the two groups of data are Group A and Group B, each group includes the innovation evaluation scores of 5 companies, with Group A containing scores of 50, 50, 50, 80, and 60, and Group B containing scores of 70, 60, 60, 70, and 60, then the squared loss for Group A is the variance of these 5 values ​​(50, 50, 50, 80, 60), which is 136, and the squared loss for Group B is the variance of these 5 values ​​(70, 60, 60, 70, 60), which is 24.

[0081] The overall loss for this layer is determined based on the obtained two squared losses and the amount of data in each group. Specifically, the overall loss for this layer can be the sum of the product of the proportion of each group's data to the total data and its squared loss. Using the above example, the overall loss for this layer is 0.5*136 + 0.5*24 = 80.

[0082] By utilizing the comprehensive loss of this layer and the comprehensive loss of the attributes of the upper-layer partitioning nodes, the partitioning node attribute with the largest change in comprehensive loss is determined as the partitioning node attribute of this layer. Each partitioning node attribute includes data type and data value. After determining the comprehensive loss of this layer for each partitioning node, based on the comprehensive loss of the upper-layer partitioning node attributes and the determined multiple comprehensive losses of this layer, the partitioning node attribute with the largest change in comprehensive loss can be determined as the partitioning node attribute of this layer. The comprehensive loss of the upper-layer partitioning node attributes refers to the variance of each innovation evaluation score in the upper-layer partitioning nodes. Selecting the partitioning node attribute with the largest change in comprehensive loss as the partitioning node attribute of this layer is to construct the regression tree in a way that maximizes information gain, thus significantly increasing the information content of each layer of the regression tree.

[0083] like Figure 3 The diagram illustrates the random forest sub-model, where N trees each predict an innovation evaluation score, and the average of these scores is used as the output of the random forest sub-model. It is understood that the tree structure shown is merely illustrative; the depth and feature selection of each tree may be the same or different depending on the training results.

[0084] The training process for a gradient boosting sub-model may include the following steps.

[0085] Obtain initial predicted values ​​and update the labels in the initial training sample set based on the initial predicted values; among them, the residual can be obtained by subtracting the initial predicted values ​​from the true innovation evaluation scores in the initial training sample set, and the residual can be used as the labels in the training sample set.

[0086] In the initial training sample set, each training sample uses the numerical values ​​of various data of the enterprise's predefined type as feature values ​​and the enterprise's innovation evaluation score as label values;

[0087] Construct the first iteration regression tree using the updated training sample set;

[0088] Repeat the following steps until the preset number of iterations is reached:

[0089] The labels in the initial training sample set are updated using the existing regression tree and initial predicted values;

[0090] Construct the regression tree for this iteration using the updated training sample set;

[0091] The gradient boosting sub-model is obtained by superimposing the initial predicted values ​​and each obtained regression tree.

[0092] When constructing the first iteration regression tree using the updated training sample set, the following steps can be repeated to determine the attributes of each partition node of the regression tree until the regression tree reaches a preset depth:

[0093] For each type of data in the predefined data types, each value in each type of data is used as an attribute of the splitting node, and the squared loss of the innovation evaluation score of the two groups of data divided by each splitting node attribute is determined.

[0094] The attribute of the partition node that minimizes the squared loss is taken as the attribute of the partition node in this layer; where each partition node's attribute includes the data type and the data value.

[0095] When updating the labels in the initial training sample set using the existing regression tree and initial predicted values, the specific method can be to accumulate the initial predicted values ​​and the predicted values ​​of the existing regression tree, and then subtract the accumulated values ​​from the actual innovation evaluation scores in the initial training sample set to obtain the residuals, which are then used as the labels in the training sample set.

[0096] Constructing the regression tree for this iteration using the updated training sample set can be achieved by repeatedly performing the following steps to determine the attributes of each partition node in the regression tree until the regression tree reaches a preset depth:

[0097] For each type of data in the predefined data types, each value in each type of data is used as an attribute of the splitting node, and the squared loss of the innovation evaluation score of the two groups of data divided by each splitting node attribute is determined.

[0098] The attribute of the partition node that minimizes the squared loss is taken as the attribute of the partition node in this layer; where each partition node's attribute includes the data type and the data value.

[0099] The output of the gradient boosting sub-model can be the sum of the initial predicted value and the innovation evaluation score of each regression tree.

[0100] That is, the gradient boosting sub-model also includes several regression trees. Multiple regression trees are trained sequentially and have a strong dependency relationship. The loss information of the current regression tree is used to train the newly added regression tree. Then, the trained regression trees are combined into a model in an accumulative form. Finally, the output of the gradient boosting sub-model includes the superposition of several regression trees to ensure the accuracy of the output as much as possible.

[0101] The training process for a linear regression model may include the following steps.

[0102] Determine the initial parameters of the linear regression model and obtain the training sample set, in which the numerical values ​​of various data of predefined types of enterprises are used as feature values ​​and the innovation evaluation scores of enterprises are used as label values;

[0103] The initial parameters of the linear regression model are tuned using the obtained training sample set and gradient descent method to obtain the trained linear regression model.

[0104] During the training of the score prediction model, a serial training method for regression trees is used to obtain the final gradient boosting sub-model, while a parallel training method is used to obtain the random forest model. Therefore, although both use the same training sample set and are trained using regression trees, the different training methods result in slightly different prediction results. The final output of the score prediction model combines the outputs of these two models, thus leading to more accurate predictions. Furthermore, both the gradient boosting sub-model and the random forest model are non-parametric models, while the linear regression model is a parametric model. Therefore, the score prediction model combines the outputs of both non-parametric and parametric models, further improving the accuracy of the model's output.

[0105] Considering that after predicting the innovation evaluation score of an enterprise based on multiple data of a predefined type, the evaluator often needs to know the impact of each data point of the predefined type on the innovation evaluation score, and also needs to know the proportion of the innovation evaluation score of each data point of the predefined type in the total score.

[0106] Therefore, this application proposes to generate a test dataset for any target data of a predefined type of data. The test dataset includes N test data, where each test data differs only in the value of the target data, and N is the number of possible values ​​for the target data. The test dataset is then input into a trained score prediction model to obtain the innovation evaluation score corresponding to each test data. Based on the obtained innovation evaluation score, the correlation between the target data and the innovation evaluation score is obtained.

[0107] For example, for target data such as "industry", a test dataset as shown in Table 2 can be generated.

[0108]

[0109] Table 2

[0110] The dataset to be tested was input into the trained score prediction model, and the innovation evaluation score corresponding to each dataset was obtained as shown in Table 3.

[0111]

[0112]

[0113] Table 3

[0114] After obtaining the data shown in Table 3, the relationship between the target data (i.e., the data within the industry) and the innovation evaluation score can be determined. Any relational model can be used to establish the association between the target data and the innovation evaluation score; this application does not impose any limitations on this. Furthermore, after establishing the association between all target data and the innovation evaluation score, the proportion of the innovation evaluation score for each type of data in the total score can be obtained.

[0115] Based on the above score prediction model, this application also proposes an innovative evaluation score prediction method, which is as follows: Figure 4 As shown:

[0116] S401: Obtain multiple types of predefined data provided by the enterprise under test, and output the multiple types of data in parallel to the random forest sub-model, linear regression sub-model, and gradient boosting sub-model of the pre-trained score prediction model;

[0117] S402, Obtain the output of the voting regression module to obtain the innovation evaluation score of the enterprise under test. The output of the voting regression module includes the average innovation evaluation score from the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model.

[0118] The input and output of this score prediction model can be referred to above, and will not be repeated here. Using the above method, the innovation evaluation score of the enterprise under test is predicted based on the pre-trained score prediction model. Compared with manual evaluation, this can greatly improve the evaluation efficiency and accuracy.

[0119] like Figure 5 As shown, based on the same inventive concept, this application also proposes a training device for a score prediction model, wherein the score prediction model includes a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model, and a voting regression module; the voting regression module is used to synthesize the output results of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model as the output result of the score prediction model; therefore, the device includes:

[0120] The sample acquisition module 510 is used to acquire a training sample set, wherein each training sample in the training sample set uses the numerical values ​​of various data of a predefined type of enterprise as feature values ​​and the enterprise's innovation evaluation score as label values.

[0121] The model training module 520 is used to train the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model respectively using the training sample set, so as to obtain the trained random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model.

[0122] Repeat the following steps until the preset stopping condition is met:

[0123] Obtain a test sample set, and generate a new training sample set using the test sample set and the output of the voting regression module; determine the target sub-model that needs to be trained further based on the output of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model; train the target sub-model using the new training sample set.

[0124] In one implementation, the model training module 520 is specifically used to determine that a test sample is a test sample to be improved when the difference between the actual innovation evaluation score in any test sample and the innovation evaluation score output by the voting regression module is greater than a first preset score.

[0125] For any test sample to be improved, obtain several data samples whose difference between the actual innovation evaluation score and the actual innovation evaluation score of the test sample to be improved is less than a second preset score, and use the obtained data samples and the test sample to be improved to form a new training sample set.

[0126] In one implementation, the model training module 520 is specifically used to determine, for any test sample to be improved, the sub-model whose output innovation evaluation score differs from the actual innovation evaluation score of the test sample to be improved by more than a third preset score among the random forest sub-model, linear regression sub-model, and gradient boosting sub-model as the target sub-model.

[0127] In one implementation, the model training module 520 is specifically used for...

[0128] Obtain an initial training sample set, in which the numerical values ​​of various data of predefined types of enterprises are used as feature values ​​and the innovation evaluation scores of enterprises are used as label values.

[0129] For the N regression trees to be generated, the initial training sample set is sampled with replacement to obtain N training sample sets corresponding to the N regression trees. Each training sample set includes several training samples.

[0130] N regression trees are generated in parallel using N training sample sets;

[0131] The output results of the N regression trees are combined to obtain the random forest sub-model.

[0132] In one implementation, the model training module 520 is specifically configured to repeatedly perform the following steps to determine the attributes of each partition node of the regression tree for any training sample set, until the regression tree reaches a preset depth:

[0133] For each type of data in the predefined data types, each value in each type of data is used as the attribute of the splitting node, and the two squared losses corresponding to the innovation evaluation scores of the two groups of data divided by each splitting node attribute are determined respectively.

[0134] The comprehensive loss for this layer is determined based on the obtained two-squared losses and the amount of data in each group.

[0135] Using the overall loss of this layer and the overall loss of the attributes of the partition nodes in the upper layer, the attribute of the partition node with the largest change in overall loss is determined as the attribute of the partition node in this layer; where the attribute of each partition node includes the data type and the data value.

[0136] In one implementation, the output of the random forest sub-model includes:

[0137] The average of the innovation evaluation scores output by the N regression trees after training.

[0138] In one implementation, the model training module 520 is specifically used to obtain initial predicted values ​​and update the labels in the initial training sample set based on the initial predicted values; wherein each training sample in the initial training sample set uses the values ​​of multiple data of a predefined type of enterprise as feature values ​​and the enterprise's innovation evaluation score as label values.

[0139] Construct the first iteration regression tree using the updated training sample set;

[0140] Repeat the following steps until the preset number of iterations is reached:

[0141] The labels in the initial training sample set are updated using the existing regression tree and initial predicted values;

[0142] Construct the regression tree for this iteration using the updated training sample set;

[0143] The gradient boosting sub-model is obtained by superimposing the initial predicted values ​​and each obtained regression tree.

[0144] In one implementation, the model training module 520 is specifically used to obtain a residual by subtracting the initial predicted value from the actual innovation evaluation score in the initial training sample set, and to use the residual as a label in the training sample set.

[0145] Repeat the following steps to determine the attributes of each partition node in the regression tree until the regression tree reaches the preset depth:

[0146] For each type of data in the predefined data types, each value in each type of data is used as an attribute of the splitting node, and the squared loss of the innovation evaluation score of the two groups of data divided by each splitting node attribute is determined.

[0147] The attribute of the partition node that minimizes the squared loss is taken as the attribute of the partition node in this layer; where each partition node's attribute includes the data type and the data value.

[0148] In one implementation, the model training module 520 is specifically used to accumulate the initial predicted value and the predicted value of the existing regression tree, and to obtain the residual by subtracting the accumulated value from the true innovation evaluation score in the initial training sample set, and to use the residual as the label in the training sample set.

[0149] Repeat the following steps to determine the attributes of each partition node in the regression tree until the regression tree reaches the preset depth:

[0150] For each type of data in the predefined data types, each value in each type of data is used as an attribute of the splitting node, and the squared loss of the innovation evaluation score of the two groups of data divided by each splitting node attribute is determined.

[0151] The attribute of the partition node that minimizes the squared loss is taken as the attribute of the partition node in this layer; where each partition node's attribute includes the data type and the data value.

[0152] In one implementation, the output of the gradient boosting sub-model includes:

[0153] The sum of the initial predicted value and the innovation evaluation score output by each regression tree.

[0154] In one implementation, the model training module 520 is further configured to generate a test dataset for any one of the target data of the predefined types of data, wherein the test dataset includes N test data, wherein each test data has only different values ​​for the target data, and N is the number of possible values ​​for the target data.

[0155] The dataset to be tested is input into the trained score prediction model to obtain the innovation evaluation score corresponding to each piece of data to be tested;

[0156] Based on the obtained innovation evaluation score, the correlation between the target data and the innovation evaluation score is obtained.

[0157] Based on the same inventive concept, this application also proposes an innovation evaluation score prediction device for the score prediction model described above, comprising:

[0158] The data acquisition module is used to acquire various types of predefined data provided by the enterprise under test, and output the various types of data in parallel to the random forest sub-model, linear regression sub-model and gradient boosting sub-model of the pre-trained score prediction model;

[0159] The result acquisition module is used to acquire the output results of the voting regression module and obtain the innovation evaluation score of the enterprise under test.

[0160] The solutions in this application embodiment can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0161] Furthermore, embodiments of the present invention also provide an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are connected via the bus. When the computer program is executed by the processor, it implements various processes in various embodiments of the training method for the score prediction model or the innovative evaluation score prediction method, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0162] For details, see Figure 6As shown, the electronic device includes a bus 1110, a processor 1120, a transceiver 1130, a bus interface 1140, a memory 1150, and a user interface 1160.

[0163] In this embodiment of the invention, the electronic device further includes: a computer program stored in a memory 1150 and executable on a processor 1120, wherein the computer program, when executed by the processor 1120, implements a training method for a score prediction model or an innovative evaluation score prediction method.

[0164] Transceiver 1130 is used to receive and send data under the control of processor 1120.

[0165] In this embodiment of the invention, a bus architecture (represented by bus 1110) is used. Bus 1110 may include any number of interconnected buses and bridges. Bus 1110 connects various circuits, including one or more processors represented by processor 1120 and memory represented by memory 1150.

[0166] Bus 1110 represents one or more of several types of bus architectures, including memory buses and memory controllers, peripheral buses, Accelerated Graphics Port (AGP), processors, or local buses using any bus architecture from various bus architectures. As an example and not a limitation, such architectures include: Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) buses, and Peripheral Component Interconnect (PCI) buses.

[0167] The processor 1120 can be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The processors mentioned above include: general-purpose processors, central processing units (CPUs), network processors (NPs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), programmable logic arrays (PLAs), microcontroller units (MCUs) or other programmable logic devices, discrete gates, transistor logic devices, and discrete hardware components. They can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. For example, the processor can be a single-core processor or a multi-core processor, and the processor can be integrated on a single chip or located on multiple different chips.

[0168] Processor 1120 can be a microprocessor or any conventional processor. The method steps disclosed in the embodiments of the present invention can be directly executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in readable storage media known in the art, such as Random Access Memory (RAM), Flash Memory, Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), registers, etc. The readable storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0169] Bus 1110 can also connect various other circuits, such as peripheral devices, voltage regulators, or power management circuits. Bus interface 1140 provides an interface between bus 1110 and transceiver 1130, all of which are well known in the art. Therefore, embodiments of the present invention will not be described further.

[0170] Transceiver 1130 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. For example, transceiver 1130 receives external data from other devices, and transceiver 1130 is used to send data processed by processor 1120 to other devices. Depending on the nature of the computer system, a user interface 1160 may also be provided, such as a touchscreen, physical keyboard, monitor, mouse, speaker, microphone, trackball, joystick, or stylus.

[0171] It should be understood that, in embodiments of the present invention, memory 1150 may further include memory remotely configured relative to processor 1120, and such remotely configured memory can be connected to a server via a network. One or more portions of the aforementioned network may be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), wireless wide area network (WWAN), metropolitan area network (MAN), Internet, public switched telephone network (PSTN), ordinary old-style telephone service (POTS), cellular telephone network, wireless network, Wi-Fi network, and combinations of two or more of the aforementioned networks. For example, cellular telephone networks and wireless networks can be Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), WiMAX, General Packet Radio Service (GPRS), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), Advanced Long Term Evolution (LTE-A), Universal Mobile Telecommunications System (UMTS), Enhanced Mobile Broadband (eMBB), Massive Machine Type Communication (mMTC), Ultra Reliable Low Latency Communications (uRLLC), etc.

[0172] It should be understood that the memory 1150 in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory. Non-volatile memory includes: read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0173] Volatile memory includes random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1150 of the electronic device described in this embodiment includes, but is not limited to, the above-described and any other suitable types of memory.

[0174] In this embodiment of the invention, the memory 1150 stores the following elements of the operating system 1151 and the application 1152: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0175] Specifically, the operating system 1151 includes various system programs, such as a framework layer, a core library layer, and a driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 1152 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this embodiment of the invention can be included in the application program 1152. The application program 1152 includes applets, objects, components, logic, data structures, and other computer system executable instructions that perform specific tasks or implement specific abstract data types.

[0176] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the various processes of the various embodiments of the above-described score prediction model training method or innovative evaluation score prediction method, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0177] Computer-readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination thereof. Computer-readable storage media include: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (e.g., punched cards or raised structures in grooves on which instructions are recorded), or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in the embodiments of the present invention, computer-readable storage media do not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0178] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0179] Computer program code for performing the operations of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or in one or more programming languages ​​or combinations thereof. The programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0180] This invention also provides a computer program comprising one or more computer instructions. When the computer instructions are loaded and executed on a processor, all or part of the processes or functions described in the embodiments of this application are generated.

[0181] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0182] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A training method for a score prediction model, characterized in that: The score prediction model includes a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model and a voting regression module; the voting regression module is used to integrate the output results of the random forest sub-model, the linear regression sub-model and the gradient boosting sub-model as the output result of the score prediction model; So the methods include: Obtaining a training sample set, wherein each training sample in the training sample set uses numerical values ​​of multiple data of predefined types of the enterprise as feature values ​​and uses the innovation evaluation score of the enterprise as a label value; Using the training sample set to train a random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model respectively, to obtain a trained random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model; The following steps are executed repeatedly until the preset stop condition is reached: Obtain a test sample set, and generate a new training sample set using the test sample set and the output results of the voting regression module; determine the target sub-model that needs to be further trained based on the output results of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model; and train the target sub-model using the new training sample set.

2. The method according to claim 1, characterized in that The generating a new training sample set by using the test sample set and the output result of the voting regression module comprises: When it is determined that the difference between the real innovation evaluation score in any test sample and the innovation evaluation score output by the voting regression module is greater than a first preset score, the test sample is determined to be a test sample to be improved; For any test sample to be improved, obtain several data samples whose difference between the real innovation evaluation score and the real innovation evaluation score of the test sample to be improved is less than a second preset score, and use the obtained several data samples and the test sample to be improved to form a new training sample set.

3. The method according to claim 2, characterized in that Determining the target sub-model to be further trained according to the output results of the random forest sub-model, the linear regression sub-model and the gradient boosting sub-model includes: For any test sample to be improved, determine among the random forest sub-model, linear regression sub-model and gradient boosting sub-model, the sub-model whose output innovation evaluation score and the actual innovation evaluation score of the test sample to be improved is greater than a third preset score as the target sub-model.

4. The method according to claim 1, characterized in that: The training process of the random forest sub-model includes: An initial training sample set is obtained, wherein the values ​​of various data of predefined types of the enterprise are used as feature values, and the innovation evaluation score of the enterprise is used as a label value in the initial training sample set; For the N regression trees to be generated, the initial training sample set is sampled with replacement to obtain N training sample sets corresponding to the N regression trees, each training sample set including a number of training samples; Use N training sample sets to generate N regression trees in parallel; The output results of the N regression trees are combined to obtain a random forest sub-model.

5. The method according to claim 4, characterized in that The method of using N training sample sets to generate N regression trees in parallel includes: For any training sample set, repeat the following steps to determine the attributes of each layer of the regression tree until the regression tree reaches the preset depth: For each of the plurality of predefined types of data, each value in each of the data is used as an attribute of a partitioning node, and two square losses corresponding to the innovation evaluation scores of two groups of data partitioned by the attribute of each partitioning node are determined; Based on the obtained two square losses and the amount of data in each group, the comprehensive loss of this layer is determined; By using the comprehensive loss of this layer and the comprehensive loss of the partition node attributes of the upper layer, the partition node attribute with the largest comprehensive loss change is determined as the partition node attribute of this layer; wherein the attribute of each partition node includes the data type and the data value.

6. The method according to claim 5, characterized in that The output results of the random forest sub-model include: The average value of the innovation evaluation scores output by the N regression trees after training.

7. The method according to claim 1, characterized in that The training process of the gradient boosting sub-model includes: Obtaining an initial prediction value, and updating the labels in the initial training sample set based on the initial prediction value; wherein each training sample in the initial training sample set uses the numerical values ​​of multiple data of predefined types of the enterprise as a feature value, and uses the innovation evaluation score of the enterprise as a label value; Use the updated training sample set to construct the first round of iterative regression tree; The following steps are executed repeatedly until the preset number of iterations is reached: Use the existing regression tree and initial prediction value to update the labels in the initial training sample set; Use the updated training sample set to build the regression tree for this iteration; The initial prediction value and each regression tree obtained are superimposed to obtain the gradient boosting sub-model.

8. The method according to claim 7, characterized in that The updating of the labels in the initial training sample set based on the initial prediction value includes: A residual is obtained by subtracting the initial predicted value from the actual innovation evaluation score in the initial training sample set, and the residual is used as a label in the training sample set; The method of constructing a first-round iterative regression tree using the updated training sample set includes: Repeat the following steps to determine the attributes of each layer of the regression tree until the regression tree reaches the preset depth: For each of the plurality of predefined types of data, each value in each of the data is used as an attribute of a partitioning node, and the square loss of the innovation evaluation score of two groups of data partitioned by the attribute of each partitioning node is determined; The attributes of the partition node with the smallest square loss are used as the attributes of the partition node of this layer; wherein the attributes of each partition node include the data type and the data value.

9. The method according to claim 7, characterized in that: The updating of labels in the initial training sample set using the current existing regression tree and the initial prediction value includes: Accumulate the initial prediction value and the prediction value of the existing regression tree, and make a difference between the accumulated value and the real innovation evaluation score in the initial training sample set to obtain the residual, which is used as the label in the training sample set; The step of constructing the regression tree of this iteration using the updated training sample set includes: Repeat the following steps to determine the attributes of each layer of the regression tree until the regression tree reaches the preset depth: For each of the plurality of predefined types of data, each value in each of the data is used as an attribute of a partitioning node, and the square loss of the innovation evaluation score of two groups of data partitioned by the attribute of each partitioning node is determined; The attributes of the partition node with the smallest square loss are used as the attributes of the partition node of this layer; wherein the attributes of each partition node include the data type and the data value.

10. The method according to claim 9, characterized in that The output results of the gradient boosting sub-model include: The initial prediction value and the superposition value of the innovation evaluation score of the output of each regression tree.

11. The method according to claim 1, characterized in that: Also includes: For any target data among the plurality of predefined types of data, a test data set is generated, wherein the test data set includes N pieces of test data, wherein only the value of the target data is different in each piece of test data, and N is the number of possible values ​​of the target data; Input the test data sets into the trained score prediction model respectively to obtain the innovation evaluation score corresponding to each test data; Based on the obtained innovation evaluation score, the correlation relationship between the target data and the innovation evaluation score is obtained.

12. A method for predicting an innovation evaluation score based on the score prediction model of claim 1, characterized in that: The method comprises: Acquire multiple data of predefined types provided by the enterprise to be tested, and output the multiple data in parallel to a random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model of a pre-trained score prediction model; The output result of the voting regression module is obtained to obtain the innovation evaluation score of the enterprise to be tested.

13. A training device for a score prediction model, the score prediction model comprising a random forest sub-model, a linear regression sub-model, a gradient boosting sub-model and a voting regression module; the voting regression module is used to integrate the output results of the random forest sub-model, the linear regression sub-model and the gradient boosting sub-model as the output result of the score prediction model; characterized in that, The device comprises: A sample acquisition module, used to acquire a training sample set, wherein each training sample in the training sample set uses the numerical values ​​of multiple data of predefined types of the enterprise as feature values ​​and uses the innovation evaluation score of the enterprise as a label value; A model training module, used to respectively train a random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model using the training sample set to obtain a trained random forest sub-model, a linear regression sub-model, and a gradient boosting sub-model; The following steps are executed repeatedly until the preset stop condition is reached: Obtain a test sample set, and generate a new training sample set using the test sample set and the output results of the voting regression module; determine the target sub-model that needs to be further trained based on the output results of the random forest sub-model, the linear regression sub-model, and the gradient boosting sub-model; and train the target sub-model using the new training sample set.

14. An innovation evaluation score prediction device based on the score prediction model of claim 1, characterized in that: The device comprises: A data acquisition module, used to acquire multiple data of predefined types provided by the enterprise to be tested, and output the multiple data in parallel to the random forest sub-model, linear regression sub-model and gradient boosting sub-model of the pre-trained score prediction model; The result acquisition module is used to obtain the output result of the voting regression module to obtain the innovation evaluation score of the enterprise to be tested.

15. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, wherein: The processor executes the computer program stored in the memory to implement the steps in the method according to any one of claims 1 to 12.

16. A computer program comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps in the method according to any one of claims 1 to 12 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps in the method according to any one of claims 1 to 12.

Images