Data processing method and device, equipment and storage medium

Abstract

The application provides a data processing method and device, equipment and a storage medium. It relates to the technical field of big data analysis. The method comprises the following steps: when receiving to-be-tested work order data, determining to-be-processed work order data corresponding to the to-be-tested work order data; processing the to-be-processed work order data based on a configuration model to determine a first evaluation result; processing the to-be-processed work order data based on an evaluation model to determine a second evaluation result; determining a target evaluation item corresponding to the to-be-processed work order data based on the first evaluation result, the second evaluation result and corresponding evaluation attributes, so as to evaluate the to-be-processed work order data based on the target evaluation item. The application solves the problem that the prior art requires manual evaluation based on product information, has a high requirement for professional knowledge and a poor evaluation efficiency, improves the evaluation efficiency and accuracy, reduces the evaluation cost and improves the test effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data data analysis, and particularly relates to a data processing method and device, equipment and a storage medium. BACKGROUND

[0002] At present, whether online products or offline products will involve product testing, and the performance of the product is continuously optimized through testing. In this process, non-functional testing of the product is involved, such as performance testing, stress testing, load testing, low resource testing, capacity testing, and repetitive testing.

[0003] In the prior art, when determining which non-functional tests need to be performed, a tester repeatedly checks product information submitted by product development technicians, and manually evaluates the product information to obtain non-functional test information. This method requires manual evaluation based on product information, and has the problems of high requirement for professional knowledge and poor evaluation efficiency. SUMMARY

[0004] The present application provides a data processing method, device, equipment and storage medium to improve the efficiency and accuracy of evaluation, reduce the cost of evaluation, and improve the testing effect.

[0005] In a first aspect, the present application provides a data processing method, comprising: determining to-be-processed work order data corresponding to to-be-tested work order data when the to-be-tested work order data is received; processing the to-be-processed work order data based on a pre-configured configuration model to determine a first evaluation result corresponding to the to-be-processed data; processing the to-be-processed work order data based on a pre-trained evaluation model to determine a second evaluation result corresponding to the to-be-processed data; determining a target evaluation item corresponding to the to-be-processed work order data based on the first evaluation result, the second evaluation result, and a corresponding evaluation attribute, and evaluating and testing the to-be-processed work order data based on the target evaluation item.

[0006] In a second aspect, the present application provides a data processing apparatus, comprising: a to-be-processed work order data determination module configured to determine to-be-processed work order data corresponding to to-be-tested work order data when the to-be-tested work order data is received; a first evaluation result determination module configured to determine a first evaluation result corresponding to the to-be-processed data by processing the to-be-processed work order data based on a pre-configured configuration model; a second evaluation result determination module configured to determine a second evaluation result corresponding to the to-be-processed data by processing the to-be-processed work order data based on a pre-trained evaluation model; and a target evaluation item determination module configured to determine a target evaluation item corresponding to the to-be-processed work order data based on the first evaluation result, the second evaluation result, and corresponding evaluation attributes, so as to evaluate and test the to-be-processed work order data based on the target evaluation item.

[0007] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data processing method according to any of the embodiments of the present application.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executable by a processor to implement the data processing method according to any of the embodiments of the present application.

[0009] In a fifth aspect, the present application provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the data processing method according to any of the embodiments of the present application.

[0010] The data processing method, device, equipment and storage medium provided by the present application, when receiving the to-be-tested work order data, determine the to-be-processed work order data corresponding to the to-be-tested work order data; process the to-be-processed work order data based on the pre-set configuration model, determine the first evaluation result corresponding to the to-be-processed data; process the to-be-processed work order data based on the pre-trained evaluation model, determine the second evaluation result corresponding to the to-be-processed data; based on the first evaluation result, the second evaluation result and the corresponding evaluation attribute, determine the target evaluation item corresponding to the to-be-processed work order data, to evaluate and test the to-be-processed work order data based on the target evaluation item, solve the problem that the prior art needs to evaluate manually according to product information, which requires high professional knowledge and has low evaluation efficiency, realize the evaluation based on the to-be-processed work order data by using the configuration model and the evaluation model respectively to obtain the first evaluation result and the second evaluation result, improve the evaluation efficiency and reduce the evaluation cost, and at the same time, the target evaluation item is determined based on the first evaluation result and the second evaluation result and the corresponding evaluation attribute, which improves the accuracy of evaluation, and the non-functional test information is represented based on the target evaluation item, thereby achieving the technical effect of improving the test effect. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0012] Figure 1 The flow of the data processing method provided by the embodiment of the present application Figure 1 ;

[0013] Figure 2 The flow of the data processing method provided by the embodiment of the present application Figure 2 ;

[0014] Figure 3 An example diagram of the data processing method provided by the embodiment of the present application;

[0015] Figure 4 An example diagram of the data processing method provided by the embodiment of the present application;

[0016] Figure 5 The structure schematic diagram of the data processing device provided by the embodiment of the present application;

[0017] Figure 6 The structure schematic diagram of the electronic device provided by the embodiment of the present application.

[0018] The specific embodiments of the application have been shown and described in the above drawings and text. These drawings and text are not meant to limit the scope of the inventive concept in any way but are meant to illustrate the inventive concept to one of ordinary skill in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0019] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same reference numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the application as detailed in the appended claims.

[0020] The technical solutions of the application and how the technical solutions solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the application will be described below with reference to the drawings. The acquisition, storage, use, processing, etc. of data in the technical solutions of the application comply with the relevant provisions of national laws and regulations.

[0021] The embodiment of the application provides a data processing method. Figure 1 The flow of the data processing method provided by the embodiment of the application Figure 1 As shown in the figure, the data processing method comprises: Figure 1 As shown in the figure, the data processing method comprises:

[0022] S101, when receiving the to-be-tested work order data, determining to-be-processed work order data corresponding to the to-be-tested work order data.

[0023] The to-be-tested work order data refers to product data that needs to be evaluated, such as production mode, production deadline, version information, code design, deployment machine room, etc.

[0024] It should be noted that, in order to improve the accuracy of the evaluation of the non-functional test information of the product, the data related to the product can be as rich and detailed as possible, and the data related to the product can be as rich and detailed as possible. The data of the to-be-tested work order data. Optionally, the to-be-tested work order data can include at least one of the following data: code change information of at least one product; version upgrade information of at least one program; deployment change information of at least one product; production index information of at least one product; change information of at least one development framework. For example, for product A, after the last test of product A, some upgrades and changes are made to product A, the code change information between this test and the last test of product A (which program code is modified), the deployment change information (deployment machine room, server, etc.), the production index information (production demand change information), the program version upgrade information (such as from 1.0 version to 1.2 version), and the development framework change information (such as from Angular to React) can be obtained., etc., these information can be used as to-be-tested work order data.

[0025] In actual application, the to-be-tested work order data can be called from the preset cache through an interface, and at this time it is considered that the system receives the to-be-tested work order data; or the user uploaded text file is detected. Further, the to-be-tested work order data can be processed, such as cleaning processing, extracting key information, etc. For example, irrelevant information (such as table name, user unselected information, etc.) in the to-be-tested work order data can be cleaned, and the key text content can be extracted to obtain the processed to-be-processed work order data.

[0026] In order to improve the efficiency of data processing and improve the accuracy and speed of subsequent model identification of text content, optionally, the to-be-tested work order data can be processed to obtain the to-be-processed work order data.

[0027] Specifically, the text content in the to-be-tested work order data can be analyzed by using the word segmentation technology, and the text content can be divided into words to obtain the to-be-processed work order data composed of word strings. For example, the user can fill in the change information of the product in the actual design process into the work order template, such as filling in which content is modified by code change, which framework is upgraded, whether the deployment framework and machine room are replaced, whether the version is changed, etc. After filling in these information, the completed work order data can be uploaded to the system, and when the system receives the uploaded work order data, it is considered that the to-be-tested work order data is received, and then the to-be-tested work order data is labeled and analyzed. Processed, such as "spring upgrade" can be divided into "spring", "do", "upgrade". Correspondingly, the processed work order data is obtained as to-be-processed work order data, so that subsequent evaluation of whether product testing is needed and which testing is needed can be based on the to-be-processed work order data.

[0028] S102, processing the to-be-processed work order data based on a pre-configured configuration model to determine a first evaluation result corresponding to the to-be-processed data.

[0029] The configuration model can be a matching model, which is used to hit a strategy by matching and determine the to-be-tested information corresponding to the strategy. The first evaluation result can be used to represent information about which non-functional items of the product need to be tested.

[0030] In actual application, the to-be-processed work order data can be input into the configuration model. The configuration model can match the input information with the information configured in the configuration model by identifying the input information. If the input information matches the configured information, it is considered that the matching is successful, at this time, it can be considered that non-functional testing is needed. The test items corresponding to the information in the matching can be used as the first evaluation result. For example, the configuration model includes text information such as "basic software upgrade" and "TPS (system throughput) is greater than a threshold value". The text information is matched with the text and items of the uploaded data in the work order. If the matching is successful, it is considered that the strategy is hit, and the information about which test items need to be performed is output.

[0031] It should be noted that, in order to improve the accuracy of matching, a regular expression can be set in the configuration model in advance. The regular expression represents a method of hitting a feature. Whether the input data hits a given strategy can be determined by string feature matching. For example, if a specified feature is found in a given string (i.e., input data), the matching is successful.

[0032] Optionally, the to-be-processed work order data is processed based on a pre-configured configuration model to determine a first evaluation result corresponding to the to-be-processed data, including: matching the to-be-processed work order data based on the regular expression corresponding to each to-be-evaluated function set in the configuration model to obtain the first evaluation result.

[0033] The to-be-evaluated function can be used to represent functional information of a software application in a non-functional aspect, such as usability, reliability, stability, scalability, etc. The regular expression refers to a string matching rule, which can be composed of numbers, letters, characters and / or strings, or a combination of at least two of them. For example, the regular expression corresponding to the domain name is: [a-zA-Z0-9][-a-zA-Z0-9]{0,62}. The first evaluation result includes at least one evaluation item matched or a preset result corresponding to an evaluation item that is not matched. The evaluation item can be used to represent non-functional test information, such as reliability or stability. The preset result can be empty, indicating that the evaluation item is not matched.

[0034] Specifically, after the to-be-processed work order data is input into the configuration model, it can be determined by a pre-set regular expression whether the to-be-processed work order data contains a set characteristic value, if yes, it is considered that non-functional testing is needed, at this time, the non-functional testing information associated with the characteristic value can be called as the first evaluation result, so as to know which non-functional items need to be tested based on each item of non-functional testing information in the first evaluation result.

[0035] It should be noted that the regular expression configured in the configuration model can be updated in real time according to actual needs, for example, there can be some new words in the product upgrading process, the regular expression can be updated based on these new word information; or, the non-functional testing items related to the characteristic value can be updated based on the characteristic value in the regular expression, the non-functional testing items can be scalability, availability, reliability, stability and the like. When a specific characteristic value is matched, it can be considered that the characteristic is hit, and the non-functional testing item corresponding to the hit characteristic is taken as the first evaluation result.

[0036] Optionally, the implementation manner of determining the regular expression corresponding to the configuration model can be: determining the regular expression corresponding to the configuration model according to the evaluation items corresponding to the historical work order data, so as to determine the first evaluation result corresponding to the to-be-processed work order data based on the regular expression.

[0037] In the embodiment, the strategy configuration of the configuration model can be performed by the historical work order data and the corresponding evaluation items, and the regular expression is generated. For example, based on the update information (such as new framework, new system, new component and the like) in the historical work order data, the regular expression corresponding thereto can be set, and the non-functional testing items to be tested corresponding to the characteristic value in the regular expression can be updated based on the evaluation items corresponding to the historical work order data.

[0038] In order to effectively manage the configuration information, the configuration information (such as keywords, regular expressions, configuration explanations and the like) of the configuration model can be stored through a unified configuration center, when the configuration center is changed, a notification will be sent to the configuration model for configuration update. So that after the to-be-processed work order data is obtained, the configuration information in the configuration model is matched with the scripts and items of the work order data by the keyword matching or regular matching mode, and the non-functional testing item corresponding to the hit characteristic of the matching success is input as the first evaluation result.

[0039] S103, processing the to-be-processed work order data based on the pre-trained evaluation model, and determining a second evaluation result corresponding to the to-be-processed data.

[0040] The evaluation model can be pre-trained to evaluate which non-functional tests the product needs to perform. The second evaluation result corresponds to the product work order data. The change information of the product in different product work order data is different, and the corresponding second evaluation result can also be different. For example, the product work order data A contains code change information, and the corresponding second evaluation result can be availability and reliability. The product work order data B contains version upgrade information, and the corresponding second evaluation result can be stability and scalability.

[0041] In actual application, the to-be-processed work order data can be taken as the input of the evaluation model. The model input information of which non-functional tests the to-be-processed work order data needs to perform is taken as the second evaluation result.

[0042] It should be noted that the above S102 to S103 can be executed sequentially or in parallel, and the specific execution order is not limited. The above order is only the order of explaining the technical solutions in each step, and is not the execution order of each step.

[0043] S104, based on the first evaluation result, the second evaluation result and the corresponding evaluation attribute, determining a target evaluation item corresponding to the to-be-processed work order data, to evaluate and test the to-be-processed work order data based on the target evaluation item.

[0044] The target evaluation item refers to the test information of which non-functional test items need to be performed to perform the test related to the target evaluation item.

[0045] In this embodiment, after determining the first evaluation result and the second evaluation result, the sum of the evaluation items in the first evaluation result and the evaluation items in the second evaluation result can be taken as the target evaluation item. For example, the first evaluation result contains reliability and stability, and the second evaluation result contains reliability and scalability. Then, reliability, stability and scalability can be taken as the target evaluation item.

[0046] It should be noted that in actual application, the evaluation model obtained by training can not be configured with high reliability of model evaluation. In this case, if the first evaluation result is empty, the second evaluation result as the target evaluation item can have the problem of inaccurate evaluation. In order to improve the accuracy of non-functional test information evaluation, the reliability of evaluation of the evaluation model and the configuration model can be determined in advance. For example, when the first evaluation result is empty, manual evaluation can be performed to obtain the evaluation result determined by manual evaluation and fed back to the user.

[0047] Optionally, the evaluation attribute can include a priority corresponding to the first evaluation result and the second evaluation result, and the priority can be used to represent the credibility of the evaluation result. The priority of the evaluation result corresponds to the model. For example, if the configuration model determines that the credibility of the evaluation result is higher than that of the evaluation model, it can be considered that the priority of the first evaluation result is higher than that of the second evaluation result. Based on the evaluation attribute of the first evaluation result and the priority corresponding to the second evaluation result, it can be determined which evaluation result corresponding to the model is the main evaluation result, and the target evaluation item corresponding to the to-be-processed work order data is obtained. Specifically, based on the first evaluation result, the second evaluation result and the corresponding evaluation attribute, the target evaluation item corresponding to the to-be-processed work order data is determined, including: if the first evaluation result with high priority includes at least one evaluation item, the union of the at least one evaluation item and the evaluation item in the second evaluation result is determined to obtain the target evaluation item; if the first evaluation result with high priority is a preset result, the target evaluation item is determined to be empty.

[0048] In the embodiment, if the first evaluation result contains at least one evaluation item, it indicates that the output result of the configuration model is not empty, and at this time, the evaluation items in the first evaluation result and the evaluation items in the second evaluation result can be processed by union to obtain a union, and each evaluation item contained in the union can be used as the target evaluation item. If the first evaluation result is a preset result, the preset result is an evaluation item that is not matched, i.e., the first evaluation result is empty, and at this time, the evaluation item in the second evaluation result can be discarded, and the target evaluation item is also empty. Further, if the target evaluation item is empty, the processing mode is converted to determine the target evaluation item. The converted processing mode can be various, such as manual evaluation or other matching algorithms (such as similarity, distance, etc.). The to-be-processed work order data is analyzed based on the converted processing mode to obtain the target evaluation item.

[0049] For example, if the priority of the first evaluation result is higher than that of the second evaluation result, if the output of the configuration model is True, it indicates that the first evaluation result is not empty, and the union of the evaluation items in the first evaluation result and the evaluation items in the second evaluation result is used as the target evaluation item; if the output of the configuration model is False, it indicates that the first evaluation result is empty, and it is considered that the model cannot be evaluated, at this time, manual operation evaluation can be performed, and the final manual evaluation result is used as the reference. Further, the manual evaluation result can be added to the training sample of the evaluation model for a new round of learning to improve the accuracy of the model evaluation.

[0050] It should be noted that after the target evaluation item is fed back to the user, the user can perform corresponding non-functional testing on the product corresponding to the to-be-tested work order data based on the target evaluation item; or when the user receives the target evaluation item and believes that the evaluation is incorrect, manual review is initiated, and manual evaluation is performed to take the manual evaluation result as the final evaluation result, so as to avoid unpredictable losses caused by evaluation errors.

[0051] The embodiment determines the to-be-processed work order data corresponding to the to-be-tested work order data when receiving the to-be-tested work order data, processes the to-be-processed work order data based on the pre-set configuration model, determines the first evaluation result corresponding to the to-be-processed data, processes the to-be-processed work order data based on the pre-trained evaluation model, determines the second evaluation result corresponding to the to-be-processed data, and determines the target evaluation item corresponding to the to-be-processed work order data based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute. The to-be-processed work order data is evaluated based on the target evaluation item, which solves the problem in the prior art that manual evaluation based on product information is required, the requirement for professional knowledge is high, and the evaluation efficiency is poor. The configuration model and the evaluation model are used to evaluate the to-be-processed work order data respectively to obtain the first evaluation result and the second evaluation result, improve the evaluation efficiency, reduce the evaluation cost, comprehensively determine the target evaluation item based on the first evaluation result and the second evaluation result and the corresponding evaluation attribute, improve the accuracy of the evaluation, represent the non-functional testing information based on the target evaluation item, and achieve the technical effect of improving the testing effect.

[0052] On the basis of the above-mentioned embodiment, when the to-be-processed work order data is processed based on the evaluation model to determine the second evaluation result corresponding to the to-be-processed data, the final second evaluation result can be obtained through the credibility of each pre-set evaluation item selected by model calculation. Correspondingly, the present application proposes the following embodiment:

[0053] Figure 2 The data processing method provided in the embodiment of the present application Figure 2 As shown in Figure 2 , the data processing method comprises the following steps:

[0054] S201, input the to-be-processed work order data into the evaluation model to obtain a probability matrix corresponding to the to-be-processed data.

[0055] The probability matrix includes at least one probability value, and each probability value can be used to represent the confidence of selection of a corresponding preset evaluation item. For example, the higher the probability value, the greater the confidence of selection of the preset evaluation item, and vice versa, the lower the probability value, the smaller the confidence of selection of the preset evaluation item. The preset evaluation items can be multiple, and each non-functional test information can be preset. The non-functional test information can be reliability or availability, and each non-functional test information can be used as an evaluation item.

[0056] In this embodiment, the to-be-processed work order data can be used as an input of the evaluation model to obtain probability values corresponding to multiple preset evaluation items. Correspondingly, a probability matrix corresponding to the to-be-processed data is obtained. For example, the probability matrix is [reliability: 0.1, availability: 0.2, scalability: 0.3, stability: 0.3, and load capacity: 0.1]. The second evaluation result is determined based on the confidence of selection of each preset evaluation item.

[0057] It should be noted that the evaluation model can be trained in advance. First, training samples required for training the evaluation model are determined, and the evaluation model is trained based on the training samples. Specifically, the determination of the training samples can be as follows: a plurality of historical work order data and label information corresponding to historical evaluation items associated with the historical work order data are obtained; based on each historical work order data, the corresponding historical evaluation item, and the corresponding label information, the training samples for training the evaluation model are determined.

[0058] The label information can be used to represent the uniqueness of the evaluation item. For example, the label information of the evaluation item A can be A1, and the label information of the evaluation item B can be B1. The training samples are used for model training. In order to obtain an evaluation model with high accuracy, as many training samples as possible are obtained, so as to obtain the trained evaluation model.

[0059] In the embodiment, in order to obtain the evaluation model corresponding to the non-functional test information for determining the evaluation work order data, the work order data (i.e., historical work order data) corresponding to the product collected in the historical evaluation process can be obtained, and the historical work order data includes the code change information, the deployment change information, the production index change information, the version upgrade information of the program, the development framework change information of the program, and the like of the product. Correspondingly, in order to correct the model parameters in the model, the evaluation items (i.e., historical evaluation items) obtained by evaluating the historical work order data and the label information corresponding to the historical evaluation items are also needed, and are used as the theoretical output result. The historical work order data and the label information corresponding to the historical evaluation items can be used as a training sample. Correspondingly, based on the historical work order data, the corresponding historical evaluation items, and the corresponding label information, a plurality of training samples for training the evaluation model can be obtained, so as to train the evaluation model based on the training samples.

[0060] Based on the above scheme, after the training sample is determined, the evaluation model can be trained based on the training sample, and the specific implementation manner can be that, for each training sample, the historical work order data of the current training sample is input into the evaluation model to obtain an actual output probability matrix; based on the actual output probability matrix and the label information corresponding to the historical evaluation item in the current training sample, a loss value is determined, so as to correct the model parameters in the evaluation model based on the loss value; and the loss function in the evaluation model is converged as a training target to obtain the evaluation model.

[0061] The probability value at each position in the actual output probability matrix is used to represent the credibility of the selection of the preset evaluation item. The current training sample can be understood as when the actual output probability matrix corresponding to each training sample needs to be determined, the actual output probability matrix of any training sample data can be determined as the actual output probability matrix of the current training sample, that is, one of the training samples can be taken as the current training sample for description. The evaluation model to be trained can be an LSTM (Long Short-Term Memory, long short-term memory network). The model parameters of the evaluation model to be trained can be default values. The evaluation model to be trained can be trained based on each training sample to obtain a trained evaluation model. The loss value refers to the error between the actual output probability matrix and the label information of the theoretical historical evaluation item. The training target refers to the model training to achieve the preset loss function convergence as the target.

[0062] It should be noted that since the model parameters of the to-be-trained evaluation model are initial values or parameters that have not been completed correction, the evaluation result obtained at this time is inaccurate, and accordingly, there is a certain difference between the evaluation items in the evaluation result and the theoretical evaluation items in the training sample. That is, for each training sample, the historical work order data in each training sample can be input into the to-be-trained evaluation model, and the to-be-trained evaluation model can output an actual output probability matrix.

[0063] In the present embodiment, the historical work order data of the current training sample can be input into the to-be-trained evaluation model to obtain an actual output probability matrix corresponding to the current training sample. The actual output probability matrix output at present can be compared with the label information of the labeled historical evaluation item to calculate a similarity error value, i.e., a loss value, and then the model parameters in the model can be adjusted based on the loss value. The training error of the loss function, i.e., the loss parameter, can be used as a condition for detecting whether the current loss function has reached convergence, such as whether the training error is less than a preset error or whether the error change trend is stable, or whether the current iteration number is equal to a preset number. If the convergence condition is detected, such as the training error of the loss function reaching less than the preset error or the error change trend being stable, it indicates that the training of the to-be-trained evaluation model is completed, and at this time the iterative training can be stopped. If it is detected that the current convergence condition has not been reached, the training sample can be further obtained to train the to-be-trained evaluation model until the training error of the loss function is within a preset range. When the training error of the loss function reaches convergence, the to-be-trained evaluation model can be used as the final trained evaluation model.

[0064] S202, determining the second evaluation result based on the probability matrix.

[0065] Specifically, the preset evaluation item corresponding to the probability value greater than the preset threshold in the probability matrix can be used as the output of the evaluation model, and accordingly, the second evaluation result is obtained.

[0066] In the present embodiment, by inputting the to-be-processed work order data into the evaluation model, a probability matrix corresponding to the to-be-processed data is obtained; based on the probability matrix, a second evaluation result is determined to improve the accuracy of the evaluation result determination and ensure the reliability of the evaluation model evaluation.

[0067] On the basis of the above embodiments, in order for those skilled in the art to further clearly understand the technical solutions of the embodiments of the present application, the data processing method will be described in detail through specific examples as follows:

[0068] Figure 3 An example diagram of the data processing method provided by the embodiments of the present application is shown in Figure 3As shown, the user can submit a non-functional test evaluation application and upload relevant information, i.e. upload the to-be-tested work order data. The evaluation robot (including the configuration model and the evaluation model) performs intelligent evaluation and determines whether the evaluation is successful. If the evaluation fails, the work order is further issued to the test personnel for manual evaluation. If the robot evaluation is successful, the evaluation result (i.e. the target evaluation item) is directly returned to the user. The user selects whether the target evaluation item is valid. If the selection is invalid, the work order can be further issued to the test personnel for evaluation to obtain the manual evaluation result. If the selection is valid, the target evaluation item is output and the evaluation is considered complete.

[0069] Figure 4 An example diagram of the data processing method provided by the embodiment of the application is shown in Figure 4 As shown, the technical solution can be realized by the configuration model, the evaluation model and the manual evaluation method. The configuration information in the configuration model can be stored through the unified configuration center. When the configuration information in the configuration center is changed, a notification is sent to the configuration model for configuration update. The configuration information can include keywords, regular expressions, configuration explanations, etc. The evaluation model can be pre-trained. The training data can be derived from historical work order data. The commonly used words in the historical work order data can be filtered and entered into the word segmentation package. The corresponding evaluation result (i.e. the historical evaluation item) existing in the historical work order data is used as a label. The training data and the test data are arranged to train the model. The final evaluation model is updated. The evaluation result of the evaluation model can be used as a bottom line of the configuration model, i.e. the priority of the evaluation result of the configuration model is higher than that of the evaluation result of the evaluation model. As an example, the to-be-tested work order data can be uploaded to the system. After the to-be-tested work order data is processed by word segmentation, the to-be-processed work order data is obtained. Further, the to-be-processed work order data is input into the configuration model and the evaluation model. The configuration model and the evaluation model perform evaluation. If the model evaluation is successful, the target evaluation item is output. If the model evaluation fails, manual evaluation can be performed. The target evaluation item determined by the manual evaluation is output. The final manual evaluation result is used as a reference. When the user receives the evaluation result and considers that the evaluation is incorrect, manual review can be initiated again. The manual evaluation result is used as the final evaluation result to avoid unpredictable losses caused by evaluation errors. The target evaluation item output can be used for non-functional testing of the product. It should be noted that the target evaluation item determined by the manual evaluation can be used as training data. The historical work order data corresponding to the target evaluation item determined by the manual evaluation can be obtained. The historical work order data is processed by word segmentation. The label information of the corresponding evaluation item is labeled. The training sample data is enhanced. A new round of model training is performed to improve the evaluation accuracy of the evaluation model.

[0070] The technical solution accelerates the process speed from ticket data submission to report output, improves project iteration speed, improves evaluation speed, reduces test labor cost and communication cost, and saves data in a structured persistent manner, avoiding loss and errors caused by personnel changes.

[0071] The embodiment determines the to-be-processed ticket data corresponding to the to-be-tested ticket data when receiving the to-be-tested ticket data, processes the to-be-processed ticket data based on a preset configuration model to determine a first evaluation result corresponding to the to-be-processed data, processes the to-be-processed ticket data based on a pre-trained evaluation model to determine a second evaluation result corresponding to the to-be-processed data, and determines a target evaluation item corresponding to the to-be-processed ticket data based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute, to evaluate the to-be-processed ticket data based on the target evaluation item. The problems of requiring manual evaluation according to product information, high requirement for professional knowledge, and poor evaluation efficiency in the prior art are solved. The configuration model and the evaluation model are used to evaluate the to-be-processed ticket data to obtain the first evaluation result and the second evaluation result, improve the evaluation efficiency, reduce the evaluation cost, and comprehensively determine the target evaluation item based on the first evaluation result and the second evaluation result and the corresponding evaluation attribute to improve the evaluation accuracy. The non-functional test information is represented based on the target evaluation item, and the technical effect of improving the test effect is achieved.

[0072] Figure 5 A structural schematic diagram of a data processing apparatus provided by the embodiment is shown in FIG. 5. Figure 5 As shown in FIG. 5, the data processing apparatus includes a to-be-processed ticket data determination module 510, a first evaluation result determination module 520, a second evaluation result determination module 530, and a target evaluation item determination module 540.

[0073] The to-be-processed ticket data determination module 510 is configured to determine to-be-processed ticket data corresponding to to-be-tested ticket data when receiving the to-be-tested ticket data. The first evaluation result determination module 520 is configured to process the to-be-processed ticket data based on a preset configuration model to determine a first evaluation result corresponding to the to-be-processed data. The second evaluation result determination module 530 is configured to process the to-be-processed ticket data based on a pre-trained evaluation model to determine a second evaluation result corresponding to the to-be-processed data. The target evaluation item determination module 540 is configured to determine a target evaluation item corresponding to the to-be-processed ticket data based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute, to evaluate the to-be-processed ticket data based on the target evaluation item.

[0074] In the embodiment, when receiving the to-be-tested work order data, the to-be-processed work order data corresponding to the to-be-tested work order data is determined; the to-be-processed work order data is processed based on a pre-set configuration model to determine a first evaluation result corresponding to the to-be-processed data; the to-be-processed work order data is processed based on a pre-trained evaluation model to determine a second evaluation result corresponding to the to-be-processed data; and based on the first evaluation result, the second evaluation result and corresponding evaluation attributes, a target evaluation item corresponding to the to-be-processed work order data is determined to evaluate the to-be-processed work order data based on the target evaluation item, thereby solving the problem in the prior art that manual evaluation based on product information is required, the problem of high requirement for professional knowledge and poor evaluation efficiency, achieving evaluation based on the to-be-processed work order data by using the configuration model and the evaluation model respectively to obtain the first evaluation result and the second evaluation result, improving evaluation efficiency and reducing evaluation cost, simultaneously determining the target evaluation item based on the first evaluation result and the second evaluation result and corresponding evaluation attributes to improve evaluation accuracy, representing non-functional test information based on the target evaluation item and achieving the technical effect of improving test effect.

[0075] In some embodiments, optionally, the to-be-tested work order data includes at least one of the following data: code change information of at least one product; deployment change information of at least one product; production index change information of at least one product; version upgrade information of at least one program; and development framework change information of at least one program.

[0076] In some embodiments, optionally, the to-be-processed work order data determination module 510 is further configured to perform word segmentation processing on the to-be-tested work order data to obtain the to-be-processed work order data.

[0077] In some embodiments, optionally, the first evaluation result determination module 520 is further configured to perform matching processing on the to-be-processed work order data based on regular expressions corresponding to each to-be-evaluated function set in the configuration model to obtain the first evaluation result.

[0078] In the first evaluation result, there is at least one matched evaluation item or a pre-set result corresponding to an unmatched evaluation item.

[0079] In some embodiments, optionally, the second evaluation result determination module 530 includes a probability matrix determination unit and a second evaluation result determination unit.

[0080] The probability matrix determination unit is configured to input the to-be-processed work order data into the evaluation model to obtain a probability matrix corresponding to the to-be-processed data.

[0081] The second evaluation result determination unit is configured to determine the second evaluation result based on the probability matrix.

[0082] In some embodiments, optionally, the probability matrix comprises at least one probability value, the at least one probability value being used to represent a confidence level of selection of a preset evaluation item.

[0083] In some embodiments, optionally, the device further comprises a regular expression determination module.

[0084] The regular expression determination module is configured to determine a regular expression corresponding to the configuration model according to an evaluation item corresponding to historical work order data, so as to determine a first evaluation result corresponding to the to-be-processed work order data based on the regular expression.

[0085] In some embodiments, optionally, the device further comprises a training sample determination module. The training sample determination module comprises a label information determination unit and a training sample determination unit.

[0086] The label information determination unit is configured to obtain a plurality of historical work order data and label information corresponding to historical evaluation items associated with the historical work order data.

[0087] The training sample determination unit is configured to determine a training sample for training the evaluation model based on each historical work order data, a corresponding historical evaluation item, and corresponding label information.

[0088] In some embodiments, optionally, the device further comprises a model training module. The model training module comprises an output probability matrix determination unit, a loss value determination unit, and an evaluation model determination unit.

[0089] The output probability matrix determination unit is configured to input historical work order data of a current training sample into the evaluation model to obtain an actual output probability matrix for each training sample. A probability value at each position in the actual output probability matrix is used to represent a confidence level of selection of a preset evaluation item.

[0090] The loss value determination unit is configured to determine a loss value based on the actual output probability matrix and label information corresponding to a historical evaluation item in the current training sample, and to correct a model parameter in the evaluation model based on the loss value.

[0091] The evaluation model determination unit is configured to take convergence of a loss function in the evaluation model as a training target to obtain the evaluation model.

[0092] In some embodiments, optionally, the evaluation attribute comprises a priority corresponding to the first evaluation result and the second evaluation result, and the target evaluation item determination module 540 comprises a target evaluation item determination unit and a preset result judgment unit.

[0093] The target evaluation item determination unit is configured to determine a union of at least one evaluation item in the first evaluation result with high priority and evaluation items in the second evaluation result to obtain the target evaluation item.

[0094] The preset result determination unit is configured to determine that the target evaluation item is empty if the first evaluation result with high priority is a preset result.

[0095] In some embodiments, the device further includes an artificial processing module configured to convert a processing mode to determine the target evaluation item if the target evaluation item is empty.

[0096] The data processing device provided by the embodiments of the present application can be used to execute the technical solutions of the data processing method in the above embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0097] It should be noted that the division of each module of the above device is only a logical function division, and all or part of the modules can be integrated into one physical entity, or can be physically separated. And these modules can all be implemented in the form of software called by a processing element; all can be implemented in the form of hardware; some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the to-be-processed work order data determination module 510 can be a separately established processing element, or can be integrated in a chip of the above device, in addition, it can also be stored in the memory of the above device in the form of program code, and the functions of the above to-be-processed work order data determination module 510 are called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, all or part of the modules can be integrated together, or can be independently implemented. The processing element here can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each module can be completed by integrated logic circuit of hardware or instruction in the form of software in the processing element.

[0098] Figure 6 The structure schematic diagram of the electronic device provided by the embodiments of the present application is shown in the figure. Figure 6 As shown in the figure, the electronic device can include a transceiver 121, a processor 122 and a memory 123.

[0099] The processor 122 executes computer-executed instructions stored in the memory to cause the processor 122 to perform the solutions in the above-described embodiments. The processor 122 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; and can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0100] The memory 123 is connected with the processor 122 through the system bus and completes mutual communication, and the memory 123 is used for storing computer program instructions.

[0101] The transceiver 121 can be used to send a target processing result corresponding to a service request.

[0102] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The transceiver is used to realize the communication between the database access device and other computers (for example, a client, a read-write library and a read-only library). The memory can include a random access memory (RAM), and can also include a non-volatile memory.

[0103] The electronic device provided by the embodiments of the present application can be the terminal device of the above-described embodiments.

[0104] The embodiments of the present application further provide a chip for executing the operation instruction, and the chip is used to execute the technical solutions of the data processing method in the above-described embodiments.

[0105] The embodiments of the present application further provide a computer readable storage medium, and the computer readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the technical solutions of the data processing method in the above-described embodiments.

[0106] The embodiments of the present application further provide a computer program product, and the computer program product includes a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and when the at least one processor executes the computer program, the technical solutions of the data processing method in the above-described embodiments can be implemented.

[0107] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0108] It is to be understood that the application is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is to be indicated by the appended claims, rather than the description and examples.

Claims

1. A data processing method, characterized by, The method comprises the following steps: Upon receiving the to-be-tested work order data, determine the to-be-processed work order data corresponding to the to-be-tested work order data; Match the to-be-processed work order data based on the regular expression corresponding to each to-be-evaluated function set in the pre-set configuration model, to obtain the first evaluation result corresponding to the to-be-processed work order data; wherein the first evaluation result includes at least one matched evaluation item or a pre-set result corresponding to an unmatched evaluation item; Input the to-be-processed work order data into the pre-trained evaluation model to obtain a probability matrix corresponding to the to-be-processed work order data; based on the probability matrix, determine the second evaluation result corresponding to the to-be-processed work order data; the probability matrix includes at least one probability value, which represents the credibility of the pre-set evaluation item being selected; Based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute, determine the target evaluation item corresponding to the to-be-processed work order data, and evaluate the to-be-processed work order data based on the target evaluation item; The determination of the second evaluation result based on the probability matrix comprises: taking the pre-set evaluation item corresponding to the probability value greater than the pre-set threshold in the probability matrix as the second evaluation result; The evaluation attribute includes the priority of the first evaluation result and the second evaluation result; the determination of the target evaluation item based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute comprises: If the first evaluation result with high priority includes at least one evaluation item, determine the union of the at least one evaluation item and the evaluation item in the second evaluation result to obtain the target evaluation item; If the first evaluation result with high priority is a pre-set result, the target evaluation item is empty; The matching processing of the to-be-processed work order data based on the regular expression corresponding to each to-be-evaluated function set in the pre-set configuration model to obtain the first evaluation result corresponding to the to-be-processed work order data comprises: searching the to-be-processed work order data for the set feature value through the pre-set regular expression full-text search; if the feature value is found, retrieve the non-functional test information associated with the feature value as the first evaluation result; The method further comprises: Obtain a plurality of historical work order data and label information corresponding to the historical evaluation items associated with the historical work order data; based on each historical work order data, the corresponding historical evaluation item, and the corresponding label information, determine the training sample for training the evaluation model; For each training sample, input the historical work order data of the current training sample into the evaluation model to obtain an actual output probability matrix; wherein the probability value at each position in the actual output probability matrix represents the credibility of the pre-set evaluation item being selected; Based on the actual output probability matrix and the label information corresponding to the historical evaluation item in the current training sample, determine a loss value to correct the model parameters in the evaluation model based on the loss value. Converge the loss function in the evaluation model as a training target to obtain the evaluation model.

2. The method of claim 1, wherein, The to-be-tested work order data includes at least one of the following data: Code change information of at least one product; Deployment change information of at least one product; Production index change information of at least one product; Version upgrade information of at least one program; Development framework change information of at least one program.

3. The method of claim 1, wherein, The to-be-processed work order data corresponding to the to-be-tested work order data is determined, including: The to-be-tested work order data is subjected to word segmentation processing to obtain the to-be-processed work order data.

4. The method of claim 1, wherein, Further comprising: According to the evaluation items corresponding to the historical work order data, the regular expression corresponding to the configuration model is determined to determine the first evaluation result corresponding to the to-be-processed work order data based on the regular expression.

5. The method of claim 1, wherein, Further comprising: If the target evaluation item is empty, the processing mode is converted to determine the target evaluation item.

6. A data processing apparatus, characterized by, Comprising: A to-be-processed work order data determination module is configured to determine to-be-processed work order data corresponding to to-be-tested work order data when the to-be-tested work order data is received; A first evaluation result determination module is configured to perform matching processing on the to-be-processed work order data based on the regular expression corresponding to each to-be-evaluated function set in the pre-set configuration model to obtain a first evaluation result corresponding to the to-be-processed work order data; wherein the first evaluation result includes at least one matched evaluation item or a pre-set result corresponding to an unmatched evaluation item; A second evaluation result determination module is configured to input the to-be-processed work order data into a pre-trained evaluation model to obtain a probability matrix corresponding to the to-be-processed work order data; based on the probability matrix, a second evaluation result corresponding to the to-be-processed work order data is determined; the probability matrix includes at least one probability value, and the at least one probability value is used to represent the credibility of the pre-set evaluation item being selected; the second evaluation result is determined based on the probability matrix, including: the pre-set evaluation item corresponding to the probability value greater than the pre-set threshold in the probability matrix is taken as the second evaluation result; A target evaluation item determination module is configured to determine a target evaluation item corresponding to the to-be-processed work order data based on the first evaluation result, the second evaluation result, and the corresponding evaluation attribute, and to evaluate and test the to-be-processed work order data based on the target evaluation item; The evaluation attribute includes the priority of the first evaluation result and the second evaluation result, and the target evaluation item determination module includes a target evaluation item determination unit and a pre-set result judgment unit; The target evaluation item determination unit is configured to determine the union of at least one evaluation item in the first evaluation result with high priority and the evaluation item in the second evaluation result to obtain the target evaluation item if the first evaluation result with high priority includes at least one evaluation item; The pre-set result judgment unit is configured to determine that the target evaluation item is empty if the first evaluation result with high priority is a pre-set result. The first evaluation result determination module is further configured to search the to-be-processed work order data by using a preset regular expression to determine whether the to-be-processed work order data contains a set characteristic value, and if the to-be-processed work order data contains the set characteristic value, call non-functional test information associated with the characteristic value as the first evaluation result. The device further includes a training sample determination module, which includes a label information determination unit and a training sample determination unit. The label information determination unit is configured to obtain a plurality of historical work order data and label information corresponding to historical evaluation items associated with the historical work order data. The training sample determination unit is configured to determine training samples for training the evaluation model based on each historical work order data, corresponding historical evaluation items, and corresponding label information. The model training module includes an output probability matrix determination unit, a loss value determination unit, and an evaluation model determination unit. The output probability matrix determination unit is configured to, for each training sample, input historical work order data of a current training sample into the evaluation model to obtain an actual output probability matrix, wherein a probability value at each position in the actual output probability matrix is used to represent a credibility of a preset evaluation item being selected. The loss value determination unit is configured to determine a loss value based on the actual output probability matrix and label information corresponding to historical evaluation items in the current training sample, and correct model parameters in the evaluation model based on the loss value. The evaluation model determination unit is configured to take convergence of a loss function in the evaluation model as a training target to obtain the evaluation model.

7. An electronic device, comprising: It includes: a processor, and a memory connected to the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the data processing method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the data processing method of any one of claims 1-5.

9. A computer program product, characterised in that, It includes a computer program, which is executed by the processor to implement the data processing method of any one of claims 1-5.

Images