Marine data platform automatic test error analysis method based on large model

Abstract

The invention relates to an ocean data platform automatic test error analysis method based on a large model, which comprises the following steps: defining a meta-learning task, collecting and preprocessing historical BUG data and error analysis data, and generating training data; a BERT model is used for performing text feature extraction, structured data processing is combined, a comprehensive feature vector is constructed, an MAML framework is combined with a large language model, an error analysis model is constructed, and model parameters are optimized through multi-task training, so that the method can quickly adapt to different error scenes. And deploying the optimized model on a test development platform, analyzing an automatic test result in real time, and generating a defect analysis report. In addition, the method also supports dynamic expansion, and when a new error scene occurs, a new meta-learning task can be defined and the model can be finely adjusted to ensure that the model continuously adapts to business requirements; a defect analysis result is integrated into a CI / CD process, and closed-loop optimization of automatic testing and error analysis is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of software automated testing, and mainly relates to a method for automated testing error analysis of an ocean data platform based on a large model. Background Art

[0002] With the rapid development of information technology, the ocean data platform, as a complex system integrating various ocean data and functions, is increasingly widely used in the fields of ocean resource management, environmental monitoring, scientific research and education. Such platforms usually involve a large amount of data processing, complex business logic, and high degree of interconnection. Therefore, the development and maintenance process has extremely high requirements for software quality. Automated testing, as an important means to ensure software quality, occupies an important position in the development process of the ocean data platform.

[0003] Automated testing can quickly discover and locate defects in software by simulating user operations, executing predefined test cases, and generating test reports. However, with the increasing complexity and scale of the ocean data platform, the coverage and data volume of automated testing are also expanding rapidly. A large amount of test result data generated during the testing process needs to be efficiently analyzed and processed in order to timely discover and fix problems, and ensure the stability and reliability of the system.

[0004] In the existing automated testing process, error analysis usually relies on the experience of developers and manual analysis means. Developers need to manually check test reports, log files, and user feedback, extract valuable information from them, and then locate the root cause of the problem. However, manual analysis requires a large amount of time and effort. Especially when facing large-scale test data, developers may miss important problems due to information overload, and manual analysis is easily affected by subjective factors, lacking the systematic analysis ability of global data, and it is difficult to comprehensively identify potential patterns and trends of problems. Existing error analysis methods are usually designed for specific scenarios and are difficult to quickly adapt to new error types or business scenarios, especially in a highly professional and dynamic field such as the ocean data platform. Therefore, there is an urgent need for an automated testing error analysis method for the ocean data platform that reduces the need for manual intervention and can quickly learn and adapt to new error scenarios. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present application provides a method for automated testing error analysis of an ocean data platform based on a large model.

[0006] The technical solution of the present application is as follows:

[0007] A method for automated testing error analysis of an ocean data platform based on a large model, the method includes:

[0008] Define a meta - learning task, obtain defect analysis data related to the ocean data platform based on the meta - learning task, including historical BUG data and corresponding error analysis data, and pre - process the defect analysis data;

[0009] Extract features from the pre - processed defect analysis data to obtain feature - characterized defect analysis data, and divide the feature - characterized defect analysis data according to the meta - learning task. Each meta - learning task corresponds to different subsets, including a training set, a validation set, and a test set;

[0010] Construct an error analysis model based on the combination of a meta - learning algorithm and the large - language model LLM, optimize the error analysis model using the divided training set, validation set, and test set to generate optimized initial parameters, and load the optimized initial parameters into the error analysis model to obtain an optimized error analysis model;

[0011] On the test development platform, configure and run automated test scripts related to the ocean data platform, collect automated test result data in real - time during the running process, and perform defect analysis based on the synchronized and optimized error analysis model to obtain defect analysis results.

[0012] As a preferred embodiment of the present invention, defining the meta - learning task specifically means defining that each specific error scenario corresponds to a meta - learning task based on the defect analysis requirements of the ocean data platform.

[0013] As a preferred embodiment of the present invention, the historical BUG data includes the BUG - affiliated module, BUG reproduction steps, actual results of BUG occurrence, expected results of BUG occurrence, BUG type, front - end and back - end personnel corresponding to the BUG - affiliated module, and BUG repair time; the error analysis data includes automated test reports, error logs, user feedback data, system monitoring data, and development documents.

[0014] As a preferred embodiment of the present invention, the pre - processing of the defect analysis data includes data cleaning, format conversion, and data augmentation, where:

[0015] The data cleaning is specifically to remove duplicate data and incorrect formats; the format conversion is specifically to convert text data into structured data; the data augmentation is specifically to perform text transformation by synonym replacement, word order transformation, and adding or deleting words, generate adversarial samples based on the text transformation content, and perform grammar transformation that preserves semantics on the adversarial samples to generate new data samples.

[0016] As a preferred embodiment of the present invention, the construction of the error analysis model by combining the meta - learning algorithm and the large - language model LLM is specifically to select Model - Agnostic Meta - Learning (MAML) as the meta - learning algorithm framework, use a pre - trained large - language model as the classifier, and embed it into the meta - learning model to obtain the error analysis model. The error analysis model includes an input layer, a feature extraction layer, a feature fusion layer, a meta - learning layer, and an output layer, where:

[0017] The input layer is used to receive the defect analysis data after feature extraction, including text data and structured data; the feature extraction layer is used to perform text feature extraction and structured data processing. For text feature extraction, the BERT model is used to tokenize the text data, embed word vectors, and perform context encoding to generate text feature vectors. The structured data processing is specifically to perform embedding processing on the structured data to generate structured feature vectors; the feature fusion layer is used to splice or weighted - fuse the text feature vectors and structured feature vectors to generate comprehensive feature vectors; the meta - learning layer includes an adaptation layer and a task prediction layer. The adaptation layer uses the MAML algorithm to quickly adapt the comprehensive feature vectors, and the task prediction layer classifies the error types and predicts the severity level according to the adapted comprehensive feature vectors; the output layer is used to output the final defect analysis results, including error types, severity levels, root causes, and repair suggestions.

[0018] As a preferred embodiment of the present invention, the method further includes training the error analysis model through multi - task training, jointly training on multiple meta - learning tasks, and each task corresponds to a specific error scenario. Among them, the meta - learning steps include initialization, inner loop, outer loop, and hyperparameter tuning. Specifically:

[0019] The initialization is to initialize the parameters of the large - language model and the meta - learning model; the inner loop is to perform one or more gradient updates on the training set of each task; the outer loop is to evaluate the model performance on the validation sets of all tasks and adjust the initial parameters of the model according to the evaluation results; the hyperparameter tuning is to adjust the hyperparameters through the performance evaluation on the validation set to optimize the training process of the model.

[0020] As a preferred embodiment of the present invention, the method further includes:

[0021] When a new error scenario appears, define a new meta - learning task, collect defect analysis data related to the new meta - learning task, and perform pre - processing to obtain a new task dataset; use the new task dataset to fine - tune the optimized error analysis model, and use the fine - tuned error analysis model as the error analysis model deployed on the test and development platform.

[0022] As a preferred embodiment of the present invention, the method further includes automatically assigning priorities based on the severity and scope of influence of the defects in the defect analysis results, feeding back the defect detection results and corresponding priorities to the repair team, and feeding back the data of the automated test results after repair to the training set for re-optimization of the error analysis model; integrating the re-optimized error analysis model into the CI / CD process, continuously running the automated test script during each integration test, and using the error analysis model to analyze the test results in real time.

[0023] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements a method for error analysis of automated testing of an ocean data platform based on a large model as described in any one of the embodiments.

[0024] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a method for error analysis of automated testing of an ocean data platform based on a large model as described in any one of the embodiments of the present invention.

[0025] Compared with the prior art, the beneficial effects of the present invention are:

[0026] 1) The present invention provides a method for error analysis of automated testing of an ocean data platform based on a large model. By defining different meta-learning tasks, the error analysis model can quickly adapt to and learn different error scenarios, and can quickly adjust and give accurate analysis results when facing new or unknown error types.

[0027] 2) The present invention provides a method for error analysis of automated testing of an ocean data platform based on a large model. By combining the meta-learning algorithm and the large language model LLM, it effectively analyzes the errors in the ocean data platform; and the automated error analysis can quickly locate the problems, reduce the time for manual troubleshooting, and improve the accuracy of error identification at the same time.

[0028] 3) The present invention provides a method for error analysis of automated testing of an ocean data platform based on a large model. Integrating the optimized error analysis model into the CI / CD process, continuously monitoring the health status of the system, and analyzing the test results in real time during each integration test, timely discovering and fixing potential problems, thereby improving the quality and efficiency of software development; at the same time, by automatically assigning priorities to defects, it ensures that the repair team gives priority to dealing with defects with greater severity and scope of influence, further improving the stability of the system and the user experience. Description of the Drawings

[0029] Figure 1 is the flowchart of the method of the embodiment of the present invention. Detailed Embodiments

[0030] The following describes the specific embodiments of the present invention to facilitate those skilled in the art of this technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of this technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0031] The present invention provides the following technical solution: An automated test error analysis method for an ocean data platform based on a large model.

[0032] Example 1:

[0033] This embodiment provides an automated test error analysis method for an ocean data platform based on a large model, where:

[0034] S1. Define a meta-learning task, obtain defect analysis data related to the ocean data platform based on the meta-learning task, including historical BUG data and corresponding error analysis data, and preprocess the defect analysis data;

[0035] S11. Defining the meta-learning task specifically means defining that each specific error scenario corresponds to a meta-learning task based on the defect analysis requirements of the ocean data platform;

[0036] Furthermore, the definition of the meta-learning task is to meet the defect analysis requirements of the ocean data platform. Each specific error scenario is defined as a meta-learning task. This means that in different modules or functions of the ocean data platform, each specific error type, the context in which the error occurs, the steps to reproduce the error, etc. are regarded as an independent meta-learning task. In this way, complex defect analysis problems can be decomposed into multiple small and manageable tasks, thereby improving the analysis efficiency and accuracy.

[0037] Suppose there is a module in the ocean data platform that is a ship positioning system. If there are frequent problems with positioning deviation in this system, then this error scenario of positioning deviation can be defined as a meta-learning task. The goal of this task may be to identify the specific reasons for the positioning deviation (such as hardware failure, software bug, network latency, etc.) and propose corresponding solutions.

[0038] S12. The historical BUG data includes the module to which the BUG belongs, the steps to reproduce the BUG, the actual result when the BUG appears, the expected result when the BUG appears, the BUG type, the front-end and back-end personnel corresponding to the module to which the BUG belongs, and the BUG repair time; the error analysis data includes the automated test report, error log, user feedback data, system monitoring data, and development documents;

[0039] Among them, the module to which the BUG belongs specifically refers to the module or functional area where the BUG occurs; the steps to reproduce the BUG specifically describe the detailed steps on how to reproduce the BUG in the system; the actual result when the BUG appears specifically describes the behavior actually shown by the system when the BUG occurs; the expected result when the BUG appears specifically describes the correct behavior that the system should show when the BUG occurs; the type of the BUG specifically classifies the type of the BUG, such as functional BUG, performance BUG, security BUG, etc.; the front-end and back-end personnel corresponding to the module to which the BUG belongs specifically record the front-end and back-end developers responsible for the module to which the BUG belongs; the time to fix the BUG specifically records the time from the report of the BUG to the completion of the fix.

[0040] The automated test report specifically refers to the report generated by the automated test tool, including the execution results of test cases, error messages, etc.; the error log specifically refers to the error information recorded during the operation of the system, usually including error codes, error messages, timestamps, etc.; the user feedback data specifically refers to the errors or problems reported by users during the use of the system, usually including screenshots, description information, etc.; the system monitoring data specifically refers to the performance data during the operation of the system, such as CPU usage, memory usage, network latency, etc.; the development documents specifically refer to the design documents, API documents, code comments, etc. of the system, which are helpful for understanding the logic and structure of the system.

[0041] S13. The preprocessing of the defect analysis data includes data cleaning, format conversion, and data augmentation, where:

[0042] The data cleaning specifically refers to removing duplicate data and incorrect formats. The removal of duplicate data specifically refers to deleting duplicate records in the historical BUG data and error analysis data to ensure that each error is only analyzed once; the removal of incorrect formats specifically refers to correcting or deleting incorrect formats in the data, such as incomplete fields, incorrect date formats, etc.

[0043] The format conversion specifically refers to converting text data into structured data, specifically converting unstructured text data (such as error logs, user feedback) into a structured format (such as tables, database records) for subsequent analysis and processing.

[0044] The data augmentation specifically involves text transformation through synonym replacement, word order transformation, and addition or deletion of words. Specifically, new text variants are generated by performing operations such as synonym replacement, word order adjustment, or addition / deletion of words on the original text; adversarial samples are generated based on the text transformation content. Specifically, using the generated text variants to generate adversarial samples, these samples are semantically consistent with the original data but slightly different grammatically, and are used to enhance the robustness of the model; further perform grammatical transformation on the adversarial samples to ensure that they are semantically consistent but slightly different in structure, thereby generating new data samples;

[0045] S2. Extract features from the preprocessed defect analysis data to obtain featureized defect analysis data. Divide the featureized defect analysis data according to the meta-learning tasks, and each meta-learning task corresponds to different subsets, including a training set, a validation set, and a test set;

[0046] S3. Construct an error analysis model based on the combination of a meta-learning algorithm and a large language model LLM. Use the divided training set, validation set, and test set to optimize the error analysis model, generate optimized initial parameters, and load the optimized initial parameters into the error analysis model to obtain an optimized error analysis model;

[0047] S31. Select model-agnostic meta-learning MAML as the meta-learning algorithm framework, use a pre-trained large language model as a classifier, and embed it into the meta-learning model to obtain an error analysis model. The error analysis model includes an input layer, a feature extraction layer, a feature fusion layer, a meta-learning layer, and an output layer, where:

[0048] The input layer is used to receive the featureized defect analysis data, including text data and structured data; the feature extraction layer is used to perform text feature extraction and structured data processing. For text feature extraction, the BERT model is used to tokenize the text data, embed word vectors, and perform context encoding to generate text feature vectors. The structured data processing specifically embeds the structured data to generate structured feature vectors; the feature fusion layer is used to splice or weight-fuse the text feature vectors and structured feature vectors to generate a comprehensive feature vector; the meta-learning layer includes an adaptation layer and a task prediction layer. The adaptation layer uses the MAML algorithm to quickly adapt the comprehensive feature vector, and the task prediction layer classifies the error type and predicts the severity based on the adapted comprehensive feature vector; the output layer is used to output the final defect analysis results, including error type, severity, root cause, and repair suggestions;

[0049] S32. Train the error analysis model through multi-task training, and conduct joint training on multiple meta-learning tasks. Each task corresponds to a specific error scenario. Among them, the meta-learning steps include initialization, inner loop, outer loop, and hyperparameter tuning. Specifically:

[0050] The initialization is to initialize the parameters of the large language model and the meta-learning model; the inner loop is to perform one or more gradient updates on the training set of each task; the outer loop is to evaluate the model performance on the validation sets of all tasks and adjust the initial parameters of the model according to the evaluation results; the hyperparameter tuning is to adjust the hyperparameters through performance evaluation on the validation set to optimize the training process of the model

[0051] S4. On the test development platform, configure and run the automated test scripts related to the ocean data platform. During the running process, collect the automated test result data in real time, and conduct defect analysis based on the optimized error analysis model deployed synchronously to obtain the defect analysis results;

[0052] S5. When a new error scenario appears, define a new meta-learning task, collect the defect analysis data related to the new meta-learning task, and perform preprocessing to obtain a new task dataset; use the new task dataset to fine-tune the optimized error analysis model, and use the fine-tuned error analysis model as the error analysis model deployed on the test development platform;

[0053] S6. Automatically assign priorities based on the severity and scope of influence of the defects in the defect analysis results, feedback the defect detection results and corresponding priorities to the repair team, and feedback the repaired automated test result data to the training set for re-optimization of the error analysis model; integrate the re-optimized error analysis model into the CI / CD process. In each integration test, continuously run the automated test scripts and use the error analysis model to analyze the test results in real time.

[0054] Embodiment 2:

[0055] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements a method for automated test error analysis of an ocean data platform based on a large model as described in any one of the embodiments.

[0056] Embodiment 3:

[0057] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a method for automated test error analysis of an ocean data platform based on a large model as described in any one of the embodiments of the present invention.

[0058] The above are only embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall similarly be included within the patent protection scope of the present invention.

Claims

1. A method for automated testing error analysis of an ocean data platform based on a large model, characterized in that: The method comprises: Define a meta-learning task, obtain defect analysis data related to the ocean data platform based on the meta-learning task, including historical BUG data and corresponding error analysis data, and pre-process the defect analysis data; Perform feature extraction on the preprocessed defect analysis data to obtain characterized defect analysis data, and divide the characterized defect analysis data according to meta-learning tasks. Each meta-learning task corresponds to a different subset, including a training set, a validation set, and a test set. An error analysis model is constructed based on the combination of a meta-learning algorithm and a large language model (LLM). The error analysis model is optimized using the divided training set, validation set, and test set to generate optimized initial parameters. The optimized initial parameters are loaded into the error analysis model to obtain an optimized error analysis model. On the test development platform, configure and run the automated test scripts related to the ocean data platform, collect automated test result data in real time during the operation, perform defect analysis based on the optimized error analysis model deployed synchronously, and obtain defect analysis results.

2. The method for analyzing errors in automated testing of a marine data platform based on a large model according to claim 1, characterized in that: The meta-learning task is defined as the defect analysis requirement based on the ocean data platform, and each specific error scenario is defined to correspond to a meta-learning task.

3. The method for analyzing errors in automated testing of a marine data platform based on a large model according to claim 1, characterized in that: The historical BUG data includes the module to which the BUG belongs, the steps to reproduce the BUG, ​​the actual result of the BUG, ​​the expected result of the BUG, ​​the BUG type, the front-end and back-end personnel corresponding to the module to which the BUG belongs, and the BUG repair time; the error analysis data includes automated test reports, error logs, user feedback data, system monitoring data and development documents.

4. The method for error analysis of automated testing of an ocean data platform based on a large model according to claim 1, characterized in that: The preprocessing of the defect analysis data includes data cleaning, format conversion and data enhancement, wherein: The data cleaning specifically includes removing duplicate data and incorrect formats; the format conversion specifically includes converting text data into structured data; the data enhancement specifically includes transforming the text through synonym replacement, word order change, and adding or deleting words, generating adversarial samples based on the text transformation content, and performing semantically preserved grammatical transformation on the adversarial samples to generate new data samples.

5. The method for automatic testing error analysis of an ocean data platform based on a large model according to claim 1 is characterized in that: The error analysis model constructed based on the combination of the meta-learning algorithm and the large language model LLM is specifically selected as the meta-learning algorithm framework, the pre-trained large language model is used as a classifier, and is embedded into the meta-learning model to obtain the error analysis model, wherein the error analysis model includes an input layer, a feature extraction layer, a feature fusion layer, a meta-learning layer, and an output layer, wherein: The input layer is used to receive the characterized defect analysis data, including text data and structured data; the feature extraction layer is used to perform text feature extraction and structured data processing, the text feature extraction uses the BERT model to perform word segmentation, word vector embedding and context encoding on the text data to generate a text feature vector, and the structured data processing specifically embeds the structured data to generate a structured feature vector; the feature fusion layer is used to concatenate or weightedly fuse the text feature vector and the structured feature vector to generate a comprehensive feature vector; the meta-learning layer includes an adaptation layer and a task prediction layer, the adaptation layer uses the MAML algorithm to quickly adapt the comprehensive feature vector, and the task prediction layer performs error type classification and severity prediction based on the adapted comprehensive feature vector; the output layer is used to output the final defect analysis results, including error type, severity, root cause and repair suggestion.

6. The method for automatic testing error analysis of an ocean data platform based on a large model according to claim 5 is characterized in that: The method further includes training the error analysis model through multi-task training, and performing joint training on multiple meta-learning tasks, each task corresponding to a specific error scenario, wherein the meta-learning step includes initialization, inner loop, outer loop and hyperparameter tuning, specifically: The initialization is to initialize the parameters of the large language model and the meta-learning model; the inner loop is to perform one or more gradient updates on the training set of each task; the outer loop is to evaluate the model performance on the validation set of all tasks and adjust the initial parameters of the model according to the evaluation results; the hyperparameter tuning is to adjust the hyperparameters and optimize the model training process through performance evaluation on the validation set.

7. The method for analyzing errors in automated testing of a marine data platform based on a large model according to claim 6, characterized in that: The method further comprises: When a new error scenario occurs, a new meta-learning task is defined, defect analysis data related to the new meta-learning task is collected, and preprocessing is performed to obtain a new task dataset. The optimized error analysis model is fine-tuned using the new task dataset, and the fine-tuned error analysis model is used as the error analysis model deployed on the test development platform.

8. The method for analyzing errors in automated testing of a marine data platform based on a large model according to claim 7, characterized in that: The method also includes automatically assigning priorities based on the severity and impact scope of the defects in the defect analysis results, feeding back the defect detection results and corresponding priorities to the repair team, and feeding back the repaired automated test result data to the training set to re-optimize the error analysis model; integrating the re-optimized error analysis model into the CI / CD process, continuously running the automated test scripts in each integrated test, and using the error analysis model to analyze the test results in real time.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for automatic testing error analysis of an ocean data platform based on a large model is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an automatic test error analysis method for an ocean data platform based on a large model as described in any one of claims 1 to 8 is implemented.

Images