Automatic testing method, device and equipment and storage medium

Abstract

The invention discloses an automatic test method. The method comprises the steps of generating a feasible test demand based on document data and test dimensions of a to-be-tested product; classifying the feasible test requirements according to requirement types, and generating a test case set and a test script set corresponding to each requirement type; executing the test script set, and obtaining a page image in the execution process; inputting the page image into a pre-constructed OCR (Optical Character Recognition) module and an image recognition module, outputting a test execution result of the test script set, and generating a test report according to the test execution result; according to the method, the test comprehensiveness and the test coverage rate can be improved, and the analysis efficiency and the accuracy of the test result are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular, to an automated testing method, device, equipment and storage medium. Background Art

[0002] Automated testing is a method of performing software testing tasks with the help of automated testing tools and scripts. During the software development process, traditional manual testing relies on manual operation and verification of software functions one by one, while automated testing converts these repetitive and cumbersome testing steps into code or scripts that can run automatically.

[0003] During the software development process, although automated testing has many advantages compared with traditional manual testing, the current automated testing methods may not fully meet all testing requirements. In some specific scenarios, manual testing is still needed to supplement, which has certain limitations, resulting in some testing scenarios not being effectively covered. If the automated tool is not suitable for the current testing requirements, it will also lead to low testing efficiency. Without effective result analysis means, it will be difficult to extract useful information from a large amount of test data, thus affecting the accuracy and effectiveness of test results. Therefore, the quality and efficiency of automated testing need to be improved. Summary of the Invention

[0004] An embodiment of the present invention provides an automated testing method, which can improve the comprehensiveness and coverage rate of testing, and improve the analysis efficiency and accuracy of test results.

[0005] In a first aspect, an embodiment of the present invention provides an automated testing method, including: Generating feasible test requirements based on the document materials and test dimensions of the product to be tested; Classifying the feasible test requirements according to the requirement type, and generating a test case set and a test script set corresponding to each requirement type; Executing the test script set and obtaining the page images during the execution process; Inputting the page images into a pre-constructed OCR recognition module and image recognition module, outputting the test execution results of the test script set, and generating a test report according to the test execution results.

[0006] Further, the generating feasible test requirements based on the document materials and test dimensions of the product to be tested includes: Obtaining and analyzing several document materials of the product to be tested and predetermined test dimensions, and generating initial test requirements; wherein, the test dimensions are used to indicate that testers test the product to be tested based on specific test objectives; Deduplicate the initial test requirements, and mark the source of each test requirement after deduplication; wherein, the source mark is used to indicate the corresponding position of the test requirement in the document materials. Use natural language processing technology to conduct a feasibility analysis on the marked initial test requirements, and screen out the feasible test requirements from the initial test requirements according to the feasibility analysis results.

[0007] Furthermore, classify the feasible test requirements according to the requirement types, and generate a test case set and a test script set corresponding to each requirement type, including: Classify the feasible test requirements according to the requirement types, and generate a meta test case set and a meta test script set; wherein, the meta test case set includes meta test cases corresponding to the feasible test requirements of each requirement type, and the meta test script set includes meta test scripts corresponding to the feasible test requirements of each requirement type. Write a test case set based on the meta test case set; wherein, each test case in the test case set includes test preconditions, test execution steps, and expected effects of step execution. Generate a test script set for the feasible test requirements according to the test case set and the meta test script set, and determine the execution order of the test scripts in the test script set.

[0008] Furthermore, input the page image into a pre-constructed OCR recognition module and an image recognition module, and output the test execution results of the test script set, including: Input the page image into a pre-constructed OCR recognition module, and output the recognized text features. Input the page image into a pre-constructed image recognition module, and output the recognized image features. Obtain the test execution results of the test script set based on the text features and the image features.

[0009] Furthermore, the OCR recognition module includes a convolutional neural network, a recurrent neural network, and a CTC loss function. Then, inputting the page image into the pre-constructed OCR recognition module and outputting the recognized text features includes: Input the page image into a pre-constructed OCR recognition module, and perform a first preprocessing operation on the page image; wherein, the first preprocessing operation includes, but is not limited to, grayscale conversion, binarization, noise reduction, and correction operations. Use the convolutional neural network to extract features from the page image after the first preprocessing, and convert the extracted features into a one-dimensional feature sequence as the input of the recurrent neural network. The recurrent neural network predicts the one-dimensional feature sequence and outputs prediction information; The CTC loss function is used to convert the prediction information obtained from the recurrent neural network into the final text features.

[0010] Furthermore, inputting the page image into a pre-constructed image recognition module to output the recognized image features includes: Inputting the page image into a pre-constructed image recognition module to perform a second preprocessing operation on the page image; wherein, the second preprocessing operation includes but is not limited to contrast adjustment, brightness adjustment, denoising processing, and sharpening operation; Construct a semantic segmentation network using an encoder-decoder structure, input the second preprocessed page image into the semantic segmentation network, segment the page image, and output the segmented image features; Use the random forest algorithm to perform image classification on the page image, and screen the segmented image features according to the feature importance to obtain the final image features.

[0011] Furthermore, obtaining the test execution result of the test script set based on the text features and the image features includes: Use a pre-trained language model as a text encoder and a convolutional neural network as an image encoder to construct a mapping network, and map the text features and the image features into the same feature space; Use a loss function to calculate the difference between the text features and the image features in the common feature space, and use an optimization algorithm to minimize the difference; Calculate the first attention weight of the text features to the image features and the second attention weight of the image features to the text features; Based on the first attention weight and the second attention weight, fuse the text features and the image features to obtain a fused feature, and use the fused feature as the test execution result of the test script set.

[0012] In a second aspect, an embodiment of the present invention provides an automated test device, including: A test requirement generation module, configured to generate feasible test requirements based on the document materials and test dimensions of the product to be tested; A test script generation module, configured to classify the feasible test requirements according to the requirement type, and generate a test case set and a test script set corresponding to each requirement type; A test script execution module, configured to execute the test script set and obtain the page images during the execution process; A test report generation module is used to input the page image into a pre-constructed OCR recognition module and an image recognition module, output the test execution result of the test script set, and generate a test report according to the test execution result.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, including: A memory for storing a computer program; A processor for executing the computer program; Wherein, when the processor executes the computer program, it implements the automated test method according to any one of the above first aspects.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed, it implements the automated test method according to any one of the above first aspects.

[0015] Compared with the prior art, the automated test method provided by the embodiment of the present invention has the beneficial effects that: generating feasible test requirements based on the document materials and test dimensions of the product to be tested; classifying the feasible test requirements according to the requirement type, and generating a test case set and a test script set corresponding to each requirement type; executing the test script set, and obtaining the page image during the execution process; inputting the page image into a pre-constructed OCR recognition module and an image recognition module, outputting the test execution result of the test script set, and generating a test report according to the test execution result; the present invention can improve the comprehensiveness and test coverage of the test, and improve the analysis efficiency and accuracy of the test result. Description of the Drawings

[0016] In order to more clearly illustrate the technical features of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. Obviously, the following described drawings are only some embodiments of the present invention, and those skilled in the art can obtain other drawings according to these drawings without creative efforts.

[0017] Figure 1 It is a flowchart of an embodiment of an automated test method provided by the present invention; Figure 2 It is a structural schematic diagram of an embodiment of an automated test device provided by the present invention; Figure 3 It is a structural schematic diagram of an embodiment of an electronic device provided by the present invention. Detailed Embodiments

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] It should be noted that although functional module division is carried out in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division in the device or a different sequence in the flowchart. Terms such as "first" and "second" in the specification, claims and the above drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0021] In a first aspect, an embodiment of the present invention provides an automated testing method. Refer to Figure 1 , which is a schematic flowchart of an embodiment of an automated testing method provided by the present invention.

[0022] As Figure 1 shown, the method includes the following steps: S1: Generate feasible test requirements based on the documentation and test dimensions of the product to be tested; S2: Classify the feasible test requirements according to the requirement type, and generate a test case set and a test script set corresponding to each requirement type; S3: Execute the test script set and obtain the page images during the execution process; S4: Input the page images into a pre-constructed OCR recognition module and an image recognition module, output the test execution results of the test script set, and generate a test report according to the test execution results.

[0023] It should be noted that the present invention uses two models to assist in implementing an automated testing method. Among them, the first model is a general model, such as Doubao, GPT4, or Llama, etc. The main functions of the general model are language generation, context understanding, and natural language processing. These models are easy to master language rules and can generate appropriate content. The second model is a reasoning model, such as DeepSeek-R1 and GPT-o3, etc. The main function of the reasoning model is logical reasoning and inference. These models can perform logical reasoning and inference based on existing data, knowledge, and rules to obtain new conclusions or prediction results. The automated testing method of the present invention combines the general model and the reasoning model for application, which can give full play to their respective advantages and achieve more efficient and accurate testing.

[0024] In specific implementation, obtain the documentation of the product to be tested, select appropriate test dimensions according to the characteristics and requirements of the product to be tested, generate feasible test requirements based on the documentation and test dimensions, classify the feasible test requirements according to different requirement types, generate a test case set corresponding to each requirement type, generate a test script set based on the test case set, and the test script set is used to automatically execute the test cases. During the execution of the test script set, obtain the page image, and use the pre-constructed OCR recognition module and image recognition module to automatically recognize the text and graphic information in the page image, so as to output the test execution result of the test script set.

[0025] By comparing the test execution result with the expected result, mark the differences between the expected result and the actual result, and use relevant prompt words to guide the first model. For example, input the difference result into the first model and prompt it to output a test record document. The first model then summarizes and classifies the difference situations to generate test documents, including but not limited to test records, exception reports, or difference comparison tables, etc.

[0026] In summary, the present invention generates feasible test requirements based on the documentation and test dimensions of the product to be tested; classifies the feasible test requirements according to requirement types to generate a test case set and a test script set corresponding to each requirement type; executes the test script set and obtains the page image during the execution process; inputs the page image into the pre-constructed OCR recognition module and image recognition module to output the test execution result of the test script set, and generates a test report based on the test execution result. The present invention can not only generate basic test cases from a wide range of functional and scenario dimensions to ensure the test coverage of common functions and conventional scenarios, but also deeply dig and supplement test cases in complex scenarios according to the business logic and data rules of the system, which can improve the comprehensiveness and test coverage of the test. Through OCR recognition and image recognition, the key information in the page can be quickly extracted, which can improve the analysis efficiency and accuracy of the test results.

[0027] In an alternative embodiment, generating feasible test requirements based on the documentation and test dimensions of the product to be tested includes: Obtaining and analyzing a number of documentation of the product to be tested and predetermined test dimensions to generate initial test requirements; wherein, the test dimensions are used to instruct testers to test the product to be tested based on specific test objectives; Removing duplicates from the initial test requirements and marking the source of each test requirement after duplicate removal; wherein, the source marking is used to indicate the corresponding position of the test requirement in the documentation; Using natural language processing technology to perform a feasibility analysis on the marked initial test requirements, and screening out feasible test requirements from the initial test requirements according to the feasibility analysis results.

[0028] Specifically, in the automated test method of the present invention, the product to be tested can be other systems and platforms such as government information systems, enterprise-level applications, or financial applications. In this embodiment, taking a government information system as an example, when the product to be tested is a government information system, obtain the documentation of the government information system, such as bidding contracts, construction plans, requirement specifications, and user manuals, etc., input the documentation and test dimensions into the first model, and through multiple rounds of prompt training of the first model, guide the first model to learn the document content and locate the requirement information.

[0029] It should be noted that before inputting the documentation into the first model, an index is constructed for the documentation. According to the content and structure of the document, a reasonable table of contents structure is divided. The table of contents structure should clearly reflect the theme, chapters, and sub-chapters of the document. An index is constructed for each table of contents structure, including chapter titles, keywords, and paragraph numbers, etc., and key chapters are marked. For example, the core requirement chapter, design constraint chapter, and risk chapter are marked, etc.; further, the test dimensions are determined. The test dimensions are used to instruct testers to test the product to be tested based on specific test objectives. The test dimensions can include functional testing, performance testing, security testing, compatibility testing, user interface testing, or other test dimensions. The test dimensions include several test sub-dimensions. Exemplarily, functional testing includes functional integrity testing, functional adaptability testing, and functional interoperability testing, etc. Functional integrity testing means that the software should include all the functions necessary to meet user requirements and business processes, without missing or lacking key functions. Functional adaptability testing means that it should be able to adapt to changes in different operating environments, data inputs, and user operation methods, etc., and maintain stable and correct operation. Functional interoperability testing means that when interacting with other software systems, hardware devices, or external interfaces, the functions should be able to work correctly with these external elements.

[0030] Input different document materials and test sub-dimensions with the index built into the first model. The model analyzes the document materials according to the test sub-dimensions to obtain the initial test requirements of the complete information system.

[0031] Input the obtained initial test requirements into the second model. The second model preprocesses them, including duplicate removal and source marking. Duplicate removal is used to identify and delete duplicate test requirements and merge test requirements with the same characteristics. Source marking indicates the origin of the test requirements and is used to indicate the corresponding position of the test requirements in the document materials so that the original requirements can be traced when generating detailed information of the test results later. After the above processing, the second model outputs the preprocessed test requirements.

[0032] After preprocessing the test requirements, the first model uses the semantic understanding algorithm in natural language processing (NLP) technology to modify the test requirements according to the principles of clarity, intuitiveness, and non-ambiguity. Specifically, by learning a large number of clearly labeled requirement texts, a semantic knowledge base is constructed. The test requirements to be analyzed are compared with similar expressions in the semantic knowledge base. The requirement text is constructed into a structured syntax tree through a syntax parsing algorithm to analyze the sentence structure and component relationships. Potential ambiguities are identified through lexical polysemy analysis and context correlation analysis techniques, so as to discover possible fuzzy, ambiguous, or unclear places in the requirement text and modify the test requirements. For example, the original requirement is "Residents can complete insurance registration on the system, fill in basic information, the system has to check the information for correctness, and can receive feedback after submission", and the requirement modified by the first model is "Verify the complete process of residents from accessing the social security handling system, filling in personal identity information, employment status, and insurance type selection, to submitting an insurance registration application. Ensure that each step operates smoothly, the page prompts are clear and accurate, and the system can respond in a timely manner after submitting the application."

[0033] Further, input the modified test requirements into the second model. The second model atomically splits the requirements, splitting each test requirement into smaller and independent test atoms, such as test steps, preconditions, and expected results, etc. Define the filtering dimensions and judgment rules, and conduct a feasibility analysis on each test requirement and its split test atoms according to the defined judgment rules. The analysis process includes precondition verification, operation observability assessment, expected result quantification check, and dependency review. Among them, precondition verification is used to check whether each precondition is met, such as interface reachability and version matching, etc. If the precondition is not met, the test requirement or test atom cannot be executed. Operation observability assessment is used to judge whether the operation result can be captured at the UI layer, such as button state change, page jump, etc. If the operation result cannot be observed, the test requirement or test atom cannot be executed. Expected result quantification check is used to verify whether the expected result has quantifiable indicators, such as database record number change, response time threshold, etc. If the expected result cannot be quantified, the test requirement or test atom cannot be executed. Dependency review is used to judge whether there is unimplemented content involved. If unimplemented content is involved, the test requirement or test atom cannot be executed. According to the feasibility analysis results, retain the test requirements that meet the judgment rules in the initial test requirements, and delete the non-executable test requirements to obtain the feasible test requirements.

[0034] In an alternative embodiment, classify the feasible test requirements according to the requirement type, and generate a test case set and a test script set corresponding to each requirement type, including: Classify the feasible test requirements according to the requirement type to generate a meta test case set and a meta test script set; wherein, the meta test case set includes meta test cases corresponding to the feasible test requirements of each requirement type, and the meta test script set includes meta test scripts corresponding to the feasible test requirements of each requirement type; Write a test case set based on the meta test case set; wherein, each test case in the test case set includes test preconditions, test execution steps, and expected effects of step execution; Generate a test script set for the feasible test requirements according to the test case set and the meta test script set, and determine the execution order of the test scripts in the test script set.

[0035] Specifically, classify the feasible test requirements according to the requirement type. Exemplarily, functional requirements can usually be classified according to specific function operations, such as classification by function types such as addition, deletion, modification, viewing, and statistics. At the same time, functional requirements can also be further classified according to the chronological order of creating data, modifying data, viewing data, and deleting data. This classification method helps to determine the execution order of various types of functional requirements, generate a meta-test case set and a meta-test script set. The meta-test case set includes meta-test cases corresponding to the feasible test requirements of each requirement type, and the meta-test script set includes meta-test scripts corresponding to the feasible test requirements of each requirement type. The meta-test script defines the basic steps and structure required to test this type of function.

[0036] Write a test case set based on the meta-test cases. The test cases include the test preconditions, test execution steps, and the expected effects obtained after each step of execution for this type of requirement, and obtain a test case set classified by type. Input the meta-test script set and the test case set into the second model. The second model writes corresponding type test scripts according to different types of test cases and determines the execution order of the test scripts.

[0037] In an alternative embodiment, the inputting the page image into a pre-constructed OCR recognition module and an image recognition module and outputting the test execution result of the test script set includes: Input the page image into a pre-constructed OCR recognition module and output the recognized text features; Input the page image into a pre-constructed image recognition module and output the recognized image features; Obtain the test execution result of the test script set based on the text features and the image features.

[0038] Specifically, construct an OCR recognition module and an image recognition module, process the page image during the script execution process, output the text features and image features recognized from the page image, and obtain the test execution result of the test script set based on the text features and the image features.

[0039] In an alternative embodiment, the OCR recognition module includes a convolutional neural network, a recurrent neural network, and a CTC loss function. Then, the inputting the page image into a pre-constructed OCR recognition module and outputting the recognized text features includes: Input the page image into a pre-constructed OCR recognition module and perform a first preprocessing operation on the page image; wherein, the first preprocessing operation includes, but is not limited to, grayscale conversion, binarization, noise reduction, and correction operations; Use a convolutional neural network to extract features from the first preprocessed page image, and convert the extracted features into a one-dimensional feature sequence as the input to the recurrent neural network; The recurrent neural network makes predictions on the one-dimensional feature sequence and outputs prediction information; Use the CTC loss function to convert the prediction information obtained from the recurrent neural network into the final text features.

[0040] Specifically, OCR (Optical Character Recognition) is an optical character recognition technology, which is a machine vision task for extracting text information from images. The OCR recognition module of the present invention includes a convolutional neural network, a recurrent neural network, and a CTC loss function. After inputting the page image into the OCR recognition module, first perform a first preprocessing operation on the image, including but not limited to grayscale conversion, binarization, noise reduction, and correction, etc. Grayscale conversion converts the color image into a grayscale image, reducing the dimension of the data and the complexity of subsequent processing. Binarization converts the image into a black and white binary image, further simplifying the data representation and highlighting the contrast between characters and the background. The noise reduction operation can remove random noise in the image, making the character edges sharper and improving the accuracy of subsequent feature extraction. The correction operation can correct the tilt and distortion of the image, ensuring that the position of the characters in the image is more regular. Through a series of preprocessing operations, the overall quality of the image can be improved and the data format of the image can be unified, making it easier to extract features from the image subsequently.

[0041] Use a pre-trained convolutional neural network (such as ResNet, VGG) as a feature extractor to extract features from the first preprocessed page image. The convolutional neural network can extract multi-scale feature maps from the input image. By fusing the feature maps output from different layers, different scale feature information can be combined. For example, the low-level feature maps contain more detailed information, such as the edges and textures of characters, while the high-level feature maps contain more semantic information, such as the shapes and categories of characters. By sampling and splicing operations, the high-resolution features at the low level and the semantic features at the high level are fused together, which can improve the accuracy of text detection, especially for the detection of small-size texts. The convolutional neural network automatically extracts representative local features and spatial information from the image through a series of operations such as convolutional layers and pooling layers, and converts the extracted features into a one-dimensional feature sequence as the input to the recurrent neural network.

[0042] Recurrent neural networks can understand the sequential order and semantic associations between characters, can process text sequences of variable lengths. After receiving a one-dimensional feature sequence, it makes predictions on it and outputs prediction information. The CTC loss function calculates the probabilities of all possible character sequences to find the most likely character sequence, realizes character alignment and recognition, and converts the prediction information obtained from the recurrent neural network into the final text features.

[0043] Each step in the OCR recognition method of the present invention cooperates with each other. The preprocessing operation improves the image quality and data uniformity. The convolutional neural network feature extraction provides rich feature information. The recurrent neural network prediction captures the context information of the text. The CTC loss function solves the character alignment problem, jointly improving the accuracy and efficiency of OCR recognition.

[0044] In an optional implementation manner, the inputting the page image into a pre-constructed image recognition module and outputting the recognized image features includes: Inputting the page image into a pre-constructed image recognition module to perform a second preprocessing operation on the page image; wherein, the second preprocessing operation includes but is not limited to contrast adjustment, brightness adjustment, denoising processing, and sharpening operation; Constructing a semantic segmentation network using an encoder-decoder structure, inputting the page image after the second preprocessing into the semantic segmentation network to segment the page image and output segmentation image features; Using a random forest algorithm to perform image classification on the page image and screening the segmentation image features according to the feature importance to obtain the final image features.

[0045] Specifically, after the page image is input into the image recognition module, a second preprocessing operation is first performed on the image, including but not limited to contrast adjustment, brightness adjustment, denoising processing, and sharpening operations. Among them, adjusting the contrast through linear transformation and other methods can make the difference between the page content and the background more obvious. High-contrast images can better highlight the key information in the page, facilitating subsequent recognition and analysis. Adjusting the brightness by directly increasing or decreasing the pixel gray value can make the overall brightness of the image more appropriate, avoiding the situation of being too dark or too bright affecting the readability of the image. Using methods such as mean filtering, median filtering, and Gaussian filtering to remove noise, these methods correspond to different types of noise respectively. Denoising processing can effectively remove the interference factors in the image and improve the clarity and quality of the image. Using the Laplace operator and other methods for sharpening operations to enhance the high-frequency components of the image to highlight the edges and details. The sharpened image can more clearly display the edge and detail information in the page, which helps to improve the subsequent recognition accuracy. Through this series of processes, the image quality can be effectively improved, background interference can be reduced, content features can be highlighted, and thus the recognition accuracy can be improved.

[0046] A semantic segmentation network is constructed using an encoder-decoder structure. The page image after the second preprocessing is input into the semantic segmentation network, and the page image is segmented and the segmented image features are output. Among them, the encoder part gradually reduces the size of the feature map through continuous convolution and downsampling operations, while increasing the degree of abstraction of the features, and extracts the high-level semantic information of the image. For example, in image recognition, it can distinguish which areas may be buttons and which are text boxes, etc. The decoder part restores the feature map output by the encoder to the same size as the input image through upsampling operations, and combines the feature information of different levels of the encoder to gradually refine the segmentation result and make the segmentation boundary more accurate.

[0047] The random forest algorithm is used to classify the page areas. All the extracted features are used as input to train the random forest model. The random forest algorithm has high accuracy and stability and can effectively classify the page image. During the training process, the random forest will sort the features according to the importance of the features. After the training is completed, check the importance scores of each feature and remove the features with low importance. For example, among the position coordinates of the page elements, the coordinate features related to the image and the key text positions have higher importance, while some coordinate features far from the page elements have very low importance because they are likely to be irrelevant features introduced by the background. Then these coordinate features with low importance can be removed to obtain the final image features.

[0048] In an alternative embodiment, obtaining the test execution result of the test script set based on the text features and the image features includes: Use a pre-trained language model as the text encoder and a convolutional neural network as the image encoder to construct a mapping network that maps text features and image features into the same feature space; Use a loss function to calculate the difference between the text features and the image features in the common feature space, and adopt an optimization algorithm to minimize the difference; Calculate the first attention weight of the text features to the image features and the second attention weight of the image features to the text features; Based on the first attention weight and the second attention weight, fuse the text features and the image features to obtain fused features, and use the fused features as the test execution result of the test script set.

[0049] Specifically, use a pre-trained language model (such as BERT) as the text encoder and a convolutional neural network (such as ResNet-50) as the image encoder to construct a fully-connected neural network as the mapping network, which maps text features and image features into the same feature space. The fully-connected neural network has strong fitting ability and can learn the complex relationship between text and image features. Each hidden layer consists of multiple neurons and is transformed through a non-linear activation function (such as ReLU), which can increase the expression ability of the network.

[0050] During the mapping process, it is necessary to define a suitable loss function to measure the difference between the text features and the image features in the common feature space, such as mean squared error loss, contrastive loss, etc., and use an optimization algorithm (such as stochastic gradient descent) to minimize this difference, so that the text features and the image features can be accurately matched in the common feature space.

[0051] Furthermore, introduce an attention mechanism to let the model automatically focus on the relevant parts between the text and the intermediate features of the image, calculate the first attention weight of the text features to the image features and the second attention weight of the image features to the text features, calculate the first attention weight by calculating the similarity between the text features and the image features, and perform weighted summation on the image features according to these weights to obtain an image feature representation related to the text. Similarly, calculate the second attention weight of the image features to the text features to obtain a text feature representation related to the image. Finally, splice or perform weighted summation on these two representations to obtain fused features, and use the fused features as the test execution result of the test script set.

[0052] In a second aspect, an embodiment of the present invention provides an automated testing device. Refer to Figure 2 , which is a schematic structural diagram of an embodiment of an automated testing device provided by the present invention.

[0053] As Figure 2 shown, the device includes: A test requirement generation module 21, configured to generate feasible test requirements based on the documentation of the product to be tested and test dimensions; A test script generation module 22, configured to classify the feasible test requirements according to requirement types, and generate a test case set and a test script set corresponding to each requirement type; A test script execution module 23, configured to execute the test script set and obtain page images during the execution process; A test report generation module 24, configured to input the page images into a pre-constructed OCR recognition module and an image recognition module, output the test execution results of the test script set, and generate a test report according to the test execution results.

[0054] In an optional implementation manner, the test requirement generation module 21 is further configured to: Obtain and analyze a number of documentation of the product to be tested and predetermined test dimensions, and generate initial test requirements; wherein, the test dimensions are used to instruct testers to test the product to be tested based on specific test objectives; Perform duplicate removal processing on the initial test requirements, and perform source marking on each test requirement after duplicate removal processing; wherein, the source marking is used to indicate the corresponding position of the test requirement in the documentation; Adopt natural language processing technology to perform feasibility analysis on the marked initial test requirements, and screen out feasible test requirements from the initial test requirements according to the feasibility analysis results.

[0055] In an optional implementation manner, the test script generation module 22 is further configured to: Classify the feasible test requirements according to requirement types, and generate a meta test case set and a meta test script set; wherein, the meta test case set includes meta test cases corresponding to the feasible test requirements of each requirement type, and the meta test script set includes meta test scripts corresponding to the feasible test requirements of each requirement type; Write a test case set based on the meta test case set; wherein, each test case in the test case set includes a test precondition, a test execution step, and an expected effect of step execution; Generate a test script set for the feasible test requirements according to the test case set and the meta test script set, and determine the execution order of the test scripts in the test script set.

[0056] In an optional implementation manner, the test report generation module 24 is further configured to: Input the page images into a pre-constructed OCR recognition module, and output the recognized text features; Input the page image into a pre - constructed image recognition module, and output the recognized image features; Obtain the test execution result of the test script set based on the text features and the image features.

[0057] In an alternative embodiment, the OCR recognition module includes a convolutional neural network, a recurrent neural network, and a CTC loss function. Then, the test report generation module 24 is further configured to: Input the page image into a pre - constructed OCR recognition module, and perform a first pre - processing operation on the page image; wherein, the first pre - processing operation includes, but is not limited to, grayscale conversion, binarization, noise reduction, and correction operations; Use the convolutional neural network to extract features from the page image after the first pre - processing, and convert the extracted features into a one - dimensional feature sequence as the input of the recurrent neural network; The recurrent neural network makes predictions on the one - dimensional feature sequence and outputs prediction information; Use the CTC loss function to convert the prediction information obtained from the recurrent neural network into the final text features.

[0058] In an alternative embodiment, the test report generation module 24 is further configured to: Input the page image into a pre - constructed image recognition module, and perform a second pre - processing operation on the page image; wherein, the second pre - processing operation includes, but is not limited to, contrast adjustment, brightness adjustment, denoising processing, and sharpening operations; Construct a semantic segmentation network using an encoder - decoder structure, input the page image after the second pre - processing into the semantic segmentation network, segment the page image, and output segmentation image features; Use a random forest algorithm to perform image classification on the page image, and screen the segmentation image features according to the feature importance to obtain the final image features.

[0059] In an alternative embodiment, the test report generation module 24 is further configured to: Use a pre - trained language model as a text encoder and a convolutional neural network as an image encoder to construct a mapping network, and map the text features and the image features into the same feature space; Use a loss function to calculate the difference between the text features and the image features in the common feature space, and use an optimization algorithm to minimize the difference; Calculate the first attention weight of the text features to the image features and the second attention weight of the image features to the text features; Fusing the text feature and the image feature based on the first attention weight and the second attention weight to obtain a fused feature, and using the fused feature as the test execution result of the test script set.

[0060] In a third aspect, an embodiment of the present invention provides an electronic device. Refer to Figure 3 As shown, it is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0061] As Figure 3 shown, the device includes: A memory 31 for storing a computer program; A processor 32 for executing the computer program; Wherein, when the processor 32 executes the computer program, it implements the automated test method described in any of the above embodiments.

[0062] Exemplarily, the computer program may be divided into one or more modules / units. The one or more modules / units are stored in the memory 31 and executed by the processor 32 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device.

[0063] The processor 32 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0064] The memory 31 can be used to store the computer program and / or modules. By running or executing the computer program and / or modules stored in the memory 31 and calling the data stored in the memory 31, the processor 32 realizes various functions of the electronic device. The memory 31 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 31 may include a high-speed random access memory and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0065] It should be noted that the above-mentioned electronic device includes, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 3 The structural schematic diagram is only an example of the above-mentioned electronic device and does not constitute a limitation on the electronic device. It may include more components than shown in the figure, or combine some components, or different components.

[0066] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, and when the computer program is executed, the automated test method described in any of the above embodiments is implemented.

[0067] It should be understood that all or part of the processes of implementing the above automated test method in the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above automated test method can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0068] As described above, it is only the preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. It should be pointed out that for those skilled in the art, without departing from the technical principle of the present invention, several equivalent obvious variant forms and / or equivalent replacement forms can be made, and these obvious variant forms and / or equivalent replacement forms should also be regarded as the protection scope of the present invention.

Claims

1. An automated testing method, characterized in that, Including: Generating feasible test requirements based on the documentation and test dimensions of the product to be tested; Classifying the feasible test requirements according to the requirement types, and generating a test case set and a test script set corresponding to each requirement type; Executing the test script set and obtaining the page images during the execution process; Inputting the page images into a pre-constructed OCR recognition module and an image recognition module, outputting the test execution results of the test script set, and generating a test report based on the test execution results.

2. The automated testing method according to claim 1, characterized in that, The generating of feasible test requirements based on the documentation and test dimensions of the product to be tested includes: Obtaining and analyzing several documentation of the product to be tested and the pre-determined test dimensions, and generating initial test requirements; wherein, the test dimensions are used to instruct testers to test the product to be tested based on specific test objectives; Removing duplicates from the initial test requirements, and marking the source of each test requirement after duplicate removal; wherein, the source marking is used to indicate the corresponding position of the test requirement in the documentation; Performing a feasibility analysis on the marked initial test requirements using natural language processing technology, and screening out feasible test requirements from the initial test requirements according to the feasibility analysis results.

3. The automated testing method according to claim 1, characterized in that, The classifying of the feasible test requirements according to the requirement types and generating a test case set and a test script set corresponding to each requirement type includes: Classifying the feasible test requirements according to the requirement types, and generating a meta test case set and a meta test script set; wherein, the meta test case set includes meta test cases corresponding to the feasible test requirements of each requirement type, and the meta test script set includes meta test scripts corresponding to the feasible test requirements of each requirement type; Writing a test case set based on the meta test case set; wherein, each test case in the test case set includes test preconditions, test execution steps, and expected effects of step execution; Generating a test script set for the feasible test requirements according to the test case set and the meta test script set, and determining the execution order of the test scripts in the test script set.

4. The automated testing method according to claim 1, characterized in that, The inputting of the page images into a pre-constructed OCR recognition module and an image recognition module and outputting the test execution results of the test script set includes: Inputting the page images into a pre-constructed OCR recognition module and outputting the recognized text features; Inputting the page images into a pre-constructed image recognition module and outputting the recognized image features; Obtaining the test execution results of the test script set based on the text features and the image features.

5. The automated testing method according to claim 4, characterized in that, If the OCR recognition module includes a convolutional neural network, a recurrent neural network, and a CTC loss function, then the inputting of the page images into a pre-constructed OCR recognition module and outputting the recognized text features includes: Inputting the page images into a pre-constructed OCR recognition module and performing a first preprocessing operation on the page images; wherein, the first preprocessing operation includes, but is not limited to, grayscale conversion, binarization, noise reduction, and correction operations; Use a convolutional neural network to extract features from the first preprocessed page image, convert the extracted features into a one-dimensional feature sequence, and use it as the input of the recurrent neural network; The recurrent neural network makes predictions on the one-dimensional feature sequence and outputs prediction information; Use the CTC loss function to convert the prediction information obtained from the recurrent neural network into the final text features.

6. The automated testing method according to claim 4, characterized in that, The inputting the page image into a pre-constructed image recognition module and outputting the recognized image features includes: Input the page image into a pre-constructed image recognition module and perform a second preprocessing operation on the page image; wherein, the second preprocessing operation includes but is not limited to contrast adjustment, brightness adjustment, denoising processing, and sharpening operation; Construct a semantic segmentation network using an encoder-decoder structure, input the second preprocessed page image into the semantic segmentation network, segment the page image, and output the segmented image features; Use the random forest algorithm to classify the page image, and screen the segmented image features according to the feature importance to obtain the final image features.

7. The automated testing method according to claim 4, characterized in that, The obtaining the test execution result of the test script set based on the text features and the image features includes: Use a pre-trained language model as the text encoder and a convolutional neural network as the image encoder to construct a mapping network, and map the text features and the image features into the same feature space; Use the loss function to calculate the difference between the text features and the image features in the common feature space, and use the optimization algorithm to minimize the difference; Calculate the first attention weight of the text features to the image features and the second attention weight of the image features to the text features; Fuse the text features and the image features based on the first attention weight and the second attention weight to obtain the fused features, and use the fused features as the test execution result of the test script set.

8. An automated testing device, characterized in that,Includes: A test requirement generation module, configured to generate feasible test requirements based on the document materials and test dimensions of the product to be tested; A test script generation module, configured to classify the feasible test requirements according to the requirement type, and generate a test case set and a test script set corresponding to each requirement type; A test script execution module, configured to execute the test script set and obtain the page images during the execution process; A test report generation module, configured to input the page images into a pre-constructed OCR recognition module and an image recognition module, output the test execution result of the test script set, and generate a test report according to the test execution result.

9. An electronic device, characterized in that, Includes: A memory, configured to store a computer program; A processor, configured to execute the computer program; Wherein, when the processor executes the computer program, it implements the automated test method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the automated test method according to any one of claims 1 to 7.

Images