Machine Learning-Based Automated Testing System and Method for Low-Code Applications

Abstract

This invention relates to an automated testing system and method for low-code applications based on machine learning, comprising the following key steps: First, receiving test requirements from the low-code application and obtaining interface element diagrams; second, processing the interface element diagrams using image processing and semantic recognition techniques to extract the interface model; then, automatically generating and executing test cases based on the interface model, and collecting test result data; finally, analyzing the test result data and generating a detailed test report. This invention also involves constructing and using a framework recognition network and a text extraction network to accurately identify the categories and text meanings of interface elements, and using clustering analysis and decision tree algorithms to deeply analyze test data, thereby effectively identifying abnormal patterns, error causes, and paths. This invention can be widely applied in the field of software development, especially for application testing on low-code platforms, improving software quality and testing efficiency.

Description

Technical Field

[0001] This invention relates to the field of low-code application technology, and in particular to an automated testing system and method for low-code applications based on machine learning. Background Technology

[0002] In today's rapidly evolving software industry, low-code application development has garnered widespread attention. Low-code platforms simplify programming, enabling even non-professional programmers to easily build applications, significantly improving development efficiency and enabling rapid application iteration. However, with the increasing prevalence of low-code applications, the complexity and challenges of their testing process have become increasingly apparent. Traditional software testing methods often rely on manually writing test cases, which is not only time-consuming and labor-intensive, but also, when dealing with low-code applications, their rapid changes and diverse interface elements often result in unsatisfactory efficiency and accuracy.

[0003] Existing automated testing technologies, such as image recognition-based testing methods, while improving testing efficiency to some extent, still face numerous challenges when handling low-code applications. The interface elements and layouts of low-code applications may change frequently, requiring testing systems to quickly adapt to these changes and accurately identify new elements and layouts. Furthermore, automated testing tools often struggle to effectively handle complex interactions and data verification issues in low-code applications. For example, when applications involve complex data logic or specific user interactions, automated test scripts may require frequent updates, increasing maintenance costs and reducing testing flexibility and efficiency.

[0004] For example, Chinese patent application CN114238110A discloses a software application testing method, apparatus, device, and storage medium. This invention obtains the code to be tested and test cases corresponding to the target software application; executes the test script corresponding to the test cases in breakpoint testing mode, and obtains the script execution result and valid executed code after the test script is completed; if the script execution result is inconsistent with the predicted result corresponding to the test case, the test code segment corresponding to the test case in the code to be tested is obtained; code coverage is determined based on the valid executed code and the test code segment, and the location of abnormal code is determined based on the code coverage. Because the test script is executed in breakpoint testing mode, the valid executed code can be accurately obtained. When the script execution result is inconsistent with the predicted result, the location of abnormal code can be quickly located based on the code coverage obtained by comparison, reducing the difficulty for developers to locate abnormal code.

[0005] All of the above patents suffer from the problems described in this background: when interface elements, layout, or interaction logic change, test scripts need to be updated manually, which is not only time-consuming but also reduces testing efficiency. The recognition accuracy of interface elements is insufficient, and existing automated testing tools have limited capabilities in simulating these complex scenarios and cannot fully cover all possible test scenarios. To solve the above problems, this application designs a low-code application automated testing system and method based on machine learning. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies by providing an automated testing system and method for low-code applications based on machine learning. First, the system receives the testing requirements of the low-code application and obtains the interface element diagram. Second, it processes the interface element diagram using image processing and semantic recognition technologies to extract the interface model. Then, it automatically generates and executes test cases based on the interface model and collects test result data. Finally, it analyzes the test result data and generates a detailed test report.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A machine learning-based method for automated testing of low-code applications includes the following steps:

[0009] S1: Receive test requirements for low-code applications and obtain the UI element diagrams of the low-code applications;

[0010] S2: Process the interface element diagram to extract the interface model of the low-code application;

[0011] S3: Based on the interface model, generate test cases, execute the test cases, and collect test result data;

[0012] S4: Analyze the test result data and generate a test report.

[0013] The specific steps of S2 are as follows:

[0014] S2.1: Convert the interface element image into an element grayscale image and preprocess the element grayscale image;

[0015] S2.2: Extract frame matching features of interface element graphs using image processing technology, construct a frame recognition network, and use the frame recognition network to identify the frame matching features to obtain the interface element categories;

[0016] S2.3: Extract text matching features from interface element graphs using semantic recognition technology, construct a text extraction network, and extract the text matching features through the text extraction network to obtain the meaning of the interface text;

[0017] S2.4: Generate the interface model of the low-code application based on the interface element categories and the meaning of the interface text.

[0018] The specific steps for extracting frame matching features from the interface element graph using image processing technology are as follows:

[0019] S2.2.1: Convert the interface element image into an element grayscale image, and preprocess the element grayscale image, wherein the preprocessing includes image denoising and image enhancement;

[0020] S2.2.2: Perform histogram statistics on the preprocessed element grayscale image, and use statistical learning methods to cut out the target region image of the element grayscale image;

[0021] S2.2.3: Extract corner features and gray-level centroids from the target region image. Calculate the frame descriptor based on the angle between the corner features and the gray-level centroids. Use an MLP network to transform the frame descriptor into high-dimensional data and add it to the gray-level centroids to generate frame matching features. The formula for calculating the frame descriptor is:

[0022]

[0023] Among them, t d This represents the frame descriptor, where 'a' represents the index of a single corner feature in the target region map, 'A' represents the total number of corner features in the target region map, and 'Ψ' represents the total number of corner features in the target region map. a β represents the angle between the feature of the a-th corner point and the gray centroid. a Let F(·) represent the weighted weight value of the feature of the a-th corner point, and let F(·) represent the Fourier transform function. c and y c c represents the x and y coordinates of the grayscale centroid. a c represents the feature of the a-th corner point of the target region map. m This represents the average value of the corner features of the target region map.

[0024] The construction framework identifies the network, including:

[0025] The input layer is used to decode the matching features of the framework through an attention mechanism to obtain the input parameters of the hidden layer;

[0026] Hidden layers are used to calculate the similarity between the input parameters of the hidden layers and the typical frame element dataset, and to obtain the classification probability of the input parameters of each hidden layer.

[0027] The voting decision layer is used to vote on the category of the hidden layer input parameters based on the classification probability, and the category of the hidden layer input parameter with the most votes is used as the category of the interface element.

[0028] The specific steps for extracting text matching features from interface element graphs using semantic recognition technology are as follows:

[0029] S2.3.1: The preprocessed grayscale image of the elements is processed by edge detection and adaptive binarization to extract the text region;

[0030] S2.3.2: Perform vertical projection on the text region, divide the row and column boundaries of the text region according to the maximum and minimum values ​​of the vertical projection function, match the function values ​​at the row and column boundaries with the function values ​​of the text region, and obtain the text content vector;

[0031] S2.3.3: Calculate the gray-level co-occurrence matrix of the text content vector, obtain the frequency distribution function of the text content vector through Fourier transform, and calculate the text matching features based on the gray-level co-occurrence matrix and the frequency distribution function. The formula for calculating the text matching features is as follows:

[0032]

[0033] Among them, T f Let E{·} represent the text matching features, E{·} represent the annealing genetic function, and K represent the text matching features. b Let F represent the Boltzmann constant, G represent the gray-level co-occurrence matrix, F represent the frequency distribution function, n represent the unit vector of the text content vector, N represent the total length of the text content vector, and f represent the frequency distribution function. n X represents the frequency of the nth text content vector. n This represents the magnitude of the nth text content vector.

[0034] The text extraction network includes:

[0035] The input layer takes the text matching features as input parameters, normalizes and matrix quantizes the text matching features, and transforms them into a 64×64 feature matrix.

[0036] The convolutional pooling layer performs four stages to convolve the input 64×64 feature matrix through a 3×3 convolution kernel, and then uses the ReLU activation function and max pooling operation to obtain a 2×2 mapping feature. The max pooling operation includes a 2×2 pooling kernel.

[0037] The output layer flattens the mapped features into a compact feature vector to obtain a text descriptor. It calculates the semantic similarity between the text descriptor and the text semantic library through character pattern matching, and determines whether the convergence condition is met. If the convergence condition is not met, the semantic similarity error is backpropagated through the loss function. If the convergence condition is met, the semantic meaning of the text corresponding to the highest semantic similarity is output as the meaning of the interface text.

[0038] The test case generation method is used to automatically generate test scripts based on the element types, location information, and semantic information in the interface model, including element identification, element interaction determination, and format conversion. The test result data collection method includes obtaining screenshots, log files, and error reports during the test process.

[0039] The specific steps for analyzing the test result data are as follows:

[0040] S4.1: Identify abnormal patterns and trends during the testing process using cluster analysis algorithms;

[0041] S4.2: Identify the causes and error paths in the testing process using decision trees;

[0042] S4.3: Statistical analysis of test pass rate and test coverage.

[0043] A machine learning-based low-code application automated testing system, comprising an element extraction module, a test case generation module, a test case execution module, a data analysis module, and a data reporting module;

[0044] The element extraction module is used to receive the test requirements of the low-code application, obtain the interface element diagram of the low-code application, and extract the interface model of the low-code application.

[0045] The test case generation module is used to automatically generate test scripts based on the element types, location information, and semantic information in the interface model.

[0046] The test case execution module is used to convert test scripts into a format executable by automated testing tools, execute the test scripts through automated testing tools, and collect test result data.

[0047] The data analysis module is used to analyze the collected test result data;

[0048] The data reporting module is used to generate test reports;

[0049] The element extraction module includes:

[0050] Image preprocessing unit: Responsible for receiving the interface element images of low-code applications, converting them into grayscale images, and performing preprocessing operations such as image denoising and enhancement to improve the accuracy of subsequent feature extraction;

[0051] Feature extraction unit: Uses image processing techniques to identify and extract frame features and text features of interface elements;

[0052] Frame recognition unit: A network trained using a machine learning model, used for high-precision identification of interface element categories;

[0053] Text extraction unit: A network trained through a machine learning model, specifically designed to understand and extract text information from the interface, including the semantic meaning of the text;

[0054] The test case generation module includes:

[0055] Interface Model Analysis Unit: Based on the element types, positions, and semantic information in the interface model, plan the logic and content of test cases;

[0056] Test script generation unit: Automatically generates test scripts based on the interface model parsing results and determines the interaction methods between elements;

[0057] The test case execution module includes:

[0058] Test script conversion unit: Converts test scripts into a format executable by automated testing tools, ensuring compatibility with various testing platforms;

[0059] Test Execution Unit: Responsible for executing test scripts using automated testing tools;

[0060] Data collection unit: Used to collect data such as screenshots, log files, and error reports generated during the testing process;

[0061] The data analysis module includes:

[0062] Data Analysis Unit: Uses algorithms such as clustering analysis and decision trees to conduct in-depth analysis of the collected test results data, and identify abnormal patterns and error paths.

[0063] Statistical Evaluation Unit: Responsible for statistical testing coverage and pass rate, as well as other key performance indicators.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] 1. By receiving test requirements from low-code applications and automatically extracting the interface model, this invention can automatically generate and execute test cases, significantly improving testing efficiency and reducing the investment of human resources.

[0066] 2. This invention utilizes image processing and machine learning techniques to accurately identify the frame features and text features of interface element diagrams, thereby accurately extracting the interface model and improving the accuracy and reliability of testing. Attached Figure Description

[0067] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0068] Figure 1This is a flowchart illustrating the automated testing method for low-code applications based on machine learning, as described in Embodiment 1 of the present invention.

[0069] Figure 2 This is a schematic diagram of the corner feature determination area in Embodiment 1 of the present invention;

[0070] Figure 3 This is a diagram of the framework recognition network structure in Embodiment 1 of the present invention;

[0071] Figure 4 This is a diagram of the text extraction network structure in Embodiment 1 of the present invention;

[0072] Figure 5 This is a block diagram of the low-code application automatic testing system based on machine learning, which is the second embodiment of the present invention. Detailed Implementation

[0073] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0074] Example 1:

[0075] Please see Figure 1 The present invention provides an embodiment of an automated testing method for low-code applications based on machine learning, comprising the following steps:

[0076] S1: Receive test requirements for low-code applications and obtain the UI element diagrams of the low-code applications;

[0077] S2: Process the interface element diagram to extract the interface model of the low-code application;

[0078] S3: Based on the interface model, generate test cases, execute the test cases, and collect test result data;

[0079] S4: Analyze the test result data and generate a test report.

[0080] The specific steps of S2 are as follows:

[0081] S2.1: Convert the interface element image into an element grayscale image, and preprocess the element grayscale image, wherein the preprocessing includes image denoising and image enhancement;

[0082] S2.2: Extract frame matching features of interface element graphs using image processing technology, construct a frame recognition network, and use the frame recognition network to identify the frame matching features to obtain interface element categories, including buttons, checkboxes, menus, text boxes, and images;

[0083] S2.3: Extract text matching features from interface element graphs using semantic recognition technology, construct a text extraction network, and extract the text matching features through the text extraction network to obtain the meaning of the interface text.

[0084] The specific steps for extracting frame matching features from the interface element graph using image processing technology are as follows:

[0085] S2.2.1: Perform histogram statistics on the preprocessed element grayscale image, and use the grayscale value between the peak and valley between two peaks as the first segmentation threshold. Calculate the second segmentation threshold based on the probability density function of the element grayscale image. Obtain the optimal segmentation threshold by weighted summation of the first and second segmentation thresholds. Compare the element grayscale image pixels with the optimal segmentation threshold. If the pixel is greater than or equal to the optimal segmentation threshold, mark it as the target region. If it is less than the optimal segmentation threshold, mark it as the background region. Cut out the target region image of the element grayscale image.

[0086] S2.2.2: Extract corner features and gray-level centroids from the target region image. Calculate the frame descriptor based on the angle between the corner features and the gray-level centroids. Use an MLP network to transform the frame descriptor into high-dimensional data and add it to the gray-level centroids to generate frame matching features. The formula for calculating the frame descriptor is:

[0087]

[0088] Among them, t d This represents the frame descriptor, where 'a' represents the index of a single corner feature in the target region map, 'A' represents the total number of corner features in the target region map, and 'Ψ' represents the total number of corner features in the target region map. a β represents the angle between the feature of the a-th corner point and the gray centroid. a Let F(·) represent the weighted weight value of the feature of the a-th corner point, and let F(·) represent the Fourier transform function. c and y c c represents the x and y coordinates of the grayscale centroid. a c represents the feature of the a-th corner point of the target region map. m This represents the average value of the corner features of the target region map;

[0089] The formula for calculating frame matching features is:

[0090]

[0091] Where i represents the frame matching feature number, T (G) d represents a high-dimensional feature vector. i and G i Let represent the gray value of the i-th gray centroid and the frame descriptor, respectively, and MLP(·) represent the multilayer perceptron network layer;

[0092] Multilayer perceptrons are used to increase the dimensionality of frame descriptors, adding the gray-level centroids and frame descriptors to form a feature vector. Each frame descriptor incorporates its coordinates and confidence score. The multilayer perceptron embeds the feature points into the high-dimensional vector. First, a five-layer fully connected perceptron is used to encode the frame descriptor G. i When converted to 256-dimensional data, the number of channels in the five-layer perceptron are 3, 32, 64, 128, and 256, respectively.

[0093] Please see Figure 2 A schematic diagram of the corner feature determination region in an embodiment of the present invention is shown. The specific steps for corner feature extraction are as follows:

[0094] S2.2.2.1: Randomly select any pixel in the target area image, mark it as point P, and calculate the gray value I of P. P ;

[0095] S2.2.2.2: Draw a circle with P as the center and a radius of 4 pixels, and select 20 pixels on the boundary of the circle;

[0096] S2.2.2.3: Set a grayscale threshold t. If the grayscale values ​​of 10 consecutive pixels out of 20 pixels are all less than I... P -t or greater than I P If +t, then point P is determined to be a corner feature of the image;

[0097] The method for extracting the gray-scale centroid includes: first, obtaining regions with significant features in the image, calculating the image salient center, second, setting a difference threshold, calculating the difference between the gray-scale values ​​of the salient center and each neighboring pixel, summing the differences and comparing them with the difference threshold, and if the sum of the differences between the salient center and each neighboring pixel is greater than the difference threshold, then the salient center is marked as the gray-scale centroid.

[0098] Please see Figure 3 The framework recognition network structure diagram of this invention embodiment, wherein constructing the framework recognition network includes:

[0099] The input layer is used to decode the matching features of the framework through an attention mechanism to obtain the input feature sequence, and to perform logarithmic operations and size reconstruction on the input feature sequence to obtain the input parameters of the hidden layer.

[0100] Hidden layers are used to calculate the similarity between the input parameters of the hidden layers and the typical frame element dataset through a normalized embedded Gaussian function. The similarity is enhanced and classified through adjacency matrix and mask to obtain the classification probability of the input parameters of each hidden layer.

[0101] The voting decision layer is used to vote on the category of the hidden layer input parameters based on classification probabilities. The category of the hidden layer input parameter with the most votes is used as the category of the interface element. If there are ties, the probability value of the vote is added as an additional weight to recalculate the vote. The specific voting method is as follows:

[0102] Find the largest classification probability in the probability sequence of the hidden layer input parameters, vote on the category to which the largest classification probability belongs, and at the same time re-sort the remaining classification probabilities of the input parameters. Take the largest classification probability among the remaining classification probabilities and compare it with the largest classification probability. If the difference between the two probabilities is less than the comparison threshold, vote on the category corresponding to the largest classification probability among the remaining classification probabilities. The comparison threshold is determined by those skilled in the art based on a large number of experiments.

[0103] The specific steps for extracting text matching features from interface element graphs using semantic recognition technology are as follows:

[0104] S2.3.1: Adaptive binarization is performed on the preprocessed element grayscale image to obtain a binarized element image. By edge detection, continuous curves in the binarized element image are extracted. Lines are fitted to the curves to obtain straight lines and vanishing lines in the binarized element image. The intersection area of ​​the vanishing lines and the straight lines is used as the text area.

[0105] S2.3.2: Perform vertical projection on the text region. Divide the text region into row and column boundaries based on the maximum and minimum values ​​of the vertical projection function. Starting from the position of the maximum value of the vertical projection function, find the position where the maximum value of the vertical integral projection function is half of the maximum value in both forward and backward directions. Use this position as the row boundary of the text region. Starting from the position of the minimum value of the vertical projection function, find the position where the minimum value of the vertical integral projection function is two-thirds of the minimum value in both forward and backward directions. Use this position as the column boundary of the text region. Match the function values ​​at the row and column boundaries with the function values ​​of the text region to obtain the text content vector.

[0106] S2.3.3: Calculate the gray-level co-occurrence matrix of the text content vector. The gray-level co-occurrence matrix includes energy features, contrast features, entropy features, and correlation features. Obtain the frequency distribution function of the text content vector through Fourier transform. Calculate the text matching features based on the gray-level co-occurrence matrix and the frequency distribution function. The text matching features include contour features, geometric features, and structural features. The calculation formula for the text matching features is as follows:

[0107]

[0108] Among them, T f Let E{·} represent the text matching features, E{·} represent the annealing genetic function, and K represent the text matching features. bLet F represent the Boltzmann constant, G represent the gray-level co-occurrence matrix, F represent the frequency distribution function, n represent the unit vector of the text content vector, N represent the total length of the text content vector, and f represent the frequency distribution function. n X represents the frequency of the nth text content vector. n This represents the magnitude of the nth text content vector;

[0109] Please see Figure 4 The text extraction network structure diagram of this embodiment of the invention includes:

[0110] The input layer takes the text matching features as input parameters, normalizes and matrix quantizes the text matching features, and transforms them into a 64×64 feature matrix.

[0111] The convolutional pooling layer performs four stages to convolve the input 64×64 feature matrix through a 3×3 convolution kernel, and then uses the ReLU activation function and max pooling operation to obtain a 2×2 mapping feature. The max pooling operation includes a 2×2 pooling kernel.

[0112] The output layer flattens the mapped features into a compact feature vector to obtain a text descriptor. It calculates the semantic similarity between the text descriptor and the text semantic library through character pattern matching, and determines whether the convergence condition is met. If the convergence condition is not met, the semantic similarity error is backpropagated through the loss function. If the convergence condition is met, the semantic meaning of the text corresponding to the highest semantic similarity is output as the meaning of the interface text.

[0113] The generated test cases are used to automatically generate test scripts based on the element types, location information, and semantic information in the interface model, including identifying elements, determining element interactions, and converting formats;

[0114] The identification element is used to identify the type of each element in the interface model;

[0115] The determination of element interaction is used to determine the operation method and expected result of each element, and to generate an interaction scenario test script covering all exchangeable elements based on the interaction relationship between elements.

[0116] The conversion format is used to convert test scripts into a format executable by automated testing tools, including:

[0117] a. Configure the automated testing environment, including installing the required browser and testing tools;

[0118] b. Initialize the testing tools and load the test scripts;

[0119] c. Execute the test script to simulate various user operations on the application interface;

[0120] d. Monitor test execution in real time, capturing application responses and interface changes;

[0121] e. Record screenshots, videos, and log files during the testing process.

[0122] The collection of test result data includes obtaining screenshots, log files, and error reports during the test process, specifically:

[0123] a. Extract screenshots and videos generated during the execution process from automated testing tools;

[0124] b. Collect log files generated by applications and testing tools, which record the detailed process and timeline of test execution;

[0125] c. If an error occurs, obtain an error report, which includes a detailed description of the error, its location, and the cause.

[0126] The specific steps for analyzing the test result data are as follows:

[0127] S4.1: Identify abnormal patterns and trends during the testing process using cluster analysis algorithms;

[0128] S4.2: Identify the causes and error paths in the testing process using decision trees;

[0129] S4.3: Statistical analysis of test pass rate and test coverage;

[0130] The test pass rate is the ratio of the number of test cases that were successfully executed without errors to the total number of test cases;

[0131] The test coverage rate is the ratio of the code or functionalities covered by the test cases to the total code or functionalities of the application.

[0132] Example 2:

[0133] Please see Figure 5 The present invention provides an embodiment of a low-code application automatic testing system based on machine learning, the system comprising an element extraction module, a test case generation module, a test case execution module, a data analysis module, and a data reporting module;

[0134] The element extraction module is used to receive the test requirements of the low-code application, obtain the interface element diagram of the low-code application, and extract the interface model of the low-code application.

[0135] The test case generation module is used to automatically generate test scripts based on the element types, location information, and semantic information in the interface model.

[0136] The test case execution module is used to convert test scripts into a format executable by automated testing tools, execute the test scripts through automated testing tools, and collect test result data.

[0137] The data analysis module is used to analyze the collected test result data;

[0138] The data reporting module is used to generate test reports.

[0139] The element extraction module includes:

[0140] Image preprocessing unit: Responsible for receiving the interface element images of low-code applications and converting them into grayscale images. This unit performs preprocessing operations such as image denoising and enhancement to improve the accuracy of subsequent feature extraction.

[0141] Feature extraction unit: Uses image processing techniques to identify and extract frame features and text features of interface elements;

[0142] Frame recognition unit: A network trained using a machine learning model, used for high-precision identification of interface element categories;

[0143] Text extraction unit: A network trained through a machine learning model, specifically designed to understand and extract text information from the interface, including the semantic meaning of the text;

[0144] The test case generation module includes:

[0145] Interface Model Analysis Unit: Based on the element types, positions, and semantic information in the interface model, plan the logic and content of test cases;

[0146] Test script generation unit: Automatically generates test scripts based on the interface model parsing results and determines the interaction methods between elements;

[0147] The test case execution module includes:

[0148] Test script conversion unit: Converts test scripts into a format executable by automated testing tools, ensuring compatibility with various testing platforms;

[0149] Test Execution Unit: Responsible for executing test scripts using automated testing tools;

[0150] Data collection unit: Collects data such as screenshots, log files, and error reports generated during the testing process;

[0151] The data analysis module includes:

[0152] Data Analysis Unit: Uses algorithms such as clustering analysis and decision trees to conduct in-depth analysis of the collected test results data, and identify abnormal patterns and error paths.

[0153] Statistical Evaluation Unit: Responsible for statistical testing coverage and pass rate, as well as other key performance indicators.

[0154] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for automatic testing of low-code applications based on machine learning, characterized in that, The method comprises the following steps: S1: receiving test requirements of a low-code application, and obtaining an interface element graph of the low-code application; S2: processing the interface element graph, and extracting an interface model of the low-code application; S3: generating a test case according to the interface model, executing the test case, and collecting test result data; S4: analyzing the test result data, and generating a test report; The specific steps of S2 are as follows: S2.1: converting the interface element graph into an element grayscale graph, and preprocessing the element grayscale graph; S2.2: extracting frame matching features of the interface element graph through image processing technology, constructing a frame recognition network, recognizing the frame matching features through the frame recognition network, and obtaining interface element categories; S2.3: extracting text matching features of the interface element graph through semantic recognition technology, constructing a text extraction network, extracting the text matching features through the text extraction network, and obtaining interface text meanings; S2.4: generating an interface model of the low-code application according to the interface element categories and the interface text meanings; The specific steps of extracting the frame matching features of the interface element graph through the image processing technology are as follows: S2.2.1: converting the interface element graph into an element grayscale graph, and preprocessing the element grayscale graph, wherein the preprocessing comprises image denoising and image enhancement; S2.2.2: performing histogram statistics on the preprocessed element grayscale graph, and cutting out a target region graph of the element grayscale graph through a statistical learning method; S2.2.3: extracting corner point features and grayscale centroids of the target region graph, calculating a frame descriptor according to an included angle between the corner point features and the grayscale centroids, converting the frame descriptor into high-dimensional data by using an MLP network, adding the high-dimensional data to the grayscale centroids, and generating the frame matching features; The specific steps of extracting the text matching features of the interface element graph through the semantic recognition technology are as follows: S2.3.1: processing the preprocessed element grayscale graph through edge detection and adaptive binarization, and extracting a text region; S2.3.2: performing vertical projection on the text region, dividing row boundaries and column boundaries of the text region according to extreme values of a vertical projection function, matching function values at the row boundaries and the column boundaries with function values of the text region, and obtaining a text content vector; S2.3.3: calculating a gray level co-occurrence matrix of the text content vector, obtaining a frequency distribution function of the text content vector through Fourier transform, and calculating the text matching features according to the gray level co-occurrence matrix and the frequency distribution function; The specific steps of analyzing the test result data are as follows: S4.1: identifying abnormal patterns and abnormal trends in a test process through a clustering analysis algorithm; S4.2: identifying error causes and error paths in the test process through a decision tree; S4.3: calculating a test pass rate and a test coverage rate.

2. The method of claim 1, wherein, The frame recognition network comprises: an input layer, configured to decode the frame matching features through an attention mechanism, and obtain input parameters of a hidden layer; a hidden layer, configured to calculate similarities between the input parameters of the hidden layer and a typical frame element data set, and obtain classification probabilities of the input parameters of each hidden layer; A voting decision layer is configured to vote for the category of the hidden layer input parameter according to the classification probability, and the category of the hidden layer input parameter with the highest number of votes is taken as the category of the interface element.

3. The method of claim 1, wherein the method further comprises: The text extraction network comprises: An input layer configured to take the text matching features as input parameters, normalize and matrix quantize the text matching features, and convert the text matching features into a 64*64 size feature matrix; A convolutional pooling layer configured to convolve the input 64*64 size feature matrix through a 3*3 size convolution kernel, and obtain a 2*2 size mapping feature through a RELU activation function and a maximum pooling operation, wherein the maximum pooling operation comprises a 2*2 size pooling kernel; An output layer configured to flatten the mapping feature into a compact feature vector to obtain a literal description sub, calculate the semantic similarity between the literal description sub and a literal semantic library through character pattern matching, determine whether a convergence condition is met, perform error back propagation on the semantic similarity through a loss function if the convergence condition is not met, and output the literal semantic corresponding to the highest semantic similarity as the interface text meaning if the convergence condition is met.

4. The method of claim 1, wherein the method further comprises: The generated test case is configured to automatically generate a test script according to the element type, position information and semantic information in the interface model, and the collection of test result data comprises obtaining screenshots, log files and error reports in the test process.

5. A machine learning based low code application automatic testing system implemented based on the method of any one of claims 1-4, characterized in that, The system comprises an element extraction module, a use case generation module, a use case execution module, a data analysis module and a data reporting module; The element extraction module is configured to receive a test requirement of a low-code application, obtain an interface element graph of the low-code application, and extract an interface model of the low-code application; The use case generation module is configured to automatically generate a test script according to the element type, position information and semantic information in the interface model; The use case execution module is configured to convert the test script into a format executable by an automated test tool, execute the test script through the automated test tool, and collect test result data; The data analysis module is configured to analyze the collected test result data; The data reporting module is configured to generate a test report.

Images