Low-code application automatic test system and method based on machine learning

Abstract

The invention relates to a low-code application automatic test system and method based on machine learning, and the method comprises the following key steps: firstly, receiving a test demand of a low-code application, and obtaining an interface element graph; secondly, processing the interface element graph through an image processing and semantic recognition technology, and extracting an interface model; then, automatically generating and executing a test case according to the interface model, and collecting test result data; and finally, analyzing test result data and generating a detailed test report. The invention further relates to construction and use of a framework recognition network and a text extraction network to accurately recognize the category of interface elements and text meanings, and test data are deeply analyzed through clustering analysis and a decision tree algorithm, so that abnormal modes, error reasons and paths are effectively recognized. The method can be widely applied to the field of software development, in particular to application testing of a low-code platform, and the software quality and the testing efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of low-code application technology, and in particular to a low-code application automatic testing system and method based on machine learning. Background Art

[0002] In the current rapidly developing software industry, the development model of low-code applications has received widespread attention. Low-code platforms simplify programming work, allowing non-professional programmers to easily build applications, greatly improving development efficiency and rapid application iteration. However, with the popularity of low-code applications, the complexity and challenges of their testing process have become increasingly prominent. Traditional software testing methods often rely on manual writing of test cases, which is not only time-consuming and labor-intensive, but also when dealing with low-code applications, due to their rapidly changing and diverse interface elements, the efficiency and accuracy of traditional methods are often unsatisfactory.

[0003] Although existing automated testing technologies, such as testing methods based on image recognition, can improve testing efficiency to a certain extent, they still face many challenges when dealing with low-code applications. The interface elements and layouts of low-code applications may change frequently, which requires the testing system to be able to quickly adapt to these changes and accurately identify new elements and layouts. In addition, automated testing tools often have difficulty effectively handling complex interactions and data validation issues in low-code applications. For example, when an application involves complex data logic or specific user interaction behaviors, automated testing scripts may need to be updated frequently, which not only increases maintenance costs, but also reduces the flexibility and efficiency of testing.

[0004] For example, a Chinese patent application numbered CN114238110A discloses a software application testing method, device, equipment, and storage medium. The invention obtains the code to be tested and the test case corresponding to the target software application; executes the test script corresponding to the test case in the breakpoint test mode, and obtains the script execution result and the effective execution code after the test script is executed; if the script execution result is inconsistent with the predicted result corresponding to the test case, the test code section corresponding to the test case in the code to be tested is obtained; the code coverage is determined based on the effective execution code and the test code section, and the abnormal code location is determined based on the code coverage. Since the test script is executed in the breakpoint test mode, the effective execution code can be accurately obtained. When the script execution result is inconsistent with the predicted result, the abnormal code location can be quickly located based on the compared code coverage, which reduces the difficulty for developers to locate the abnormal code location.

[0005] The above patents all have the problems raised by this background technology: when the interface elements, layout or interaction logic changes, the test script needs to be manually updated, which is not only time-consuming but also reduces the test efficiency. The recognition accuracy of interface elements is insufficient. Existing automated testing tools have limited capabilities when simulating these complex scenarios and cannot fully cover all possible test scenarios. To solve the above problems, this application designs a low-code application automatic testing system and method based on machine learning. Summary of the invention

[0006] The technical problem to be solved by the present invention is to address the shortcomings of the existing technology and provide a low-code application automatic testing system and method based on machine learning. First, the test requirements of the low-code application are received and the interface element diagram is obtained; secondly, the interface element diagram is processed by image processing and semantic recognition technology to extract the interface model; then, test cases are automatically generated and executed according to the interface model, and test result data is collected; finally, the test result data is analyzed and a detailed test report is generated.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The low-code application automatic testing method based on machine learning includes the following steps:

[0009] S1: Receive the test requirements of the low-code application and obtain the interface element diagram of the low-code application;

[0010] S2: Process the interface element diagram to extract the interface model of the low-code application;

[0011] S3: Generate test cases according to the interface model, execute the test cases, and collect test result data;

[0012] S4: Analyze the test result data and generate a test report.

[0013] The specific steps of S2 are as follows:

[0014] S2.1: converting the interface element image into an element grayscale image and preprocessing the element grayscale image;

[0015] S2.2: extracting frame matching features of the interface element graph by image processing technology, constructing a frame recognition network, and identifying the frame matching features by the frame recognition network to obtain the interface element category;

[0016] S2.3: extracting text matching features of the interface element graph by using semantic recognition technology, constructing a text extraction network, extracting the text matching features by using the text extraction network, and obtaining the meaning of the interface text;

[0017] S2.4: Generate an interface model of the low-code application based on the interface element category and the interface text meaning.

[0018] The specific steps of extracting the framework matching features of the interface element graph by image processing technology are as follows:

[0019] S2.2.1: converting the interface element image into an element grayscale image, and preprocessing the element grayscale image, wherein the preprocessing includes image denoising and image enhancement;

[0020] S2.2.2: Perform histogram statistics on the preprocessed element grayscale image, and cut out the target area map of the element grayscale image through statistical learning method;

[0021] S2.2.3: Extract the corner features and grayscale centroid of the target area map, calculate the frame descriptor based on the angle between the corner features and the grayscale centroid, use the MLP network to convert the frame descriptor into high-dimensional data, and add it to the grayscale centroid to generate the frame matching feature. The calculation formula of the frame descriptor is:

[0022]

[0023] Among them, t d represents the frame descriptor, a represents the serial number of a single corner feature of the target area map, A represents the total number of corner features of the target area map, Ψ a represents the angle between the ath corner feature and the grayscale centroid, β a represents the weighted value of the feature of the ath corner point, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c a represents the ath corner feature of the target area map, c m Represents the average value of the corner features of the target area map.

[0024] The construction framework identification network includes:

[0025] An input layer, used to decode the frame matching features through an attention mechanism to obtain input parameters of a hidden layer;

[0026] Hidden layer, used to calculate the similarity between the input parameters of the hidden layer and the typical framework element data set, and obtain the classification probability of the input parameters of each hidden layer;

[0027] The voting decision layer is used to vote on the categories of the hidden layer input parameters according to the classification probability, and the hidden layer input parameter category with the most votes is counted as the interface element category.

[0028] The specific steps of extracting text matching features of the interface element graph by semantic recognition technology are as follows:

[0029] S2.3.1: Process the preprocessed element grayscale image through edge detection and adaptive binarization to extract the text area;

[0030] S2.3.2: Perform vertical projection on the text area, divide the row boundary and column boundary of the text area according to the maximum value of the vertical projection function, match the function value at the row boundary and column boundary with the function value of the text area, and obtain the text content vector;

[0031] S2.3.3: Calculate the gray-level co-occurrence matrix of the text content vector, obtain the frequency distribution function of the text content vector through Fourier transform, and calculate the text matching feature based on the gray-level co-occurrence matrix and the frequency distribution function. The calculation formula of the text matching feature is:

[0032]

[0033] Among them, T f represents the text matching feature, E{·} represents the annealing genetic function, K b represents the Boltzmann constant, G represents the gray-level co-occurrence matrix, F represents the frequency distribution function, n represents the unit vector of the text content vector, N represents the total length of the text content vector, and f n represents the frequency of the nth text content vector, X n Represents the magnitude of the nth text content vector.

[0034] The text extraction network comprises:

[0035] The input layer takes the text matching features as input parameters, normalizes and matrix quantizes the text matching features, and converts them into a feature matrix of size 64×64;

[0036] The convolution pooling layer convolves the input 64×64 feature matrix with a 3×3 convolution kernel in four stages, and obtains a 2×2 mapping feature through a RELU activation function and a maximum pooling operation, wherein the maximum pooling operation includes a 2×2 pooling kernel;

[0037] The output layer flattens the mapping features into a compact feature vector to obtain the text descriptor. The semantic similarity between the text descriptor and the text semantic library is calculated through character pattern matching to determine whether the convergence condition is met. If the convergence condition is not met, the semantic similarity is back-propagated through the loss function. If the convergence condition is met, the text semantics corresponding to the highest semantic similarity is output as the interface text meaning.

[0038] The generated test cases are used to automatically generate test scripts according to the element types, location information and semantic information in the interface model, including identifying elements, determining element interactions and conversion formats, and the collected test result data includes obtaining screenshots, log files and error reports during the test process.

[0039] The specific steps of analyzing the test result data are as follows:

[0040] S4.1: Identify abnormal patterns and trends during testing using cluster analysis algorithms;

[0041] S4.2: Identify the causes of errors and error paths during testing through decision trees;

[0042] S4.3: Count test pass rate and test coverage.

[0043] A low-code application automatic testing system based on machine learning, the system comprising an element extraction module, a use case generation module, a use case execution module, a data analysis module and a data reporting module;

[0044] The element extraction module is used to receive the test requirements of the low-code application, obtain the interface element diagram of the low-code application, and extract the interface model of the low-code application;

[0045] The use case generation module is used to automatically generate a test script according to the element type, position information and semantic information in the interface model;

[0046] The test case execution module is used to convert the test script into a format executable by the automated testing tool, execute the test script through the automated testing tool, and collect test result data;

[0047] The data analysis module is used to analyze the collected test result data;

[0048] The data reporting module is used to generate a test report;

[0049] The element extraction module comprises:

[0050] Image preprocessing unit: responsible for receiving the interface element images of low-code applications, converting them into grayscale images, and performing preprocessing operations such as image denoising and enhancement to improve the accuracy of subsequent feature extraction;

[0051] Feature extraction unit: uses image processing technology to identify and extract frame features and text features of interface elements;

[0052] Frame recognition unit: A network trained by a machine learning model, used to identify interface element categories with high accuracy;

[0053] Text extraction unit: A network trained by a machine learning model, which is specifically used to understand and extract text information on the interface, including the semantic meaning of the text;

[0054] The use case generation module includes:

[0055] Interface model parsing unit: plans the logic and content of test cases based on the element type, position and semantic information in the interface model;

[0056] Test script generation unit: automatically generates test scripts based on the interface model analysis results and determines the interaction method between elements;

[0057] The use case execution module includes:

[0058] Test script conversion unit: converts test scripts into a format executable by automated testing tools to ensure compatibility with various test platforms;

[0059] Test execution unit: responsible for executing test scripts using automated testing tools;

[0060] Data collection unit: used to collect data such as screenshots, log files, and error reports generated during the test;

[0061] The data analysis module comprises:

[0062] Data Analysis Unit: Use cluster analysis, decision tree and other algorithms to conduct in-depth analysis of the collected test result data to identify abnormal patterns and error paths.

[0063] Statistical Evaluation Unit: Responsible for statistical test coverage and pass rate, as well as other key performance indicators.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. By receiving the test requirements of low-code applications and automatically extracting interface models, the present invention can automatically generate and execute test cases, significantly improve test efficiency, and reduce human resource investment;

[0066] 2. The present invention utilizes image processing and machine learning techniques to accurately identify the framework features and text features of the interface element diagram, and then accurately extract the interface model, thereby improving the accuracy and reliability of the test. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0068] Figure 1This is a flowchart of a low-code application automatic testing method based on machine learning according to Embodiment 1 of the present invention;

[0069] Figure 2 This is a schematic diagram of a corner feature determination area according to Embodiment 1 of the present invention;

[0070] Figure 3 This is a network structure diagram of the framework identification of Example 1 of the present invention;

[0071] Figure 4 This is a text extraction network structure diagram of Example 1 of the present invention;

[0072] Figure 5 This is a module diagram of a low-code application automatic testing system based on machine learning according to Example 2 of the present invention. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0074] Embodiment 1:

[0075] See also Figure 1 , an embodiment provided by the present invention: a low-code application automatic testing method based on machine learning, comprising the following steps:

[0076] S1: Receive the test requirements of the low-code application and obtain the interface element diagram of the low-code application;

[0077] S2: Process the interface element diagram to extract the interface model of the low-code application;

[0078] S3: Generate test cases according to the interface model, execute the test cases, and collect test result data;

[0079] S4: Analyze the test result data and generate a test report.

[0080] The specific steps of S2 are as follows:

[0081] S2.1: converting the interface element image into an element grayscale image, and preprocessing the element grayscale image, wherein the preprocessing includes image denoising and image enhancement;

[0082] S2.2: extracting frame matching features of the interface element graph by image processing technology, constructing a frame recognition network, identifying the frame matching features by the frame recognition network, and obtaining interface element categories, wherein the interface element categories include buttons, check boxes, menus, text boxes, and pictures;

[0083] S2.3: Extract text matching features of the interface element diagram through semantic recognition technology, build a text extraction network, extract the text matching features through the text extraction network, and obtain the meaning of the interface text.

[0084] The specific steps of extracting the framework matching features of the interface element graph by image processing technology are as follows:

[0085] S2.2.1: Perform histogram statistics on the preprocessed element grayscale image, take the peak-to-valley grayscale value between the two peaks as the segmentation threshold 1, calculate the segmentation threshold 2 according to the probability density function of the element grayscale image, obtain the optimal segmentation threshold by weighted summation of the segmentation threshold 1 and the segmentation threshold 2, compare the pixel point of the element grayscale image with the optimal segmentation threshold, if it is greater than or equal to the optimal segmentation threshold, mark the pixel point as the target area, if it is less than the optimal segmentation threshold, mark the pixel point as the background area, and cut out the target area map of the element grayscale image;

[0086] S2.2.2: Extract the corner features and grayscale centroid of the target area map, calculate the frame descriptor based on the angle between the corner features and the grayscale centroid, use the MLP network to convert the frame descriptor into high-dimensional data, and add it to the grayscale centroid to generate the frame matching feature. The calculation formula of the frame descriptor is:

[0087]

[0088] Among them, t d represents the frame descriptor, a represents the serial number of a single corner feature of the target area map, A represents the total number of corner features of the target area map, Ψ a represents the angle between the ath corner feature and the grayscale centroid, β a represents the weighted value of the feature of the ath corner point, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c a represents the ath corner feature of the target area map, c m Represents the average value of the corner features of the target area map;

[0089] The calculation formula of the frame matching feature is:

[0090]

[0091] Where i represents the frame matching feature number, T (G) represents a high-dimensional feature vector, d i and G i denote the grayscale value of the i-th grayscale centroid and the frame descriptor respectively, and MLP(·) denotes the multi-layer perceptron network layer;

[0092] The multi-layer perceptron is used to increase the dimension of the frame descriptor, so that the grayscale centroid and the frame descriptor are added to form a feature vector. Each frame descriptor is combined with its coordinates and confidence. The multi-layer perceptron is used to embed the feature points into the high-dimensional vector. First, the frame descriptor G is transformed into i Converted into 256-dimensional data, the number of channels of the five-layer perceptron are 3, 32, 64, 128 and 256 respectively.

[0093] See also Figure 2 , a schematic diagram of a corner feature judgment area according to an embodiment of the present invention, wherein the specific steps of extracting corner features are as follows:

[0094] S2.2.2.1: Randomly select any pixel in the target area map, mark it as point P, and calculate the gray value I of P P ;

[0095] S2.2.2.2: Draw a circle with P as the center and 4 pixels as the radius, and select 20 pixels on the edge of the circle;

[0096] S2.2.2.3: Set the grayscale threshold t. If the grayscale values ​​of 10 consecutive pixels out of 20 pixels are less than I P -t or greater than I P +t, then point P is determined to be a corner feature of the image;

[0097] The grayscale centroid extraction method includes: first obtaining an area with significant features in the image, calculating the significant center of the image, and then setting a difference threshold, calculating the difference between the grayscale value of the significant center and each pixel in the neighborhood, and comparing the accumulated difference with the difference threshold. If the sum of the accumulated grayscale value differences between the significant center and each pixel in the neighborhood is greater than the difference threshold, the significant center is marked as the grayscale centroid.

[0098] See also Figure 3 , the framework recognition network structure diagram of the embodiment of the present invention, the construction of the framework recognition network includes:

[0099] An input layer, used for decoding the frame matching features through an attention mechanism, obtaining an input feature sequence, performing logarithmic operations and resizing the input feature sequence, and obtaining input parameters of a hidden layer;

[0100] Hidden layer, used to calculate the similarity between the input parameters of the hidden layer and the typical framework element data set through the normalized embedded Gaussian function, enhance and classify the similarity through the adjacency matrix and mask, and obtain the classification probability of the input parameters of each hidden layer;

[0101] The voting decision layer is used to vote on the category of the hidden layer input parameters according to the classification probability. The hidden layer input parameter category with the most votes is counted as the interface element category. If the number of votes is the same, the probability value of the vote is added as an additional weight to recalculate the number of votes. The specific voting method is as follows:

[0102] Find the largest classification probability in the probability sequence of the hidden layer input parameters, vote for the category to which the largest classification probability belongs, and re-sort the remaining classification probabilities of the input parameters, take the largest classification probability among the remaining classification probabilities, and compare it with the largest classification probability. If the difference between the two probabilities is less than the comparison threshold, vote for the category corresponding to the largest classification probability among the remaining classification probabilities. The comparison threshold is determined by technicians in this field based on a large number of experiments.

[0103] The specific steps of extracting text matching features of the interface element graph by semantic recognition technology are as follows:

[0104] S2.3.1: Adaptively binarize the preprocessed element grayscale image to obtain a binarized element image. Extract the continuous curve in the binarized element image through edge detection. Perform straight line fitting on the curve to obtain the straight line and significant vanishing line in the binarized element image. The intersection area of ​​the significant vanishing line and the straight line is taken as the text area.

[0105] S2.3.2: Perform vertical projection on the text area, divide the row boundary and column boundary of the text area according to the maximum value of the vertical projection function, search for the position of half the maximum value of the vertical integral projection function value in the forward and backward directions starting from the maximum value position of the vertical projection function, and use it as the row boundary of the text area; search for the position of two-thirds of the minimum value of the vertical integral projection function value in the forward and backward directions starting from the minimum value position of the vertical projection function, and use it as the column boundary of the text area; match the function values ​​at the row boundary and column boundary with the function value of the text area to obtain the text content vector;

[0106] S2.3.3: Calculate the gray level co-occurrence matrix of the text content vector, the gray level co-occurrence matrix includes energy features, contrast features, entropy features and correlation features, obtain the frequency distribution function of the text content vector through Fourier transform, calculate the text matching features according to the gray level co-occurrence matrix and the frequency distribution function, the text matching features include contour features, geometric features and structural features, and the calculation formula of the text matching features is:

[0107]

[0108] Among them, T f represents the text matching feature, E{·} represents the annealing genetic function, K brepresents the Boltzmann constant, G represents the gray-level co-occurrence matrix, F represents the frequency distribution function, n represents the unit vector of the text content vector, N represents the total length of the text content vector, and f n represents the frequency of the nth text content vector, X n represents the magnitude of the nth text content vector;

[0109] See also Figure 4 , a text extraction network structure diagram of an embodiment of the present invention, wherein the text extraction network comprises:

[0110] The input layer takes the text matching features as input parameters, normalizes and matrix quantizes the text matching features, and converts them into a feature matrix of size 64×64;

[0111] The convolution pooling layer convolves the input 64×64 feature matrix with a 3×3 convolution kernel in four stages, and obtains a 2×2 mapping feature through a RELU activation function and a maximum pooling operation, wherein the maximum pooling operation includes a 2×2 pooling kernel;

[0112] The output layer flattens the mapping features into a compact feature vector to obtain the text descriptor. The semantic similarity between the text descriptor and the text semantic library is calculated through character pattern matching to determine whether the convergence condition is met. If the convergence condition is not met, the semantic similarity is back-propagated through the loss function. If the convergence condition is met, the text semantics corresponding to the highest semantic similarity is output as the interface text meaning.

[0113] The test case generation is used to automatically generate a test script according to the element type, location information and semantic information in the interface model, including identifying elements, determining element interactions and conversion formats;

[0114] The identification element is used to identify the type of each element in the interface model;

[0115] The determining element interaction is used to determine the operation method and expected result of each element, and generate an interaction scenario test script covering all exchangeable elements according to the interaction relationship between the elements;

[0116] The conversion format is used to convert the test script into a format executable by the automated testing tool, including:

[0117] a. Configure the automated testing environment, including installing the required browsers and testing tools;

[0118] b. Initialize the test tool and load the test script;

[0119] c. Execute the test script to simulate various operations performed by users on the application interface;

[0120] d. Monitor test execution in real time and capture application responses and interface changes;

[0121] e. Record screenshots, videos, and log files during the test.

[0122] The collection of test result data includes obtaining screenshots, log files and error reports during the test process, specifically:

[0123] a. Extract screenshots and videos generated during the operation of the automated testing tool;

[0124] b. Collect log files generated by applications and test tools, which record the detailed process and timeline of test execution;

[0125] c. If an error occurs, obtain an error report that contains a detailed description of the error, where it occurred, and why it occurred.

[0126] The specific steps of analyzing the test result data are as follows:

[0127] S4.1: Identify abnormal patterns and trends during testing using cluster analysis algorithms;

[0128] S4.2: Identify the causes of errors and error paths during testing through decision trees;

[0129] S4.3: Statistics of test pass rate and test coverage;

[0130] The test pass rate is the ratio of the number of test cases that are successfully executed without errors to the total number of test cases;

[0131] The test coverage is the ratio of the code or function points covered by the test cases to the total code or function points of the application.

[0132] Embodiment 2:

[0133] See also Figure 5 , the present invention provides an embodiment: a low-code application automatic testing system based on machine learning, the system comprising an element extraction module, a use case generation module, a use case execution module, a data analysis module and a data reporting module;

[0134] The element extraction module is used to receive the test requirements of the low-code application, obtain the interface element diagram of the low-code application, and extract the interface model of the low-code application;

[0135] The use case generation module is used to automatically generate a test script according to the element type, position information and semantic information in the interface model;

[0136] The test case execution module is used to convert the test script into a format executable by the automated testing tool, execute the test script through the automated testing tool, and collect test result data;

[0137] The data analysis module is used to analyze the collected test result data;

[0138] The data reporting module is used to generate a test report.

[0139] The element extraction module comprises:

[0140] Image preprocessing unit: responsible for receiving the interface element images of low-code applications and converting them into grayscale images. This unit performs preprocessing operations such as image denoising and enhancement to improve the accuracy of subsequent feature extraction;

[0141] Feature extraction unit: uses image processing technology to identify and extract frame features and text features of interface elements;

[0142] Frame recognition unit: A network trained by a machine learning model, used to identify interface element categories with high accuracy;

[0143] Text extraction unit: A network trained by a machine learning model, which is specifically used to understand and extract text information on the interface, including the semantic meaning of the text;

[0144] The use case generation module includes:

[0145] Interface model parsing unit: plans the logic and content of test cases based on the element type, position and semantic information in the interface model;

[0146] Test script generation unit: automatically generates test scripts based on the interface model analysis results and determines the interaction method between elements;

[0147] The use case execution module includes:

[0148] Test script conversion unit: converts test scripts into a format executable by automated testing tools to ensure compatibility with various test platforms;

[0149] Test execution unit: responsible for executing test scripts using automated testing tools;

[0150] Data collection unit: collects data such as screenshots, log files, and error reports generated during the test;

[0151] The data analysis module comprises:

[0152] Data Analysis Unit: Use cluster analysis, decision tree and other algorithms to conduct in-depth analysis of the collected test result data to identify abnormal patterns and error paths.

[0153] Statistical Evaluation Unit: Responsible for statistical test coverage and pass rate, as well as other key performance indicators.

[0154] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A low-code application automatic testing method based on machine learning, characterized in that: The following steps are involved: S1: Receive the test requirements of the low-code application and obtain the interface element diagram of the low-code application; S2: Process the interface element diagram to extract the interface model of the low-code application; S3: Generate test cases according to the interface model, execute the test cases, and collect test result data; S4: Analyze the test result data and generate a test report.

2. According to the low-code application automatic testing method based on machine learning in claim 1, it is characterized in that: The specific steps of S2 are as follows: S2.1: converting the interface element image into an element grayscale image and preprocessing the element grayscale image; S2.2: extracting frame matching features of the interface element graph by image processing technology, constructing a frame recognition network, and identifying the frame matching features by the frame recognition network to obtain the interface element category; S2.3: extracting text matching features of the interface element graph by using semantic recognition technology, constructing a text extraction network, extracting the text matching features by using the text extraction network, and obtaining the meaning of the interface text; S2.4: Generate an interface model of the low-code application based on the interface element category and the interface text meaning.

3. According to the low-code application automatic testing method based on machine learning according to claim 2, it is characterized in that: The specific steps of extracting the framework matching features of the interface element graph by image processing technology are as follows: S2.2.1: converting the interface element image into an element grayscale image, and preprocessing the element grayscale image, wherein the preprocessing includes image denoising and image enhancement; S2.2.2: Perform histogram statistics on the preprocessed element grayscale image, and cut out the target area map of the element grayscale image through statistical learning method; S2.2.3: Extract the corner features and grayscale centroid of the target area map, calculate the frame descriptor based on the angle between the corner features and the grayscale centroid, use the MLP network to convert the frame descriptor into high-dimensional data, and add it to the grayscale centroid to generate the frame matching feature. The calculation formula of the frame descriptor is: Among them, t d represents the frame descriptor, a represents the serial number of a single corner feature of the target area map, A represents the total number of corner features of the target area map, Ψ a represents the angle between the ath corner feature and the grayscale centroid, β a represents the weighted value of the feature of the ath corner point, F(·) represents the Fourier transform function, x c and c Represents the horizontal and vertical coordinates of the grayscale centroid, c a represents the ath corner feature of the target area map, c m Represents the average value of the corner features of the target area map.

4. According to the low-code application automatic testing method based on machine learning according to claim 3, it is characterized in that: The construction framework identification network includes: An input layer, used to decode the frame matching features through an attention mechanism to obtain input parameters of a hidden layer; Hidden layer, used to calculate the similarity between the input parameters of the hidden layer and the typical framework element data set, and obtain the classification probability of the input parameters of each hidden layer; The voting decision layer is used to vote on the categories of the hidden layer input parameters according to the classification probability, and the hidden layer input parameter category with the most votes is counted as the interface element category.

5. According to the low-code application automatic testing method based on machine learning according to claim 2, it is characterized in that: The specific steps of extracting text matching features of the interface element graph by semantic recognition technology are as follows: S2.3.1: Process the preprocessed element grayscale image through edge detection and adaptive binarization to extract the text area; S2.3.2: Perform vertical projection on the text area, divide the row boundary and column boundary of the text area according to the maximum value of the vertical projection function, match the function value at the row boundary and column boundary with the function value of the text area, and obtain the text content vector; S2.3.3: Calculate the gray-level co-occurrence matrix of the text content vector, obtain the frequency distribution function of the text content vector through Fourier transform, and calculate the text matching feature based on the gray-level co-occurrence matrix and the frequency distribution function. The calculation formula of the text matching feature is: Among them, T f represents the text matching feature, E{·} represents the annealing genetic function, K b represents the Boltzmann constant, G represents the gray-level co-occurrence matrix, F represents the frequency distribution function, n represents the unit vector of the text content vector, N represents the total length of the text content vector, and f n represents the frequency of the nth text content vector, X n Represents the magnitude of the nth text content vector.

6. The low-code application automatic testing method based on machine learning according to claim 5 is characterized in that: The text extraction network comprises: The input layer takes the text matching features as input parameters, normalizes and matrix quantizes the text matching features, and converts them into a feature matrix of size 64×64; The convolution pooling layer convolves the input 64×64 feature matrix with a 3×3 convolution kernel in four stages, and obtains a 2×2 mapping feature through a RELU activation function and a maximum pooling operation, wherein the maximum pooling operation includes a 2×2 pooling kernel; The output layer flattens the mapping features into a compact feature vector to obtain the text descriptor. The semantic similarity between the text descriptor and the text semantic library is calculated through character pattern matching to determine whether the convergence condition is met. If the convergence condition is not met, the semantic similarity is back-propagated through the loss function. If the convergence condition is met, the text semantics corresponding to the highest semantic similarity is output as the interface text meaning.

7. The low-code application automatic testing method based on machine learning according to claim 1 is characterized in that: The generated test cases are used to automatically generate test scripts according to the element types, location information and semantic information in the interface model, including identifying elements, determining element interactions and conversion formats, and the collected test result data includes obtaining screenshots, log files and error reports during the test process.

8. The low-code application automatic testing method based on machine learning according to claim 1 is characterized in that: The specific steps of analyzing the test result data are as follows: S4.1: Identify abnormal patterns and trends during testing using cluster analysis algorithms; S4.2: Identify the causes of errors and error paths during testing through decision trees; S4.3: Count test pass rate and test coverage.

9. A low-code application automatic testing system based on machine learning, implemented based on the method according to any one of claims 1 to 8, characterized in that: The system includes an element extraction module, a use case generation module, a use case execution module, a data analysis module and a data reporting module; The element extraction module is used to receive the test requirements of the low-code application, obtain the interface element diagram of the low-code application, and extract the interface model of the low-code application; The use case generation module is used to automatically generate a test script according to the element type, position information and semantic information in the interface model; The test case execution module is used to convert the test script into a format executable by the automated testing tool, execute the test script through the automated testing tool, and collect test result data; The data analysis module is used to analyze the collected test result data; The data reporting module is used to generate a test report.

Images