Automatic software testing method and system based on multi-modal AI collaboration

Abstract

The present application relates to the technical field of automatic testing, more particularly, to an automatic software testing method and system based on multi-modal AI collaboration. The scheme includes encapsulating optical character recognition, speech recognition and natural language processing analysis tools into independent containers, and inputting the analysis results into a multi-modal semantic correlation model in real time; calculating the semantic similarity between image description text and corresponding speech transcription text; constructing a test requirement complexity evaluation model; defining a use case quality index, and establishing an error sample library to store artificially annotated problem use cases and their correction labels; finally determining the intended requirements according to the screening results of the three-level filter; setting a test case template, optimizing the path according to the fusion quality, improving the use case generation efficiency, and synchronously updating the path to the knowledge base associated with the error sample library. The scheme realizes automatic software testing through multi-modal AI collaboration, containerized deployment and model dynamic optimization, thereby improving the efficiency and recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic testing technology, more particularly, to an automatic software testing method and system based on multi-modal AI collaboration. BACKGROUND

[0002] In the field of automatic testing, the automatic software testing method is to integrate various modal data such as text, image, audio, and code, and to realize multi-dimensional and full-scene automatic testing of software by means of the collaborative work of different AI technologies such as natural language processing, computer vision, and speech recognition. This method breaks through the limitations of traditional single-modal testing in coverage and depth, and can more accurately simulate real user scenarios and efficiently identify potential defects in software interaction logic, interface display, and function implementation. Its research significance and importance are remarkable: on the one hand, as software systems become increasingly complex, multi-modal interaction becomes the mainstream trend, and traditional testing methods cannot meet the testing needs in cross-modal scenarios. Multi-modal AI collaboration can effectively improve the comprehensiveness and accuracy of testing, and greatly reduce the cost and time consumption of manual testing; on the other hand, this method promotes the upgrade of automatic testing from "rule-based" to "intelligent perception and decision-making", providing a new technical path for building a more robust and reliable software quality assurance system, and helping the software industry to achieve high-quality delivery in a rapidly changing market environment, which plays an important role in improving the overall competitiveness of the software industry.

[0003] Prior to the present application, existing automatic software testing based on multi-modal AI collaboration mainly uses natural language processing and computer vision technologies to analyze and process multi-modal data such as text and images involved in software. For example, natural language processing technology is used to analyze test requirement documents, and computer vision technology is used to identify UI elements of application programs to generate test scenarios and automated scripts. The technical difficulty lies in the compatibility and robustness of multi-modal analysis. Different sources of PDF, Word, and image formats are diverse, and the analysis tools may have environmental dependence problems. The accuracy of recognizing complex tables and handwritten images needs to be improved. The controllability of large model generated content also faces challenges, such as unstable factors in the number of use cases, structured output analysis, etc. In addition, when searching and integrating large-scale knowledge bases, how to ensure efficient vector retrieval and organic integration of retrieved knowledge and current requirements to avoid information redundancy and conflict is also a problem to be solved. The key point is to build a model that can accurately understand and process multi-modal data, so that it can effectively simulate real user scenarios and accurately identify software defects, while optimizing the collaboration mechanism between different modal technologies to ensure the smoothness and efficiency of multi-modal AI collaboration. SUMMARY

[0004] In view of the above problems, the application provides an automatic software testing method and system based on multi-modal AI collaboration, which realizes automatic software testing by multi-modal AI collaboration, containerized deployment, model dynamic optimization, etc., and improves efficiency and recognition accuracy.

[0005] According to a first aspect of an embodiment of the application, an automatic software testing method based on multi-modal AI collaboration is provided.

[0006] In one or more embodiments, preferably, the automatic software testing method based on multi-modal AI collaboration comprises:

[0007] An optical character recognition (OCR), an automatic speech recognition (ASR), a natural language processing (NLP) analysis tool and its dependent library are encapsulated into an independent container, a unified analysis service is provided to the outside through a gRPC interface, a container CPU and memory load are dynamically monitored, a load threshold is set, the load threshold is set to 80% of CPU usage and 75% of memory usage, the number of container replicas is automatically scaled according to real-time request volume and the load threshold, and a multi-modal semantic correlation model is input with real-time analysis results;

[0008] A multi-modal semantic correlation model based on a Transformer architecture is established, and a semantic similarity between image description text and corresponding speech transcription text is calculated, wherein a cosine similarity algorithm is used for semantic similarity calculation, a threshold is set to 0.7, manual review is triggered when the similarity is lower than 0.7, and confirmed analysis error samples are synchronized to an error sample library, and the analysis model parameters are fine-tuned at 3 a.m. every day to improve the analysis accuracy;

[0009] A test requirement complexity evaluation model is constructed, the number of front-end interaction points, the depth of back-end API call chain and the number of boundary conditions are used as input features, and the interval [N_min, N_max] of the number of basic test cases is output, an XGBoost-based redundancy predictor is trained to predict the redundancy of test cases, and test cases with a redundancy higher than 0.6 are determined as redundant and excluded, wherein the number of front-end interaction points is counted by summing up three types of clicks, inputs and pop-up windows, the depth of back-end API call chain is calculated by the number of nodes of the longest call path, and the number of boundary conditions is summed up by counting the number of value, type and business rule boundaries;

[0010] A test case quality indicator is defined, an error sample library is established to store problem test cases and their corrected labels, and the error sample library is used to provide training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model;

[0011] Three filters are set for the input test requirement, the input is a test requirement text preprocessed by the multi-modal semantic correlation model, and the intended requirement is finally determined according to the screening results of the three filters;

[0012] The test case template is set, the intention demand determined by the three-level filter and the basic case number interval output by the test requirement complexity evaluation model are taken as inputs, and the semantic repeated units are merged through DBSCAN clustering, wherein the clustering radius is set to 0.3; the case execution success rate and the requirement coverage degree are evaluated, and the optimization path is adjusted according to the fusion quality, the case generation efficiency is improved, and the optimization path is updated to the knowledge base associated with the error sample library.

[0013] In one or more embodiments, preferably, the optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tools and their dependent libraries are packaged into independent containers, and a unified analysis service is provided through a gRPC interface, the CPU and memory loads of the containers are dynamically monitored, and load thresholds are set, wherein the load thresholds are set to 80% CPU usage and 75% memory usage, the number of container replicas is automatically scaled according to real-time request volume and load thresholds, and the analysis results are input into a multi-modal semantic correlation model in real time, specifically including:

[0014] The optical character recognition tool, the speech recognition tool and the natural language processing tool are packaged into independent containers;

[0015] A unified analysis service is provided for the independent containers through a gRPC interface;

[0016] The current CPU usage, memory usage and corresponding target indicator values are obtained;

[0017] The target replica number is calculated using a first calculation formula;

[0018] The number of container replicas is adjusted according to the target replica number;

[0019] The first calculation formula is:

[0020] Podm=ceil[Podd×(max(CPUd,Memd)÷max(CPUm,Memm))]

[0021] Wherein, Podm is the target replica number, Podd is the current replica number, CPUd is the current CPU indicator value, CPUm is the target CPU indicator value, Memd is the current memory indicator value, Memm is the target memory indicator value, and ceil() is a rounding up function;

[0022] After adjustment, the load change is continuously monitored, and when the CPU and memory usage are both stable within the target indicator value ± 10% range for three consecutive monitoring periods, the scaling operation is stopped.

[0023] In one or more embodiments, preferably, the Transformer architecture-based multi-modal semantic correlation model is established, and the semantic similarity between the image description text and the corresponding speech transcription text is calculated, wherein the cosine similarity algorithm is used for semantic similarity calculation, the threshold is set to 0.7, and manual review is triggered when the similarity is lower than 0.7, the confirmed analysis error samples are synchronized to the error sample library, and the analysis model parameters are fine-tuned every day at 3 a.m. to improve the analysis accuracy, specifically including:

[0024] A Transformer architecture-based multi-modal semantic correlation model is established, which includes an image feature extraction layer, a speech text feature extraction layer, and a cross-attention fusion layer.

[0025] The multi-modal semantic correlation model is used to calculate the semantic similarity between the image description text and the corresponding speech transcription text, and the cosine similarity algorithm is used for calculation.

[0026] The similarity is compared with the pre-set threshold every hour, and manual review is triggered when the similarity is lower than the pre-set threshold, and the confirmed analysis error samples are injected into the training data set, and the analysis model parameters are periodically fine-tuned to improve the accuracy of the analysis model. Otherwise, no processing is performed.

[0027] In one or more embodiments, preferably, the test requirement complexity evaluation model is constructed, the front-end interaction point number, the back-end API call chain depth, and the boundary condition number are used as input features, and the basic use case number interval [N_min, N_max] is output. A redundancy predictor based on XGBoost is trained to predict the redundancy of use cases, and use cases with a redundancy higher than 0.6 are determined as redundant and excluded, wherein the front-end interaction point number is counted by clicking, inputting, and pop-up window, and the sum is calculated, the back-end API call chain depth is calculated by the number of nodes of the longest call path, and the boundary condition number is calculated by summing the number, type, and business rule boundary, specifically including:

[0028] A test requirement complexity evaluation model is constructed, which uses the front-end interaction point number, the back-end API call chain depth, and the boundary condition number as input features, and outputs the basic use case number interval [N_min, N_max]. A redundancy predictor based on XGBoost is trained to predict the redundancy of use cases.

[0029] Periodically, when the number of use cases generated by the test requirement complexity evaluation model satisfies the second calculation formula, the rule engine is activated to filter duplicate use cases and retain non-redundant use cases; otherwise, when the third calculation formula is satisfied, the requirement feature vector is injected into the test requirement complexity evaluation model to generate supplementary use cases and supplement them to the basic use case set.

[0030] The second calculation formula is:

[0031] M>1.2N_max

[0032] Wherein, M is the number of use cases generated by the test requirement complexity evaluation model, N_max is the maximum value of the basic use case number interval;

[0033] The third calculation formula is:

[0034] M<0.8N_min

[0035] Wherein, M is the number of use cases generated by the test requirement complexity evaluation model, N_min is the minimum value of the basic use case number interval.

[0036] In one or more embodiments, preferably, the definition of the use case quality index, the establishment of the error sample library stores the problem use case and its correction label annotated by artificial, the error sample library provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model at the same time, the full connection layer weight of the large model is fine-tuned based on the new samples in the error sample library at 2 o'clock in the morning every day, and the accuracy and coverage of use case generation are optimized, wherein the use case quality index includes step coverage, abnormal trigger rate and input parameter boundary violation times, the correction label contains error type and correction suggestion, and specifically includes:

[0037] Defining a use case quality index, wherein the use case quality index includes step coverage (calculated based on business process node coverage ratio), abnormal trigger rate (abnormal use case execution success ratio) and input parameter boundary violation times, wherein the step coverage is calculated based on business process node coverage ratio, and the abnormal trigger rate is the abnormal use case execution success ratio;

[0038] Establishing an error sample library to store problem use cases and their correction labels annotated by artificial, wherein the labels include error type and correction suggestion;

[0039] Fine-tuning the full connection layer weight of the large model based on the new samples in the error sample library every 24 hours;

[0040] Real-time monitoring of the unit time use case generation fluctuation value and the structured analysis success rate, as the basis for judging whether to enhance the format constraint strength of the generation template;

[0041] When the unit time use case generation fluctuation value and the structured analysis success rate of the continuous 3 times sampling do not meet the fourth calculation formula, the format constraint strength of the generation template is automatically enhanced;

[0042] The fourth calculation formula is:

[0043] CY∈[Cmin,Cmax]

[0044] CY, Cmax, and Cmin are the sample value, the maximum value deviating from the preset baseline, and the minimum value deviating from the preset baseline, respectively.

[0045] In one or more embodiments, preferably, a three-level filter is set for the input test requirement, the input being the test requirement text preprocessed by the multi-modal semantic correlation model, and the final intention requirement is determined according to the screening result of the three-level filter, specifically including:

[0046] An intention vector of the input test requirement text is extracted by using a BERT model.

[0047] A first-level filter is set, which is used to screen test requirement entries meeting a fifth calculation formula, and the test requirement corresponding to the intention vector passing through the first-level filter is taken as the remaining first-level test requirement.

[0048] A second-level filter is set, which is used to check the compatibility of the knowledge entry and the device type and the operating system version of the current test scene, and the first-level test requirement is screened by the second-level filter by querying a preset compatibility list, and the test requirement corresponding to the intention vector passing through the second-level filter is taken as the remaining second-level test requirement.

[0049] A third-level filter is set, which is used to identify candidate knowledge entries having logical contradictions with the selected entries based on a knowledge conflict detection model, and outputs entries meeting a sixth calculation formula to an isolation area, and the second-level test requirement is screened by the third-level filter, and the test requirement not moved to the isolation area is taken as the test requirement corresponding to the intention vector passing through the third-level filter, which is taken as the finally determined intention test requirement.

[0050] The fifth calculation formula is:

[0051] ZXS>0.8

[0052] Wherein, ZXS is the knowledge vector similarity.

[0053] The sixth calculation formula is:

[0054] PC>0.7

[0055] Wherein, PC is the conflict probability, which is output by the knowledge conflict detection model.

[0056] In one or more embodiments, preferably, the test case template is set, the number of basic use cases output by the intention demand determined by the three-level filter and the test demand complexity evaluation model is input, the semantic repeated units are merged by DBSCAN clustering, the clustering radius is set to 0.3, the use case execution success rate and the demand coverage degree are evaluated, and the optimization path is updated according to the fusion quality, the use case generation efficiency is improved, and the optimization path is updated to the knowledge base associated with the error sample library, and specifically includes:

[0057] The XML Schema of the test case template is taken as a skeleton, wherein the skeleton decomposes the retrieved knowledge into reusable units of input constraints, expected outputs and exception handling;

[0058] The semantic repeated units are merged by DBSCAN clustering, the Levenshtein distance minimum expression is retained, and the clustering radius is set to 0.3;

[0059] The attention weight matrix of the knowledge unit and the template node is added to the GPT fusion module to enhance the fusion strength of the key knowledge;

[0060] The fusion quality evaluation function calculation value is calculated by using the seventh calculation formula;

[0061] When the Q value is greater than a preset threshold, the corresponding fusion path is recorded to the knowledge base, otherwise the manual optimization is triggered;

[0062] The seventh calculation formula is:

[0063] Q = alpha * X + beta * Y - gamma * Z

[0064] Wherein Q is the fusion quality evaluation function calculation value, alpha is the coverage degree weight coefficient, beta is the executability weight coefficient, gamma is the conflict number weight coefficient, and alpha + beta + gamma = 1, Z is the conflict number, Y is the executability, which is a score between 0 and 1, evaluated by the automated execution engine, X is the coverage degree, which represents the demand coverage ratio;

[0065] The eighth calculation formula is:

[0066] Delta Q > 0.3

[0067] Wherein Delta Q is the improvement ratio of the fusion quality evaluation function calculation value in the manual correction case, when the formula is satisfied, the optimization path in the knowledge base is updated.

[0068] According to the second aspect of the embodiment of the application, an automatic software testing system based on multi-modal AI collaboration is provided.

[0069] In one or more embodiments, preferably, the multi-modal AI collaboration-based automated software testing system comprises a service cluster construction module, a collaborative correction module, a supplementary use case module, a use case real-time monitoring module, a three-level retrieval filtering module, and a fusion optimization module.

[0070] According to a third aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions, when executed by a processor, implement the method according to any one of the first aspect of the embodiments of the present application.

[0071] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, which comprises a memory and a processor, the memory is configured to store one or more computer program instructions, and the one or more computer program instructions are executed by the processor to implement the method according to any one of the first aspect of the embodiments of the present application.

[0072] The technical scheme provided by the embodiments of the present application can include the following beneficial effects:

[0073] In the scheme of the present application, the multi-modal AI collaboration is combined with dynamic optimization to improve the testing accuracy.

[0074] In the scheme of the present application, the containerization deployment is combined with the intelligent filtering mechanism to significantly improve the testing efficiency compared with the traditional method.

[0075] Other features and advantages of the present application will be described in the following description, and some will become apparent from the description, or will be understood through implementation of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0076] The technical scheme of the present application will be further described in detail below with the help of the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical scheme in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description can only be some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0078] Figure 1 is a flowchart of a multi-modal AI collaboration-based automated software testing method according to an embodiment of the present application.

[0079] Figure 2 is a flowchart of step S101 in the multi-modal AI collaboration-based automated software testing method according to an embodiment of the present application.

[0080] Figure 3 is a flowchart of step S102 in the automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0081] Figure 4 is a flowchart of step S103 in the automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0082] Figure 5 is a flowchart of step S104 in the automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0083] Figure 6 is a flowchart of step S105 in the automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0084] Figure 7 is a flowchart of step S106 in the automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0085] Figure 8 is a structural diagram of the automated software testing system based on multi-modal AI collaboration according to an embodiment of the present application.

[0086] Figure 9 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0087] In some of the flowcharts described in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed in the order in which they appear in this text or in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these flowcharts can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0088] In the embodiments of the present application, an automated software testing method and system based on multi-modal AI collaboration are provided. Through multi-modal AI collaboration, containerized deployment, model dynamic optimization, etc. are combined to realize automated software testing, and the efficiency and recognition accuracy are improved.

[0089] According to a first aspect of an embodiment of the present application, an automated software testing method based on multi-modal AI collaboration is provided.

[0090] Figure 1 is a flowchart of an automated software testing method based on multi-modal AI collaboration according to an embodiment of the present application.

[0091] In one or more embodiments, preferably, the automated software testing method based on multi-modal AI collaboration comprises:

[0092] S101, encapsulate optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tools and their dependent libraries into independent containers, provide unified analysis services through gRPC interface, dynamically monitor container CPU and memory load, set load threshold, wherein the load threshold is set to CPU usage rate 80%, memory usage rate 75%, automatically scale the number of container replicas according to real-time request volume and load threshold, and input the analysis result into a multi-modal semantic correlation model in real time;

[0093] S102, establish a multi-modal semantic correlation model based on the Transformer architecture, and calculate the semantic similarity between the image description text and the corresponding speech transcription text, wherein the semantic similarity calculation uses the cosine similarity algorithm, and the threshold is set to 0.7, when the similarity is lower than 0.7, trigger manual review, confirmed analysis error samples are synchronized to the error sample library, and the analysis model parameters are fine-tuned at 3 am every day to improve the analysis accuracy;

[0094] S103, construct a test requirement complexity evaluation model, take the number of front-end interaction points, the depth of back-end API call chain, and the number of boundary conditions as input features, output the basic case number interval [N_min, N_max], train an XGBoost-based redundancy predictor to predict the case redundancy, and cases with redundancy higher than 0.6 are determined as redundant and excluded, wherein the number of front-end interaction points is counted by clicking, inputting, and pop-up window, and then summed, the depth of back-end API call chain is calculated by the number of nodes of the longest call path, and the number of boundary conditions is counted by value, type, and business rule boundary, and then summed;

[0095] S104, define the case quality index, establish an error sample library to store problem cases and their correction labels annotated by humans, and the error sample library provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model;

[0096] S105, set a three-level filter for the input test requirement, the input is the test requirement text preprocessed by the multi-modal semantic correlation model, and finally determine the intended requirement according to the screening results of the three-level filter;

[0097] S106, set the test case template, to the intention requirement determined by the three-level filter and the basic case interval output by the test requirement complexity evaluation model as input, merge the semantic repeated units by DBSCAN clustering, wherein the clustering radius is set to 0.3, evaluate based on the use case execution success rate and the requirement coverage, and optimize the path according to the fusion quality, improve the use case generation efficiency, and update the optimized path to the knowledge base associated with the error sample library synchronously.

[0098] The embodiment takes the multi-modal interaction function test of the intelligent vehicle-mounted system as an example, and the specific implementation process is as follows: the OCR tool for recognizing the text on the vehicle-mounted screen, the ASR tool for transcribing the voice instruction and the NLP tool for analyzing the natural language are packaged into a Docker independent container, a unified analysis service is provided through a gRPC interface, the container load is monitored by Kubernetes (the threshold is set to 80% of CPU usage and 75% of memory usage), when the ASR container CPU reaches 90% due to the surge of voice requests in the morning peak, the number of replicas is increased from 2 to 3 according to the formula, and after stabilization, the scaling is stopped, and the analysis result is transmitted into the multi-modal model in real time; a Transformer multi-modal semantic correlation model containing a ResNet-50 image feature extraction layer, a Whisper text encoder and 8 cross-attention layers is constructed, the cosine similarity of the image of the vehicle-mounted screen display "turn left lane" and the corresponding voice transcribed text is calculated, if 0.85 (higher than the threshold 0.7) is obtained, it is determined to be consistent, if the image "speed limit 60km / h" and the voice "speed limit 80km / h" (similarity 0.62) are encountered, manual review is triggered, error samples are labeled and injected into the training set, and the model parameters are fine-tuned at 3 o'clock every morning; a test requirement complexity evaluation model is constructed, the front-end interaction points (click, input, pop-up window, a total of 10), the back-end API chain depth (3 nodes) and the boundary conditions (5) are taken as inputs, the output use case number interval is [20, 30], the XGBoost redundancy predictor is trained to remove 3 repeated use cases with a score >0.6, and 25 valid use cases are reserved; quality indicators such as step coverage (83%) and abnormal trigger rate (80%) are defined, an error sample library is established to store problem use cases and correction labels, the large model weight is fine-tuned based on the newly added samples at 2 o'clock every morning, and the template constraint is enhanced when the use case generation fluctuation exceeds the baseline ±20%; for the "dialect voice temperature control" requirement, after the BERT extraction vector, the first filter (similarity 0.82>0.8), the second filter (compatible with Android 12) and the third filter (conflict probability 0.2<0.7) are sequentially screened to determine the intended requirement; taking XML Schema as the template skeleton, the repeated units are merged through DBSCAN clustering (radius 0.3), the GPT fusion module adds attention weight, the quality value is calculated as Q=0.4×coverage+0.4×executability-0.2×conflict number (0.66>threshold 0.6) and the path is recorded, the Q value is improved by 0.35 (>0.3) after manual correction, the knowledge base is updated, and finally 22 valid use cases are generated, the test efficiency is improved by 40%, and the error recognition accuracy reaches 92%.

[0099] Figure 2 is a flow chart of step S101 in the multi-modal AI collaborative automatic software testing method according to an embodiment of the application.

[0100] As Figure 2As shown, in one or more embodiments, preferably, the optical character recognition (OCR), speech recognition (ASR), natural language processing (NLP) parsing tools and their dependent libraries are packaged into independent containers, a unified parsing service is provided externally through a gRPC interface, the CPU and memory load of the container is dynamically monitored, a load threshold is set, the load threshold is set to 80% CPU usage and 75% memory usage, the number of container replicas is automatically scaled according to real-time request volume and load threshold, and the parsing result is input into a multi-modal semantic correlation model in real time. Specifically, it includes:

[0101] S201, the optical character recognition tool, the speech recognition tool and the natural language processing tool are packaged into independent containers;

[0102] S202, a unified parsing service is provided externally for the independent container through a gRPC interface;

[0103] S203, the current CPU usage, memory usage and corresponding target indicator value are obtained;

[0104] S204, the target replica number is calculated using a first calculation formula;

[0105] S205, the number of container replicas is adjusted according to the target replica number;

[0106] The first calculation formula is:

[0107] Podm=ceil[Podd×(max(CPUd,Memd)÷max(CPUm,Memm))]

[0108] Wherein, Podm is the target replica number, Podd is the current replica number, CPUd is the current CPU indicator value, CPUm is the target CPU indicator value, Memd is the current memory indicator value, Memm is the target memory indicator value, and ceil() is the ceiling function;

[0109] S206, the load change is continuously monitored after adjustment, and when the CPU and memory usage are stable within the target indicator value ± 10% range for 3 consecutive monitoring periods, the scaling operation is stopped.

[0110] In the embodiment, the optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tools and their dependent libraries are packaged into independent containers, and a unified analysis service is provided through a gRPC interface. The CPU and memory loads of the containers are dynamically monitored, and load thresholds (the load thresholds are set to 80% for CPU usage and 75% for memory usage). The number of container replicas is automatically scaled according to real-time request volume and load thresholds. The specific implementation of the multi-modal semantic correlation model for real-time input of analysis results is as follows: First, select Tesseract as the optical character recognition tool, Whisper as the speech recognition tool, and spaCy as the natural language processing tool. Package the above tools and their respective dependent library files (such as the leptonica library dependent on Tesseract, the PyTorch library dependent on Whisper, and the numpy library dependent on spaCy) into independent Docker containers. Each container contains only one analysis tool and its complete runtime environment, ensuring that the tools run in isolation. Then, configure a gRPC server for each Docker container and define a unified analysis service interface protocol (in protobuf format) that includes input parameters (such as image data, speech data, and text data to be analyzed) and output parameters (such as OCR recognition results, ASR transcribed text, and NLP analyzed structured information), allowing external systems to call the analysis services of different containers through the same interface. Subsequently, use a container orchestration tool (such as Kubernetes) to monitor the CPU usage and memory usage of each container in real time. The target CPU indicator value CPUm in the set load threshold is 80%, and the target memory indicator value Memm is 75%. Also, record the current replica number Podd and the current CPU indicator value CPUd and the current memory indicator value Memd corresponding to the real-time request volume. When the current CPU usage CPUd of an OCR container is 90%, the current memory usage Memd is 70%, and the current replica number Podd is 2, calculate the target replica number Podm using the first calculation formula Podm = ceil[Podd x (max(CPUd ÷ CPUm, Memd ÷ Memm))]. First, calculate max(90% ÷ 80%, 70% ÷ 75%) = max(1.125, 0.933) = 1.125. Then, calculate 2 x 1.125 = 2.25. After processing by the ceil() rounding function, the target replica number Podm is 3. At this time, the container orchestration tool automatically adds one OCR container replica. After adjustment, each 5-minute period is a monitoring period, and three consecutive monitoring periods are monitored. When the CPU usage of the OCR container cluster is stable at 72%-88% (80% ± 10%) and the memory usage is stable at 67.5%-82% (75% ± 10%) for three consecutive periods, the CPU usage of the OCR container cluster is stable at 72%-88% (80% ± 10%), and the memory usage is stable at 67.5%-82%.5% (75% ± 10%) range, stop the scaling operation of the number of container copies; finally, the analysis results returned by each container through the gRPC interface (such as image text recognized by OCR, voice text transcribed by ASR, and demand keywords analyzed by NLP) are transmitted in real time to a multi-modal semantic association model based on a Transformer architecture for subsequent semantic similarity calculation.

[0111] Figure 3 is a flowchart of step S102 in the multi-modal AI collaborative-based automated software testing method according to an embodiment of the present application.

[0112] As shown in Figure 3 in one or more embodiments, preferably, the multi-modal semantic association model based on the Transformer architecture is established to calculate the semantic similarity between the image description text and the corresponding voice transcription text, wherein the cosine similarity algorithm is used for semantic similarity calculation, and the threshold is set to 0.7. When the similarity is lower than 0.7, manual review is triggered, and the confirmed analysis error samples are synchronized to the error sample library. The analysis model parameters are fine-tuned at 3 a.m. every day to improve the analysis accuracy, specifically including:

[0113] S301, a multi-modal semantic association model based on the Transformer architecture is established, and the model includes an image feature extraction layer, a voice text feature extraction layer, and a cross-attention fusion layer;

[0114] S302, the multi-modal semantic association model is used to calculate the semantic similarity between the image description text and the corresponding voice transcription text, and the cosine similarity algorithm is used for calculation;

[0115] S303, the similarity is compared with the pre-set threshold every hour. When the similarity is lower than the pre-set threshold, manual review is triggered, and the confirmed analysis error samples are injected into the training data set to improve the accuracy of the analysis model by periodically fine-tuning the analysis model parameters; otherwise, no processing is performed.

[0116] In this embodiment, the Transformer architecture-based multi-modal semantic correlation model is established, the semantic similarity between the image description text and the corresponding speech transcription text is calculated (using the cosine similarity algorithm, and the threshold is set to 0.7), and when the similarity is lower than 0.7, the manual review is triggered, the confirmed analysis error sample is synchronized to the error sample library, and the specific implementation of adjusting the analysis model parameters every day at 3 a.m. to improve the analysis accuracy is as follows: first, a Transformer architecture-based multi-modal semantic correlation model is constructed, which is composed of an image feature extraction layer, a speech text feature extraction layer and a cross-attention fusion layer. The image feature extraction layer adopts a pre-trained ResNet-50 model (the first 10 layers of parameters are frozen) to extract visual feature vectors in the image; the speech text feature extraction layer adopts the text encoder of the Whisper model to convert the ASR transcribed speech text into semantic feature vectors; the cross-attention fusion layer uses an 8-head multi-head attention mechanism to realize the interaction and fusion of image features and speech text features, and outputs a multi-modal feature vector with a dimension of 768. Taking a test scene of an intelligent vehicle system as an example, the system interface screenshot contains a visual prompt (image data) of "changing lanes to the left", and at the same time, the voice navigation issues an instruction (voice data) of "please change lanes to the left". The image is input into the image feature extraction layer to obtain an image feature vector V_image with a dimension of [1, 768], and the speech-transcribed text "please change lanes to the left" is input into the speech text feature extraction layer to obtain a text feature vector V_text with a dimension of [1, 768]. In the cross-attention fusion layer, the correlation weight between V_image and V_text is calculated through the interaction mechanism of query (Query), key (Key) and value (Value), and the fused feature vector V_fusion is obtained. Then, the cosine similarity algorithm is used to calculate the semantic similarity between V_image and V_text, and the calculation formula is: Sim = cosine(V_image, V_text) = (V_image · V_text) / (||V_image|| · ||V_text||). After substituting the specific numerical value, the similarity value is obtained as 0.85, and since 0.85 is greater than the preset threshold 0.7, the system determines that the image description text and the speech transcription text are semantically consistent, and does not trigger the manual review process. If in another scene, the image shows "turn right at the upcoming intersection", and the speech-transcribed text is "turn left at the upcoming intersection", the calculated similarity value is 0.6, and since 0.6 is less than 0.7, the system immediately triggers the manual review process, and sends a review task sheet containing the image, voice and similarity value to the test team through WeChat Enterprise. After the test personnel confirm that this is an analysis error sample (image and voice instruction conflict), the sample is labeled as "inconsistent direction instruction" and synchronized to the error sample library.At 3 a.m. every day, the system automatically extracts the newly added parsing error samples in the past 24 hours from the error sample library to form a mini-batch training data with a size of 128, and uses the Adam optimizer (learning rate set to 1e-5) to fine-tune the cross-attention fusion layer parameters of the multi-modal semantic correlation model. During the fine-tuning process, the triplet loss function is used to ensure that the distance between the positive sample pairs (semantically consistent image-text pairs) is less than the distance between the negative sample pairs by at least 0.5. After each fine-tuning is completed, the model performance is evaluated using the validation set (containing 500 labeled samples), and if the average similarity accuracy improves by more than 2%, the current model parameters are saved as a new version. Through this periodic model fine-tuning, the parsing accuracy of the multi-modal semantic correlation model for image description text and speech transcription text is gradually improved from the initial 82% to 91%.

[0117] Figure 4 is a flowchart of step S103 in the multi-modal AI collaborative automatic software testing method according to an embodiment of the application.

[0118] As shown in Figure 4 in one or more embodiments, preferably, the test requirement complexity evaluation model is constructed, with the number of front-end interaction points, the depth of back-end API call chain, and the number of boundary conditions as input features, and the interval [N_min, N_max] of the number of basic use cases as output. An XGBoost-based redundancy predictor is trained to predict the redundancy of use cases, and use cases with a redundancy higher than 0.6 are determined as redundant and excluded. Specifically, the number of front-end interaction points is counted by summing up clicks, inputs, and pop-up windows, the depth of back-end API call chain is calculated by the number of nodes in the longest call path, and the number of boundary conditions is summed up by counting numerical, type, and business rule boundaries.

[0119] S401, a test requirement complexity evaluation model is constructed, with the number of front-end interaction points, the depth of back-end API call chain, and the number of boundary conditions as input features, and the interval [N_min, N_max] of the number of basic use cases as output. An XGBoost-based redundancy predictor is trained to predict the redundancy of use cases;

[0120] S402, when the number of use cases generated by the test requirement complexity evaluation model satisfies the second calculation formula, activate the rule engine to filter duplicate use cases and retain non-redundant use cases; otherwise, when the third calculation formula is satisfied, inject a requirement feature vector into the test requirement complexity evaluation model to generate supplementary use cases and supplement them to the basic use case set;

[0121] The second calculation formula is:

[0122] M>1.2N_max

[0123] wherein M is the number of use cases generated by the test requirement complexity evaluation model, and N_max is the maximum value of the basic use case number interval;

[0124] The third calculation formula is:

[0125] M < 0.8N_min

[0126] wherein M is the number of use cases generated by the test requirement complexity evaluation model, and N_min is the minimum value of the basic use case number interval.

[0127] In the embodiment, the test requirement complexity evaluation model is constructed, the number of front-end interaction points, the depth of back-end API call chain, and the number of boundary conditions are input features (the input features include the semantic consistency score output by the multi-modal semantic association model), and the basic use case number interval [N_min, N_max] is output. The redundancy predictor based on XGBoost is trained to predict the use case redundancy. The use case with a redundancy higher than 0.6 is determined as redundant and eliminated. The specific implementation is as follows: first, the test requirement complexity evaluation model is constructed, the number of front-end interaction points, the depth of back-end API call chain, and the number of boundary conditions are input features, wherein the number of front-end interaction points is counted according to three types of clicks, inputs, and pop-up windows, and then summed up, the depth of back-end API call chain is calculated according to the number of nodes of the longest call path, and the number of boundary conditions is counted according to the number value, type, and business rule boundary, and then summed up. Taking the product detail page of an e-commerce APP as an example, the number of front-end interaction points is counted as follows: the number of click-type interactions (such as adding to cart, buying immediately, and viewing comments) is 5, the number of input-type interactions (such as inputting quantity and coupon code) is 2, and the number of pop-up window-type interactions (such as confirming purchase and deleting product prompt) is 3, so the total number of front-end interaction points is 5+2+3=10; the depth of back-end API call chain is counted as follows: starting from the client-initiated product detail request, passing through user authentication service, product information service, inventory service, price calculation service, and finally to order preview service, the longest call path contains 5 API nodes, so the depth of back-end API call chain is 5; the number of boundary conditions is counted as follows: the number of value boundaries (such as the minimum value of product quantity 1 and the maximum value of product quantity 999) is 2, the number of type boundaries (such as the product ID must be a numeric string and the coupon code must be an alphanumeric combination) is 3, and the number of business rule boundaries (such as members can enjoy discounts and cannot place an order when the inventory is insufficient) is 4, so the total number of boundary conditions is 2+3+4=9. The number of front-end interaction points 10, the depth of back-end API call chain 5, and the number of boundary conditions 9 are input into the test requirement complexity evaluation model, and the semantic consistency score 0.85 output by the multi-modal semantic association model is added (indicating that the consistency of the text and image description in the requirement document is good), the model outputs the basic use case number interval [25, 35], that is, N_min=25 and N_max=35.

[0128] Next, the XGBoost-based redundancy predictor is trained using historical test case data (containing 1000 samples of test cases labeled as redundant or non-redundant) for training, and the features include 20 dimensions such as precondition similarity, operation step similarity, and expected result similarity of the test case. For the current generated test case set, a total of 38 test cases are generated, i.e. M = 38. At this time, the relationship between M and the interval of the base test cases is determined, since 38 > 1.2 x 35 = 42, it does not satisfy the second calculation formula M > 1.2 N_max, and at the same time 38 > 0.8 x 25 = 20, it also does not satisfy the third calculation formula M < 0.8 N_min, so the rule engine filtering or supplementary test case operation is not triggered.

[0129] Then, the trained XGBoost redundancy predictor is used to predict the redundancy of the 38 test cases, and the redundancy score of each test case is obtained. For example, the redundancy score of a test case "checking whether the price calculation is correct when a member user uses a full-reduction coupon to purchase a single product" is 0.65, since 0.65 > 0.6, the test case is determined to be a redundant test case and is removed; while the redundancy score of another test case "checking whether the system correctly prompts when a non-member user uses an expired coupon" is 0.45, since 0.45 < 0.6, the test case is retained as a non-redundant test case. After redundancy filtering, 32 valid test cases are finally retained as the test case set of the product detail page.

[0130] Periodically (every hour) determine the relationship between the number of test cases M generated by the test requirement complexity evaluation model and the interval [N_min, N_max] of the base test cases: if in a certain evaluation, the number of generated test cases M = 45, at this time 45 > 1.2 x 35 = 42, it satisfies the second calculation formula M > 1.2 N_max, then the rule engine is activated to filter repeated test cases, and the rule engine compares the similarity between test cases (such as operation step similarity > 80% and expected result similarity > 90%), and marks the test cases with high similarity as repeated test cases and removes them; if in another evaluation, the number of generated test cases M = 20, at this time 20 < 0.8 x 25 = 20, it satisfies the third calculation formula M < 0.8 N_min, then a new requirement feature vector is injected into the test requirement complexity evaluation model (such as adding the boundary condition of "price calculation during promotion activities"), which drives the model to generate supplementary test cases and supplement them to the base test case set, ensuring the sufficiency of the test cases.

[0131] Figure 5 is a flowchart of step S104 in the embodiment of the application.

[0132] As Figure 5As shown, in one or more embodiments, preferably, the definition use case quality indicators, establish an error sample library to store artificially labeled problem use cases and their correction labels, and the error sample library provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model. Specifically, it includes:

[0133] S501, defining use case quality indicators, wherein the use case quality indicators include step coverage (calculated based on business process node coverage ratio), abnormal trigger rate (abnormal use case execution success ratio), and input parameter boundary violation times, wherein the step coverage is calculated based on the business process node coverage ratio, and the abnormal trigger rate is the abnormal use case execution success ratio;

[0134] S502, establishing an error sample library to store artificially labeled problem use cases and their correction labels, wherein the labels include error types and correction suggestions;

[0135] S503, fine-tuning the full connection layer weight of the large model based on the newly added samples in the error sample library every 24 hours;

[0136] S504, real-time monitoring of the unit time use case generation fluctuation value and the structured analysis success rate as the basis for determining whether to enhance the generation template format constraint strength;

[0137] S505, when the unit time use case generation fluctuation value and the structured analysis success rate of the three consecutive samples do not meet the fourth calculation formula, automatically enhancing the format constraint strength of the generation template;

[0138] The fourth calculation formula is:

[0139] CY∈[Cmin,Cmax]

[0140] Wherein, CY is the sampling value, the sampling value is the unit time use case generation fluctuation value, Cmax is the maximum value deviating from the preset baseline, and Cmin is the minimum value deviating from the preset baseline.

[0141] In the embodiment, the definition use case quality indicators, the establishment of error sample library storage artificial marking problem use case and its correction label, based on the error sample library at 2 o'clock in the morning every day, the full connection layer weight of the large model is fine-tuned to optimize the use case generation quality, the unit time use case generation quantity fluctuation value and the structured analysis success rate are monitored in real time, and whether the generation template format constraint strength is enhanced is judged according to the fourth calculation formula. The specific implementation mode is as follows: taking the payment function test of a take-out APP as an example, first, define the use case quality indicators, wherein the step coverage is calculated based on the business process node coverage ratio - the business process of the payment function includes "select payment method → input amount → verify payment password → initiate payment → receive payment result" 5 nodes, if a test case covers 4 nodes, the step coverage is 4 / 5x100%=80%; The abnormal trigger rate is the abnormal test case execution success rate, 10 abnormal test cases are designed for this function (such as "input negative amount", "use expired coupons", "network interruption during payment" and the like), if 8 of them successfully trigger system abnormal prompt in actual execution, the abnormal trigger rate is 8 / 10x100%=80%; The number of input parameter boundary violation times refers to the number of times the input parameter exceeds the preset boundary in the test case, for example, when testing "input amount", the preset amount range is 0.01-10000 yuan, and 10001 yuan is input in a test case, which is recorded as 1 time of violation. Then, the error sample library is established to store the artificial marking problem use case and its correction label: for example, a test case "verify whether the order details page is jumped after WeChat payment" does not contain the "page jump when payment fails" step, the artificial marking error type is "scene missing", and the correction suggestion is "add the jump verification step of the payment failure scene", the use case and the label are stored in the sample library. Subsequently, the full connection layer weight of the large model is fine-tuned based on the newly added error samples (such as 30 newly added samples in the day) in the sample library every 24 hours (at 2 o'clock in the morning every day), specifically, the SGD optimizer is used, the learning rate is set to 0.001, and 3 epochs are trained to minimize the error rate of the use case generation.Meanwhile, the unit time use case generation fluctuation value and structured analysis success rate are monitored in real time: the unit time is set to be 1 hour, the preset use case generation baseline is 60 per hour, Cmax is +20% of the baseline (i.e. 72), Cmin is -20% of the baseline (i.e. 48), the structured analysis success rate baseline is 90%, Cmax thereof is 95%, and Cmin thereof is 85%; 1 sample is taken per hour, if in the 3 consecutive samples, 45 are generated in the first hour (fluctuation value -25%, lower than Cmin), 40 are generated in the second hour (fluctuation value -33%), and 42 are generated in the third hour (fluctuation value -30%), none of them meets the fourth calculation formula CY ∈ [Cmin, Cmax] (i.e. 48-72), and the structured analysis success rate is 82%, 80%, and 79% (all lower than Cmin=85%) for 3 consecutive times, the format constraint strength of the generation template is automatically enhanced, for example, a mandatory field "must contain payment scenario (success / failure)" is added in the template to limit the use of ambiguous expressions. In this embodiment, the specific implementation of defining the use case quality index, establishing an error sample library to store problem use cases and their correction labels, adjusting the weights of the full connection layer of the large model based on the error sample library to optimize the use case generation quality at 2 a.m. every day, and monitoring the unit time use case generation fluctuation value and structured analysis success rate and determining whether to enhance the format constraint strength of the generation template according to the fourth calculation formula is as follows: taking the return function test of an e-commerce platform as an example, first, the use case quality index is defined, wherein the step coverage is calculated based on the business process node coverage ratio - the return process includes "submitting a return application → uploading product photos → filling in return reasons → passing the audit → taking the goods → refunding to the account", if a test case covers 5 of the 6 nodes, the step coverage is 5 / 6*100%≈83.3%; the abnormal trigger rate is the ratio of the number of abnormal use cases that are successfully triggered to the total number of abnormal use cases, for this function, 10 abnormal use cases are designed (such as "submitting an application without uploading photos", "filling in illegal return reasons", "applying for taking goods when the audit is not passed", etc.), and in actual execution, 8 are successfully triggered to system abnormal prompt (such as "please upload product photos"), so the abnormal trigger rate is 8 / 10*100% = 80%; the input parameter boundary violation times refer to the number of times that the input parameters in the use case exceed the preset boundary, for example, the return amount is preset to be 0.01-5000 yuan, and 5001 yuan is input in a use case, which is counted as 1 boundary violation. Then, an error sample library is established to store problem use cases and their correction labels: for example, a test case "verify whether the taking time after passing the audit is correct" does not contain the business rule of "delay of taking time on weekends", the error type is manually labeled as "omission of business rule", and the correction suggestion is "add a verification step of taking time on weekends", and the use case and the label are stored in the sample library.Subsequently, the full connection layer weight of the large model is fine-tuned based on the newly added error samples in the sample library every 24 hours (2 a.m. every day), specifically, the Adam optimizer (learning rate 0.0005) is used, the newly added samples are used as training data, 4 epochs are trained, the weight is updated through back propagation, and the error rate generated by the use case is reduced. At the same time, the unit time use case generation fluctuation value and the structured analysis success rate are monitored in real time: set the unit time as 1 hour, the use case generation baseline as 50 per hour (i.e. the preset reference value), Cmax as +20% of the baseline (60), Cmin as -20% of the baseline (40), the structured analysis success rate refers to the proportion of the structured data (such as preconditions, operation steps, and expected results) of the use case that is correctly parsed, and the baseline is 92%, Cmax=96%, and Cmin=88%; sample 1 time per hour, if in the last 3 samplings, 38 use cases are generated in the first hour (fluctuation value-24%, lower than Cmin), 35 use cases are generated in the second hour (fluctuation value-30%), and 36 use cases are generated in the third hour (fluctuation value-28%), all of which do not meet the fourth calculation formula CY∈[Cmin,Cmax] (i.e. 40-60), and the structured analysis success rate is 86%, 84%, and 83% (all lower than Cmin=88%) for 3 times in a row, the format constraint strength of the generated template is automatically enhanced, for example, the template is forced to require "operation steps to be written in 'preconditions→action→expected results' three-section structure", and the use of ambiguous expressions such as "maybe" and "probably" is limited, so as to improve the stability and parsing accuracy of the use case generation.

[0142] Figure 6 is a flowchart of step S105 in the multi-modal AI collaborative automatic software testing method according to an embodiment of the present application.

[0143] As shown in Figure 6 in one or more embodiments, preferably, a three-level filter is set for the input test requirement, the input is the test requirement text preprocessed by the multi-modal semantic association model, and the final intention requirement is determined according to the screening result of the three-level filter, specifically including:

[0144] S601, using a BERT model to extract an intention vector of the input test requirement text;

[0145] S602, setting a first-level filter, the first-level filter is used to screen test requirement items meeting a fifth calculation formula, and the test requirement corresponding to the intention vector passing through the first-level filter is used as the remaining first-level test requirement;

[0146] S603, set a second filter, the second filter is used to verify the knowledge item and the device type of the current test scene, the operating system version compatibility, the first test requirement is filtered by querying the preset compatibility list using the second filter, and the test requirement corresponding to the intention vector of the second filter is used as the remaining second test requirement;

[0147] S604, set a third filter, the third filter is used to identify candidate knowledge items that have logical contradictions with the selected items based on a knowledge conflict detection model, output the items satisfying the sixth calculation formula to the isolation area, filter the second test requirement using the third filter, and the test requirement corresponding to the intention vector of the third filter is considered to pass the third filter and is used as the finally determined intention test requirement;

[0148] The fifth calculation formula is:

[0149] ZXS>0.8

[0150] Wherein, ZXS is the knowledge vector similarity;

[0151] The sixth calculation formula is:

[0152] PC>0.7

[0153] Wherein, PC is the conflict probability, which is output by the knowledge conflict detection model.

[0154] In this embodiment, the third filter is set for the input test requirement (the input is the test requirement text preprocessed by the multi-modal semantic association model), and the specific implementation manner of finally determining the intention requirement according to the screening result is as follows: taking the test requirement processing of a certain medical device management system as an example, first, the intention vector of the test requirement text is extracted by using the BERT model (adopting the pre-trained ClinicalBERT model, fine-tuned in the medical field corpus). For example, the input test requirement text is preprocessed by the multi-modal semantic association model to obtain "verify the accuracy of the drug reminder function for diabetic patients on iOS16 system", which is input into the BERT model, and the output vector of the [CLS] mark position is used to obtain the intention vector V_req=[v1, v2,..., v 768 ] with a dimension of 768.

[0155] Then a first-level filter is set to calculate the knowledge vector similarity ZXS (using cosine similarity algorithm) between the intention vector and the existing demand vector in the knowledge base. Assuming that there are demand vectors V_base1 (corresponding to "verify the accuracy of the drug reminder function for hypertension patients on iOS15 system") and V_base2 (corresponding to "verify the stability of blood glucose monitoring data synchronization function on Android system") in the knowledge base, the cosine similarity of V_req and V_base1, V_base2 is calculated to obtain ZXS1=0.85 and ZXS2=0.62. Since the fifth calculation formula requires ZXS>0.8, the test requirement meets the similarity condition with V_base1, passes the first-level filter, and becomes the remaining first-level test requirement.

[0156] Then a second-level filter is set to check the compatibility of the requirement with the device type and operating system version of the current test scenario. The preset compatibility list records that the test environment supports iOS16 system, but only supports iPhone 13 and above models. Although the requirement text mentions iOS16 system, it does not specify the device model, and further queries the context information (such as user operation log showing that the current test device is iPhone 14) output by the multi-modal semantic association model confirms that the requirement is compatible with the test scenario, passes the second-level filter, and becomes the remaining second-level test requirement.

[0157] Finally, a third-level filter is set to identify potential logical contradictions based on a knowledge conflict detection model (using a combination of rule-based reasoning and BERT model). Assuming that the currently selected test requirement item is "verify the system to save the patient data not submitted when the network is disconnected", the knowledge conflict detection model calculates the conflict probability PC of the demand to be screened and the selected item. The model first extracts the key entities (such as "drug reminder function", "network disconnection", "data saving") and constraint conditions (such as "iOS16 system", "network disconnection state") of the two requirements, judges whether there are mutual exclusion conditions (such as the drug reminder function depends on network connection, while the data saving function does not depend on) through rule-based reasoning, and calculates the semantic conflict score using the BERT model. Finally, the conflict probability PC=0.65 is output. Since the sixth calculation formula requires PC>0.7, the requirement is not moved to the isolation area, passes the third-level filter, and is determined as the final intended test requirement.

[0158] Through the above three-level filtering process, the initial test requirement text "verify the accuracy of the drug reminder function for diabetes patients on iOS16 system" passes the knowledge vector similarity screening, device / system compatibility verification, and logical conflict detection in turn, and is finally confirmed as an effective test requirement, entering the subsequent test case generation stage.

[0159] Figure 7is a flow chart of step S106 in the multi-modal AI collaborative-based automated software testing method of one embodiment of the application.

[0160] As shown in the drawings, Figure 7 In one or more embodiments, preferably, the test case template is set, the number of basic use cases output by the intention requirement and test requirement complexity evaluation model determined by the three-level filter is taken as input, and the semantic repeated units are merged by DBSCAN clustering, wherein the clustering radius is set to 0.3, the use case execution success rate and requirement coverage degree are evaluated, and the optimization path is updated to the knowledge base associated with the error sample library according to the fusion quality, the use case generation efficiency is improved, and the optimization path is updated to the knowledge base associated with the error sample library, and specifically includes:

[0161] S701, taking the XML Schema of the test case template as a skeleton, wherein the skeleton decomposes the retrieved knowledge into reusable units of input constraints, expected outputs, and exception handling;

[0162] S702, merging semantic repeated units by DBSCAN clustering, retaining the expression with the smallest Levenshtein distance, and setting the clustering radius to 0.3;

[0163] S703, adding an attention weight matrix of knowledge units and template nodes to the GPT fusion module to enhance the fusion strength of key knowledge;

[0164] S704, calculating the fusion quality evaluation function value by using the seventh calculation formula;

[0165] S705, when the Q value is greater than the preset threshold, the corresponding fusion path is recorded to the knowledge base, otherwise manual optimization is triggered;

[0166] The seventh calculation formula is:

[0167] Q = alpha * X + beta * Y - gamma * Z

[0168] Wherein, Q is the fusion quality evaluation function value, alpha is the coverage weight coefficient, beta is the executability weight coefficient, gamma is the conflict number weight coefficient, and alpha + beta + gamma = 1, Z is the conflict number, Y is the executability, which is a score between 0 and 1, evaluated by the automated execution engine, X is the coverage, which represents the requirement coverage ratio;

[0169] The eighth calculation formula is:

[0170] Delta Q > 0.3

[0171] Wherein, Delta Q is the improvement ratio of the fusion quality evaluation function value in the manual correction case, when the formula is satisfied, the optimization path in the knowledge base is updated.

[0172] In the embodiment, the test case template is set, the intention requirement determined by the three-level filter and the basic case number interval output by the test requirement complexity evaluation model are taken as the input, the semantic repeated units are merged by DBSCAN clustering, the case execution success rate and the requirement coverage degree are evaluated, the path is optimized according to the fusion quality, the case generation efficiency is improved, and the specific implementation of the path synchronous update to the knowledge base associated with the error sample library is as follows: taking the transfer function test of a certain financial APP as an example, the intention requirement determined by the three-level filter is "verify the limit and arrival time of real-time transfer between different bank cards", and the basic case number interval output by the test requirement complexity evaluation model is [15, 25]. First, the XML Schema of the test case template is taken as the skeleton, which divides the retrieved knowledge into reusable units of input constraints (such as "transfer-out card type", "transfer-in card type", "transfer amount"), expected output (such as "transfer success prompt", "arrival time display"), and exception handling (such as "exceeding daily limit prompt", "system busy prompt"). For example, the input constraint unit is defined as <inputconstraint>Tag, containing sub-tags CardType, AmountRange, etc., expected output unit defined as <expectedoutput>Tags, including sub-tags such as SuccessMessage, ArrivalTime, etc.

[0173] Next, the semantic duplicate units are merged by DBSCAN clustering. The 20 retrieved knowledge items related to transfer (e.g., "Test the real-time transfer function of ICBC to CCB", "Verify the large transfer limit of ABC to CCB") are converted into word vectors (using the Sentence-BERT model), and the Levenshtein distance between each other is calculated. Set the clustering radius to 0.3, when the distance between two knowledge items is less than 0.3, they are classified into the same cluster. For example, the distance between "Test the real-time transfer function of ICBC to CCB" and "Verify the instant transfer service of ICBC to CCB" is 0.25, which is classified into the same cluster, and the expression "Test the real-time transfer function of ICBC to CCB" with the smallest Levenshtein distance is retained as the representative of the cluster.

[0174] Then, the attention weight matrix of knowledge units and template nodes is added in the GPT fusion module to enhance the fusion strength of key knowledge. For example, the "transfer amount" knowledge unit and the <amountrange>Node association, through self-attention mechanism to calculate attention weight, so that in the generation of test cases, the knowledge related to "transfer amount" can be more accurately filled into the corresponding position of the template. The fusion quality evaluation function is calculated by using the seventh calculation formula Q = a x X + b x Y - g x Z, wherein a = 0.4 (coverage weight coefficient), b = 0.4 (executability weight coefficient), g = 0.2 (conflict number weight coefficient), Z is the number of conflicts (such as the number of times of mismatch between input constraints and expected output), Y is the executability score (evaluated by the automation execution engine according to historical execution results, which is 0.85 here), X is the coverage (demand coverage ratio, which is 0.9 here). Substituting the calculation gives Q = 0.4 x 0.9 + 0.4 x 0.85 - 0.2 x 1 = 0.66, since the Q value is greater than the preset threshold 0.6, the corresponding fusion path (such as "knowledge item A -> template node B") is recorded to the knowledge base.

[0175] If the Q value generated at a certain time is 0.58 (less than 0.6), manual optimization is triggered. The test expert modifies the generated test case, for example, supplements the missing exception handling scenario, and recalculates the Q value to 0.82 after modification, the improvement ratio AQ = (0.82-0.58) / 0.58≈0.41, which satisfies the eighth calculation formula AQ>0.3, and the optimization path in the knowledge base is updated, and "supplementing exception handling scenario" is saved as a new optimization rule. Finally, based on the optimized template and knowledge base, 18 test cases (in the basic case number interval [15, 25]) are generated, covering 8 bank card combinations, 5 transfer amount intervals and 5 abnormal situations, the test case execution success rate is improved from 75% to 92%, the demand coverage is improved from 80% to 95%, and the optimization path is updated to the error sample library associated knowledge base, providing a reference for subsequent similar test requirements.

[0176] According to the second aspect of the embodiment of the application, an automatic software testing system based on multi-modal AI collaboration is provided.

[0177] Figure 8 is a structural diagram of an automatic software testing system based on multi-modal AI collaboration according to an embodiment of the application.

[0178] In one or more embodiments, preferably, the automatic software testing system based on multi-modal AI collaboration comprises:

[0179] The service cluster construction module 801 is configured to encapsulate optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tools and their dependent libraries into independent containers, provide unified analysis services through a gRPC interface, dynamically monitor container CPU and memory load, set load thresholds, wherein the load thresholds are set to 80% CPU usage and 75% memory usage, automatically scale the number of container replicas according to real-time request volume and load thresholds, and input analysis results into a multi-modal semantic correlation model in real time.

[0180] The collaborative correction module 802 is configured to establish a multi-modal semantic correlation model based on a Transformer architecture, and calculate the semantic similarity of image description text and corresponding speech transcription text, wherein the semantic similarity is calculated using a cosine similarity algorithm, and a threshold of 0.7 is set, and manual review is triggered when the similarity is less than 0.7, and confirmed analysis error samples are synchronized to an error sample library, and the analysis model parameters are fine-tuned at 3 a.m. every day to improve the analysis accuracy.

[0181] The supplementary use case module 803 is configured to construct a test requirement complexity evaluation model, take the number of front-end interaction points, the depth of back-end API call chain and the number of boundary conditions as input features, output a basic use case number interval [N_min, N_max], train an XGBoost-based redundancy predictor to predict use case redundancy, and determine use cases with a redundancy higher than 0.6 as redundant and eliminate them, wherein the number of front-end interaction points is counted by summing up clicks, inputs and pop-up windows, the depth of back-end API call chain is calculated by the number of nodes in the longest call path, and the number of boundary conditions is counted by summing up numerical, type and business rule boundaries.

[0182] The use case real-time monitoring module 804 is configured to define use case quality indicators, establish an error sample library to store problem use cases and their correction labels, and provide training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model.

[0183] The three-level retrieval filtering module 805 is configured to set three-level filters for input test requirements, the input is test requirement text preprocessed by the multi-modal semantic correlation model, and finally determines the intended requirements according to the screening results of the three-level filters.

[0184] The fusion optimization module 806 is configured to set a test use case template, take the intended requirements determined by the three-level filters and the basic use case number interval output by the test requirement complexity evaluation model as inputs, and merge semantic repeated units through DBSCAN clustering, wherein the clustering radius is set to 0.3, and the use case execution success rate and requirement coverage are evaluated to optimize the path according to the fusion quality, improve the use case generation efficiency, and update the optimized path to the knowledge base associated with the error sample library.

[0185] According to a third aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer program instructions. The computer program instructions, when executed by a processor, implement the method according to any one of the first aspect of the embodiments of the present application.

[0186] According to a fourth aspect of the embodiments of the present application, an electronic device is provided. Figure 9 is a structural diagram of an electronic device in an embodiment of the present application. Figure 9 The electronic device shown is a general automated software testing device based on multi-modal AI collaboration. Referring to Figure 9 , the electronic device can be a smart phone, a tablet computer, or the like. The electronic device 900 includes a processor 901 and a memory 902. The processor 901 is electrically connected to the memory 902.

[0187] The processor 901 is the control center of the electronic device 900, and connects various parts of the electronic device through various interfaces and lines. The processor 901 executes various functions of the electronic device and processes data by running or calling computer programs stored in the memory 902 and calling data stored in the memory 902, thereby monitoring the entire electronic device.

[0188] In this embodiment, the processor 901 in the electronic device 900 loads the instructions corresponding to the processes of one or more computer programs into the memory 902, and runs the computer programs stored in the memory 902 by the processor 901, thereby realizing various functions.

[0189] In some embodiments, the electronic device 900 can further include a display 903, a radio frequency circuit 904, an audio circuit 905, a wireless fidelity module 906, and a power supply 907. The display 903, the radio frequency circuit 904, the audio circuit 905, the wireless fidelity module 906, and the power supply 907 are electrically connected to the processor 901, respectively.

[0190] The display 903 can be used to display information input by a user or information provided to a user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. The display 903 can include a display panel, which can be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED) in some embodiments.

[0191] The radio frequency circuit 904 can be used for transmitting and receiving radio frequency signals to establish wireless communication with network devices or other electronic devices.

[0192] The audio circuit 905 can be used for providing an audio interface between the user and the electronic device through a speaker and a microphone.

[0193] The wireless fidelity module 906 can be used for short-distance wireless transmission, which can help users send and receive emails, browse websites, access streaming media, and the like, and provides users with wireless broadband Internet access.

[0194] The power supply 907 can be used for powering various components of the electronic device 900. In some embodiments, the power supply 907 can be logically connected to the processor 901 through a power management system, so that the functions of managing charging, discharging, and power consumption management can be realized through the power management system.

[0195] Although Figure 9 The electronic device 900 can also include a camera, a Bluetooth module, and the like, which are not shown in the figure.

[0196] The technical solutions provided by the embodiments of the present application can include the following beneficial effects:

[0197] In the present application, the multi-modal AI collaboration and dynamic optimization are combined to improve the test accuracy.

[0198] In the present application, the container deployment and the intelligent filtering mechanism are combined to significantly improve the test efficiency compared with the traditional method.

[0199] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.< / amountrange> < / expectedoutput> < / inputconstraint>

Claims

1. An automated software testing method based on multi-modal AI collaboration, characterized in that, The method comprises: The optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tool and its dependent library are packaged into an independent container, a unified analysis service is provided to the outside through a gRPC interface, the CPU and memory load of the container is dynamically monitored, a load threshold is set, wherein the load threshold is set as 80% of the CPU usage and 75% of the memory usage, the number of container replicas is automatically scaled according to the real-time request volume and the load threshold, and the analysis result is input into a multi-modal semantic correlation model in real time; A multi-modal semantic correlation model based on a Transformer architecture is established to calculate the semantic similarity of image description text and corresponding speech transcription text, wherein the semantic similarity calculation adopts a cosine similarity algorithm, a threshold of 0.7 is set, manual review is triggered when the similarity is lower than 0.7, and the confirmed analysis error samples are synchronized to an error sample library, and the analysis model parameters are fine-tuned at 3 a.m. every day to improve the analysis accuracy; A test requirement complexity evaluation model is constructed, the front-end interaction point number, the back-end API call chain depth and the boundary condition number are taken as input features, and the output is the basic test case number interval [N_min, N_max], an XGBoost-based redundancy predictor is trained to predict the test case redundancy, and the test case with a redundancy higher than 0.6 is determined as redundant and eliminated, wherein the front-end interaction point number is counted by summing up the click, input and pop-up window, the back-end API call chain depth is calculated by the node number of the longest call path, and the boundary condition number is summed up by the value, type and business rule boundary; A test case quality index is defined, an error sample library is established to store the problem test cases and the corrected labels manually labeled, and the error sample library provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model; Three-level filters are set for the input test requirements, the input is the test requirement text preprocessed by the multi-modal semantic correlation model, and the intended requirements are finally determined according to the screening results of the three-level filters; A test case template is set, the intended requirements determined by the three-level filters and the basic test case number interval output by the test requirement complexity evaluation model are taken as inputs, and the semantic repeated units are merged through DBSCAN clustering, wherein the clustering radius is set as 0.3, the test case execution success rate and the requirement coverage are evaluated, the path is optimized according to the fusion quality, the test case generation efficiency is improved, and the optimized path is updated to the knowledge base associated with the error sample library.

2. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, The optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tool and its dependent library are packaged into an independent container, a unified analysis service is provided to the outside through a gRPC interface, the CPU and memory load of the container is dynamically monitored, a load threshold is set, wherein the load threshold is set as 80% of the CPU usage and 75% of the memory usage, the number of container replicas is automatically scaled according to the real-time request volume and the load threshold, and the analysis result is input into a multi-modal semantic correlation model in real time; The optical character recognition tool, the speech recognition tool and the natural language processing tool are packaged into an independent container. The independent container provides a unified resolution service to the outside through a gRPC interface; Obtain the current CPU usage, memory usage, and corresponding target indicator values; Calculate the target number of replicas using a first calculation formula; Adjust the number of container replicas according to the target number of replicas; The first calculation formula is: Podm=ceil [Podd×(max (CPUd, Memd)÷max (CPUm, Memm))] Wherein, Podm is the target number of replicas, Podd is the current number of replicas, CPUd is the current CPU indicator value, CPUm is the target CPU indicator value, Memd is the current memory indicator value, Memm is the target memory indicator value, and ceil() is the ceiling function; After adjustment, continuously monitor the load changes. When the CPU and memory usage are stable within the target indicator value ±10% range for 3 consecutive monitoring periods, stop the scaling operation.

3. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, The multi-modal semantic correlation model based on the Transformer architecture is established to calculate the semantic similarity between the image description text and the corresponding speech transcription text. The cosine similarity algorithm is used for semantic similarity calculation, and a threshold of 0.7 is set. When the similarity is lower than 0.7, manual review is triggered. The confirmed analysis error samples are synchronized to the error sample library, and the analysis model parameters are fine-tuned every day at 3 am to improve the analysis accuracy. Specifically, it includes: A multi-modal semantic correlation model based on the Transformer architecture is established, which includes an image feature extraction layer, a speech text feature extraction layer, and a cross-attention fusion layer. The multi-modal semantic correlation model is used to calculate the semantic similarity between the image description text and the corresponding speech transcription text, and the cosine similarity algorithm is used for calculation. The similarity is judged once an hour, and the size relationship with the pre-set threshold is determined. When the similarity is lower than the pre-set threshold, manual review is triggered, and the confirmed analysis error samples are injected into the training data set to improve the accuracy of the analysis model by periodically fine-tuning the analysis model parameters. Otherwise, no processing is done.

4. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, The test requirement complexity evaluation model is constructed, taking the front-end interaction point number, back-end API call chain depth, and boundary condition number as input features, and outputting the basic test case number interval [N_min, N_max]. The redundancy predictor based on XGBoost is trained to predict the test case redundancy. Test cases with a redundancy higher than 0.6 are determined as redundant and excluded. Specifically, the front-end interaction point number is counted by clicking, inputting, and pop-up window, and the sum is calculated. The back-end API call chain depth is calculated by the number of nodes in the longest call path. The boundary condition number is calculated by summing the value, type, and business rule boundaries. ​ When the test requirement complexity evaluation model generates a number of use cases that satisfy a second calculation formula, activate the rule engine to filter duplicate use cases and retain non-redundant use cases; otherwise, when a third calculation formula is satisfied, inject a requirement feature vector into the test requirement complexity evaluation model to drive the generation of supplementary use cases and supplement them to the basic use case set; The second calculation formula is: M>1.2N_max Wherein, M is the number of use cases generated by the test requirement complexity evaluation model, and N_max is the maximum value of the basic use case number interval; The third calculation formula is: M<0.8N_min Wherein, M is the number of use cases generated by the test requirement complexity evaluation model, and N_min is the minimum value of the basic use case number interval.

5. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, Define a use case quality indicator, establish an error sample library to store artificially annotated problem use cases and their correction labels, and the error sample library simultaneously provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model. At 2 a.m. every day, based on the newly added samples in the error sample library, fine-tune the full connection layer weight of the large model to optimize the accuracy and coverage of use case generation. The use case quality indicator includes step coverage, abnormal trigger rate, and input parameter boundary violation times. The correction label includes error type and correction suggestion, which specifically includes: Define a use case quality indicator, wherein the use case quality indicator includes step coverage, abnormal trigger rate, and input parameter boundary violation times, wherein the step coverage is calculated based on the business process node coverage ratio, and the abnormal trigger rate is the success rate of abnormal use case execution; Establish an error sample library to store artificially annotated problem use cases and their correction labels, and the label includes error type and correction suggestion; Fine-tune the full connection layer weight of the large model based on the newly added samples in the error sample library every 24 hours; Real-time monitoring of unit time use case generation fluctuation value and structured analysis success rate as the basis for determining whether to enhance the format constraint strength of the generation template; When the unit time use case generation fluctuation value and the structured analysis success rate of the last 3 samples do not satisfy a fourth calculation formula, automatically enhance the format constraint strength of the generation template; The fourth calculation formula is: CY∈[Cmin,Cmax] Wherein, CY is the sampling value, which is the unit time use case generation fluctuation value, Cmax is the maximum value deviating from the preset baseline, and Cmin is the minimum value deviating from the preset baseline.

6. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, Set a three-level filter for the input test requirement, the input is the test requirement text preprocessed by the multi-modal semantic correlation model, and finally determine the intended requirement according to the screening results of the three-level filter, specifically including: Use the BERT model to extract the intent vector of the input test requirement text; Set a first-level filter, which is used to screen test requirement items that satisfy a fifth calculation formula, and the test requirements corresponding to the intent vector of the first-level filter are used as the remaining first-level test requirements; The second filter is configured to check compatibility of the knowledge item with the device type and the operating system version of the current test scene, and to filter the first test requirement by querying a preset compatibility list, and the test requirement corresponding to the intention vector of the second filter is taken as the remaining second test requirement; The third filter is configured to identify candidate knowledge items having logical contradictions with the selected item based on a knowledge conflict detection model, output the item satisfying a sixth calculation formula to an isolation area, filter the second test requirement by the third filter, and take the test requirement corresponding to the intention vector of the third filter that is not moved to the isolation area as the finally determined intention test requirement; The fifth calculation formula is: ZXS>0.8 wherein ZXS is the knowledge vector similarity; The sixth calculation formula is: PC>0.7 wherein PC is the conflict probability, which is output by the knowledge conflict detection model.

7. The automated software testing method based on multi-modal AI collaboration of claim 1, wherein, The test case template is set, the intention requirement determined by the third filter and the basic case interval output by the test requirement complexity evaluation model are taken as inputs, and the semantic repeated units are clustered and merged by DBSCAN, wherein the clustering radius is set to 0.3, the execution success rate and the requirement coverage are evaluated, the path is optimized according to the fusion quality, the test case generation efficiency is improved, the optimized path is updated to the knowledge base associated with the error sample library, and specifically includes: The XML Schema of the test case template is taken as a skeleton, wherein the skeleton disassembles the retrieved knowledge into reusable units of input constraints, expected outputs and exception handling; The semantic repeated units are clustered and merged by DBSCAN, and the expression with the minimum Levenshtein distance is retained, and the clustering radius is set to 0.3; The attention weight matrix of the knowledge unit and the template node is added to the GPT fusion module to enhance the fusion strength of the key knowledge; The fusion quality evaluation function is calculated by using a seventh calculation formula; When the Q value is greater than a preset threshold, the corresponding fusion path is recorded to the knowledge base, otherwise manual optimization is triggered; The seventh calculation formula is: Q=α×X+β×Y-γ×Z wherein Q is the fusion quality evaluation function calculation value, α is the coverage weight coefficient, β is the executability weight coefficient, γ is the conflict number weight coefficient, and α+β+γ=1, Z is the conflict number, Y is the executability, which is a score between 0 and 1, evaluated by the automatic execution engine, X is the coverage, which represents the requirement coverage ratio; The eighth calculation formula is: △Q＞0.3 wherein ΔQ is the improvement ratio of the fusion quality evaluation function calculation value in the manual correction case, and when the formula is satisfied, the optimized path in the knowledge base is updated.

8. An automated software testing system based on multi-modal AI collaboration, characterized in that, The system is used to implement the method of any one of claims 1-7, and the system comprises: The service cluster construction module encapsulates optical character recognition (OCR), automatic speech recognition (ASR), natural language processing (NLP) analysis tools and their dependent libraries into independent containers, provides unified analysis services to the outside through a gRPC interface, dynamically monitors the CPU and memory load of the containers, sets a load threshold, wherein the load threshold is set to 80% CPU usage and 75% memory usage, automatically scales the number of container replicas according to real-time request volume and load threshold, and inputs the analysis results into a multi-modal semantic correlation model in real time. The collaborative correction module establishes a multi-modal semantic correlation model based on a Transformer architecture, calculates the semantic similarity between image description text and corresponding speech transcription text, wherein the semantic similarity calculation uses a cosine similarity algorithm, and sets a threshold of 0.7, triggering manual review when the similarity is less than 0.7, confirmed analysis error samples are synchronized to the error sample library, and the analysis model parameters are fine-tuned at 3 a.m. every day to improve the analysis accuracy. The supplementary use case module constructs a test requirement complexity evaluation model, taking the number of front-end interaction points, the depth of back-end API call chains, and the number of boundary conditions as input features, and outputting a basic use case number interval [N_min, N_max]. An XGBoost-based redundancy predictor is trained to predict use case redundancy. Use cases with a redundancy higher than 0.6 are determined as redundant and excluded. The number of front-end interaction points is counted by clicking, inputting, and pop-up windows, and then summed. The depth of back-end API call chains is calculated by the number of nodes in the longest call path. The number of boundary conditions is counted by value, type, and business rule boundaries, and then summed. The use case real-time monitoring module defines use case quality indicators, establishes an error sample library to store problem use cases and their correction labels, and provides training data for the multi-modal semantic correlation model and the test requirement complexity evaluation model. The three-level retrieval filtering module sets three filters for the input test requirements. The input is test requirement text preprocessed by the multi-modal semantic correlation model. The final intended requirements are determined based on the screening results of the three filters. The fusion optimization module sets a test case template, taking the intended requirements determined by the three filters and the basic use case number interval output by the test requirement complexity evaluation model as input. It clusters and merges semantic repeated units through DBSCAN clustering, with a clustering radius of 0.

3. Based on the use case execution success rate and requirement coverage, it evaluates and optimizes the path according to the fusion quality, improves the use case generation efficiency, and updates the optimized path to the knowledge base associated with the error sample library.

9. A computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1-7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any one of claims 1-7.

Images