Automatic testing method and device based on requirement semantic analysis, medium and product

Abstract

The embodiments of the present disclosure disclose an automatic testing method and device based on demand semantic analysis, a medium and a product. The method comprises: modifying the target business code based on the business demand and the code management branch, obtaining the code difference content after the modification is completed; calling an AI large model to perform semantic analysis on the code difference content to generate a code change analysis result; generating a test email based on the code change analysis result; and testing the target business system based on the test email to obtain a test report. The method can realize full-process automation closed loop from demand to test through demand-driven branch management and AI code semantic analysis, greatly reduce the cost of manual intervention and communication, and significantly improve the test accuracy and business quality control efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of automated testing technology, and in particular to an automated testing method, apparatus, medium, and product based on requirement semantic analysis. Background Technology

[0002] In typical DevOps practices, the industry commonly adopts a combined approach of a project management system (JIRA) and a remote repository (GitLab). The JIRA system manages the entire process of requirements and defects, while the remote repository handles code hosting and version control. The two platforms synchronize data fields via webhooks and integrate static code analysis tools (such as SonarQube and Checkstyle) into the code merge request process to scan and detect code syntax, coding standards, and security vulnerabilities. This solution achieves seamless integration of requirements and code processes, establishes a fundamental R&D quality control mechanism, and is currently the mainstream approach for integrating R&D processes.

[0003] However, the above integration methods only reach the field-level data mapping and synchronization, failing to understand the business semantics corresponding to code changes. Branch management relies on manual operation, and inconsistent naming can easily lead to version confusion. Information such as requirements, code, defects, and test cases are scattered across different systems, requiring manual aggregation of test-related information, which can easily result in test omissions. Furthermore, static analysis only focuses on code syntax and style, lacking in-depth analysis of requirement semantics and the scope of business impact. Communication costs between personnel in different roles are high, and there is a significant risk of missed tests and quality issues. Summary of the Invention

[0004] In view of this, the present disclosure provides an automated testing method, apparatus, medium, and product based on requirement semantic analysis, which can realize a fully automated closed loop from requirements to testing through requirement-driven branch management and AI code semantic analysis, significantly reducing manual intervention and communication costs, and significantly improving testing accuracy and business quality control efficiency.

[0005] In a first aspect, embodiments of this disclosure provide an automated testing method based on requirement semantic analysis, employing the following technical solution: Based on the business requirements and the code management branch, the target business code is modified, and after the modification is completed, the code difference content is obtained. The AI ​​model is invoked to perform semantic analysis on the code differences, generating code change analysis results. Based on the code change analysis results, a test submission email is generated; Based on the test request email, the target business system is tested, and a test report is obtained.

[0006] Optionally, obtaining business requirements and creating code management branches based on those requirements includes: A browser plugin is embedded in the project management system page. When business requirements are entered into the project management system, the browser plugin is used to extract the business requirements from the project management system page, assign a unique number and category identifier to the business requirements, and send a branch creation request to the remote repository interface. Based on the branch creation request, a code management branch is created using the remote repository, and a branch identifier for the code management branch is constructed based on the unique number and the category identifier.

[0007] Optionally, the code change analysis results include at least one of the following: code save record summary, risk level, risk checklist, list of affected business modules, and risk warning; Key content from the code change analysis results is selected and synchronously written to the remote repository merge request description, browser plugin local cache, and project management system activity log.

[0008] Optionally, generating a test submission email based on the code change analysis results includes: A test submission button is embedded in the page of the project management system. When the test submission button is triggered, the unique number of the business requirement, the branch identifier of the code management branch, and the description of the remote repository merge request are encapsulated into JSON format data. Based on the JSON format data and the code change analysis results, the test email template variables are populated, and a test email is generated and sent.

[0009] Optionally, the automated testing method based on requirement semantic analysis further includes: When the risk level exceeds the preset level, a message reminder is pushed to the developer's terminal; The content of the test submission email is subject to risk assessment. When the assessment result is high risk, a message reminder is sent to the tester's terminal.

[0010] Optionally, the automated testing method based on requirement semantic analysis further includes: When it is detected that the test email has been edited, the differences in the email content are obtained; When it is identified that the email difference content contains revised content based on the code change analysis results and that the revised content is valid, a fine-tuning sample is generated based on the revised content; Based on the fine-tuning samples, the large AI model is fine-tuned; When new code differences are obtained, the finely tuned AI model is invoked to perform semantic analysis on the new code differences.

[0011] Optionally, the automated testing method based on requirement semantic analysis further includes: Prioritize the target business system issues contained in the test report; When the priority of the target business system problem is greater than the preset level, new business requirements are obtained based on the target business system problem; Based on the new business requirements, the target business system was tested again.

[0012] Secondly, this disclosure also provides an automated testing system based on requirement semantic analysis, employing the following technical solution: Create a module to obtain business requirements, and create a code management branch based on the business requirements; The modification module is used to modify the target business code based on the business requirements and the code management branch, and obtain the code difference content after the modification is completed; The analysis module is used to call an AI big data model to perform semantic analysis on the code differences and generate code change analysis results. The generation module is used to generate a test submission email based on the code change analysis results. The testing module is used to test the target business system based on the test request email and obtain a test report.

[0013] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform any of the above-described automated testing methods based on requirement semantic analysis.

[0014] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute any of the above-described automatic testing methods based on demand semantic analysis.

[0015] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0016] The automated testing method based on requirement semantic analysis provided in this disclosure first automatically creates code management branches according to business requirements, then modifies the target business code based on the requirements and branches and extracts the code differences. This achieves strong binding and unified management of requirements, branches, and code changes, avoiding naming confusion and version inconsistencies caused by manual branch creation, and ensuring a one-to-one correspondence between code changes and business requirements. By calling a large AI model to perform semantic analysis on code differences, it can understand the code changes at the business semantic level, rather than just performing shallow checks such as syntax and formatting. This achieves a precise association between code changes and business intent, overcoming the shortcomings of traditional static analysis in that it cannot understand business semantics and lacks sufficient analytical depth. This solution automatically generates test request emails based on AI-generated code change analysis results. It automatically collects and structures scattered requirements, code, and test information, eliminating the need for manual summarization, organization, and communication. This reduces cross-role communication costs and the probability of human error. Based on the test request emails, it then conducts tests on the target business system and outputs test reports. This forms a complete automated testing chain from requirements gathering, branch management, code modification, semantic analysis to automatic test request and test execution. It improves the automation and standardization of the testing process, reduces test omissions and quality risks from the process perspective, and improves the completeness and reliability of test coverage.

[0017] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the automated testing method based on requirement semantic analysis provided in this embodiment of the disclosure; Figure 2 A flowchart illustrating the method for generating test emails provided in this embodiment of the disclosure; Figure 3 This is a deployment topology diagram provided for embodiments of the present disclosure; Figure 4 A flowchart illustrating the AI ​​large model fine-tuning method provided in this embodiment of the disclosure; Figure 5 This is a flowchart illustrating the iterative testing method based on test reports provided in an embodiment of this disclosure. Figure 6 A schematic diagram of an automated testing system based on demand semantic analysis provided in this embodiment of the disclosure; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0020] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0021] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0023] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0024] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0025] Reference Figure 1This disclosure provides an automated testing method based on requirement semantic analysis, including the following steps: S1: Obtain business requirements and create a code management branch based on those requirements; S2: Based on the business requirements and the code management branch, modify the target business code, and after the modification is completed, obtain the code difference content; S3: Call the AI ​​large model to perform semantic analysis on the code differences and generate code change analysis results; S4: Based on the code change analysis results, generate a test submission email; S5: Based on the test request email, conduct tests on the target business system and obtain a test report.

[0026] This disclosed automated testing method based on requirement semantic analysis first automatically creates code management branches according to business requirements, then modifies the target business code based on the requirements and branches, and extracts the code differences. This achieves strong binding and unified management of requirements, branches, and code changes, avoiding naming confusion and version inconsistencies caused by manual branch creation, and ensuring a one-to-one correspondence between code changes and business requirements. By calling a large AI model to perform semantic analysis on code differences, it can understand the code changes at the business semantic level, rather than just performing shallow checks such as syntax and formatting. This achieves a precise association between code changes and business intent, overcoming the shortcomings of traditional static analysis in that it cannot understand business semantics and lacks sufficient analytical depth.

[0027] This solution automatically generates test request emails based on AI-generated code change analysis results. It automatically collects and structures scattered requirements, code, and test information, eliminating the need for manual summarization, organization, and communication. This reduces cross-role communication costs and the probability of human error. Based on the test request emails, it then conducts tests on the target business system and outputs test reports. This forms a complete automated testing chain from requirements gathering, branch management, code modification, semantic analysis to automatic test request and test execution. It improves the automation and standardization of the testing process, reduces test omissions and quality risks from the process perspective, and improves the completeness and reliability of test coverage.

[0028] In S1, business requirements are written into the project management system (JIRA). JIRA, developed by Atlassian, is a project and task tracking tool primarily used for requirements management, defect tracking, and agile development collaboration. JIRA can be simply understood as a "work order system for development teams." A browser plugin is created for the project management system page. This plugin is a small piece of code written using front-end technologies (HTML / CSS / JS), packaged, and installed in browsers such as Chrome, Edge, and Firefox to "add functionality" to the webpage or the browser itself. This browser plugin is used to obtain the document element (webpage source code) of the entire project management system page and filter out peripheral elements. For example, the top navigation bar, right sidebar, and bottom content are removed, leaving only the core content of the business requirements. To facilitate management, each business requirement is assigned a unique number (JIRA number) and a category identifier. Business requirement types include new features and defect fixes. The category results can be determined by analyzing business requirements using a large AI model (such as DeepSeek). Specifically, AI-based identification of task requirement types can be achieved by recognizing keywords. Keywords such as "new" indicate a new feature; keywords such as "fix" indicate a defect fix. The types can be further subdivided, for example: feature (requirement / new function), fix (online defect), hotfix (emergency hotfix), refactor (code refactoring with no business changes), test (only adding test cases), and doc (only document updates), etc.

[0029] The unique ID dynamically increments based on the number of business requirements. For example, [Requirement ID: 10001]; Title: Trigger a graphic verification code when a user logs in with an incorrect password more than 5 times; Description: To prevent brute-force attacks, it is necessary to add a trial-and-error count to the login interface...

[0030] After analyzing business requirements, the backend server's API service (Application Programming Interface service) calls a remote repository (such as GitLab) to create a code management branch (also known as a Git branch). GitLab is an open-source platform for code hosting, collaborative development, continuous integration / continuous delivery (CI / CD), and DevOps lifecycle management. The code management branch is an independent workline in Git version control, automatically created through the GitLab API (a programmatic interface provided by GitLab). For ease of recording and management, a unified naming convention is set for code management branches; branch naming follows... <type>-<unique identifier> format, where <type>This serves as a category identifier for business requirements, with <Unique Number> being the unique identifier for that requirement. The name should be all lowercase, separated by hyphens, and must not contain spaces or version numbers. Example: feature-10001.

[0031] In S2, the remote repository sends code management branches to the developer's terminal. Developers can log in to the project management system from the terminal, view business requirements on the project management system page, clarify the functions to be implemented or the defects to be fixed, view the code management branch with the same unique number as the business requirement on the project management system page, write code from that branch, and push the code to the remote repository via `git push` after the code is written. After receiving the push, the remote repository triggers a webhook, sending the push event to the ` / webhook / gitlab` interface (GitLab webhook address) of the API service (in this functional context, it can also be called the AI ​​middleware service). Both the AI ​​large model and the GitLab API reside in the API service. The API service first verifies the X-GitLab-Token in the request header to ensure the source is trustworthy, then writes the event data to Redis Stream or Kafka (high-performance message queue), and immediately returns an HTTP 200 response to prevent the remote repository from timing out. Afterwards, the API service consumes events from the message queue and calls the GitLab API: GET / projects / :id / repository / compare?from=oldrev&to=newrev to obtain a comparison of the project repository from the old version to the new version, and obtains a unified Diff result that includes added, modified, and deleted files as well as line-level diffs. This Diff result is the difference content.

[0032] There is usually more than one developer. For example, since modern software uses a front-end and back-end separation architecture, the developers include front-end developers and back-end developers. All developers can be allowed to view the created code management branches and business requirements from the project management system page, and then discuss offline who will be responsible for the code editing tasks. Alternatively, a developer can be designated from the beginning, and only the designated developer can be given access to view the code management branches and business requirements. New task reminders can be issued, and the designated developer will be responsible for the subsequent code editing tasks, thus achieving decoupled development and avoiding risk isolation.

[0033] Optionally, after business requirements are written into JIRA, the authenticity and rationality of the business requirements are first evaluated, and unreasonable or pseudo-requirements without clear business value are eliminated. For the real and valid requirements that pass the evaluation, two allocation methods are supported: online and offline. Online, business requirements can be directly assigned to the corresponding developer accounts and bound in JIRA. Offline, business requirements can be allocated manually. Both methods can enter the subsequent development process.

[0034] In S3, the differences and context information are encapsulated into a JSON request body. An example of a request body is as follows: { "diff": "...", / / Code difference text "jiraKeys": ["PROJ-10001"], / / Parse from branch name or commit message "lang": "java,js", / / File extensions will be automatically recognized. "maxTokens": 800 / / Maximum output length of the model } The JSON request body is sent as an inference parameter to the AI ​​model. The AI ​​model is then invoked, and semantic analysis prompts are used to guide it in performing semantic analysis of the discrepancies. The output of the code change analysis results includes at least one of the following: a code save record summary, a risk level, a risk checklist (also known as a test regression checklist), a list of affected modules (referring to the modified code modules), and a risk observation. The code save record summary provides a brief overview of the code modifications and their purpose; the risk level assesses the degree of risk based on the scope of the code modifications and their business impact; the risk checklist outlines the key verification items after the code modifications, also called a test regression checklist; the list of affected modules refers to the business modules involved in and affected by this code modification; and the risk observation outlines potential problems and testing considerations related to the code modification.

[0035] The semantic analysis prompt template is as follows: "You are a senior technical manager. Please summarize the following code changes in Chinese: 1. Business impact; 2. Impact of front-end and back-end service and technology changes; 3. Scope of test and regression impact (listed item by item); 4. Risk level (high / medium / low)." The AI ​​model returns a structured code storage record summary, risk level, and risk checklist in JSON format. An example of the returned format is shown below. { "summary": "Fixed the null pointer in the order placement interface and added inventory verification", "checkList": "

Interface

Page

Unit

[0036] The intermediate service selects key contents from the code change analysis results and synchronously writes them into the description of the remote repository merge request (GitLab MR), the local cache of the browser plugin, and the activity log of the project management system. The key contents are selected according to actual needs. For example, the key contents include the code save record summary, risk level, and risk check list. The description of the remote repository merge request refers to the "Description" column in the remote repository merge request, which is used to display information such as the key points of the changes, risks, and test lists for reviewers to quickly understand the changes. For the description of the remote repository merge request, call the update project merge request (PUT / projects / :id / merge_requests / :iid), and append the AI block at the end of the description field (description) of this merge request, and display the results returned by the AI large model using a Markdown table (Markdown format table); for the local cache of the browser plugin, write the results into LocalStorage (local storage) to achieve instant page opening and offline readability; for the activity log of the project management system, use the POST / rest / api / 3 / issue / {key} / comment (JIRA comment interface) to write the results returned by the AI large model as a comment into the corresponding requirement to complete the context closed-loop of requirement-code-defect.

[0037] ​Through the above process, the system achieves a complete automated chain of "code push → difference acquisition → semantic analysis → multi-terminal result injection," providing clear risk guidance and traceable records for subsequent testing, code review, and deployment. This solution is equivalent to equipping the remote repository with an AI quality inspector. Every time someone pushes code, it automatically unpacks and inspects it, and pastes a "human-readable" inspection report onto the project management system, while also writing a remote repository merge request description for easy reference during code review.

[0038] Writing the code change analysis results returned by the AI ​​model into the merge request description field enables mandatory visibility, mandatory traceability, standardized formatting, and mandatory consensus of information. The merge request description is displayed at the top of the page by default, ensuring that reviewers cannot overlook relevant information; it is displayed in Markdown table format, which is convenient for reviewers to read quickly and supports machine parsing; the description content is permanently saved with the merge request and is not easily tampered with, facilitating subsequent problem investigation and version auditing; at the same time, it can establish a mandatory standard in the code review process to view the AI ​​analysis results before conducting code review, effectively avoiding blind review and improving the quality and security of code review.

[0039] Optionally, to prevent code snippets from being directly leaked to the AI ​​model, a code obfuscation process is added. Before inputting the code differences into the AI ​​model, the code differences are first encrypted and obfuscated, and then the obfuscated content is sent to the AI ​​model for semantic analysis.

[0040] A new MCP (Model Control Plane) compatible operation mechanism for AI large models has been added. This mechanism uniformly summarizes and manages the code content reading permissions of remote repositories. After code is committed, the MCP component automatically intercepts the developer's code commit request, reads the committed code content and transmits it to the AI ​​large model for analysis, effectively solving the problem of insufficient read permissions when the AI ​​large model directly accesses the code repository.

[0041] In S4, refer to Figure 2 The flowchart illustrating the method for generating test submission emails shows that "generating test submission emails based on the code change analysis results" includes the following steps: S41: Embed a test submission button in the page of the project management system. When the test submission button is triggered, encapsulate the unique number of the business requirement, the branch identifier of the code management branch, and the description of the remote repository merge request into JSON format data. S42: Based on the JSON format data and the code change analysis results, populate the test email template variables, generate and send the test email.

[0042] In S41 and S42, a standardized "Submit Test" button is embedded in the DOM node in the upper right corner of the project management system's requirement details page via a browser plugin's content script. The button's style is consistent with the page's native style to avoid interfering with developers' normal operations. A click event is bound to this "Submit Test" button. When a developer triggers the test operation by clicking the mouse, the browser plugin immediately executes the front-end information collection logic, accurately extracting three core data items: the unique ID of the business requirement (JIRAKey), the branch identifier of the code management branch, and the description of the remote repository merge request. These are then encapsulated into a standardized JSON request body, carrying authentication information and a request ID to ensure secure transmission and traceability. The body is then sent to a pre-defined intermediate service interface via an HTTPS POST request, completing the front-end data reporting.

[0043] After receiving a POST request from the browser plugin, the middleware service first verifies the request's validity and data integrity. If the verification passes, it parses three core pieces of information from the JSON data: a unique identifier, a branch identifier, and a description of the remote repository merge request. Then, using the unique identifier as the search key, the middleware service calls the JIRA REST API to retrieve the title and description of the corresponding business requirement. Simultaneously, it performs semantic analysis on the requirement description using an AI big data model, optionally generating a requirement priority. Using the branch identifier and the remote repository merge request description as search criteria, it calls the GitLab API to retrieve the corresponding MapReduce link and associated commit records. It also synchronously loads pre-stored code change analysis results, including a summary of code save records generated by the AI ​​big data model, a Markdown-formatted risk checklist (automatically converted to HTML), and a list of affected business modules. Based on the list of affected business modules, it automatically maps the R&D personnel configuration table, matching the corresponding product owner, test owner, and development owner as email recipients.

[0044] After completing the full data aggregation, the intermediate service loads a pre-defined standardized test submission email template, populating the corresponding template variables with all the aggregated data. The email subject is generated by concatenating the following characters: "[Test Submission] + Unique Number + Business Requirement Title + Sender Name". The email body renders the requirement information, MR link, code save record summary, risk checklist, list of affected business modules, and relevant personnel information into a structured HTML table, ensuring the email content is clear, standardized, and easy to read. Finally, the intermediate service calls the company's SMTP email service, using a pre-defined sender account and a pre-configured recipient list, to send the rendered test submission email. After the email is sent, the intermediate service records the operation log and returns the execution result to the browser plugin, completing the entire process of generating and sending the test submission email.

[0045] Optionally, when the risk level exceeds a preset level, a message reminder is pushed to the developer's terminal, enabling them to promptly rectify code risks; the content of the test submission email is risk-assessed, and when the assessment result is high-risk, a message reminder is sent to the tester's terminal, enabling them to conduct targeted testing. Precise reminders for these two roles achieve layered risk management, improving code delivery quality and testing efficiency. The message reminders are sent through a preset messaging software, such as DingTalk or Lark. For example, if the risk level is high, an additional DingTalk WebHook is invoked to instantly @ the relevant responsible person. When using DingTalk robot Markdown cards for message notification, the message content template is as follows: Title: [Test Submission] JIRA Title (Unique Number) - 10001 - [Current Operator] Contents: / Requirement JIRA URL / Test Address / Changes and Scope of Impact / Risk Level / MR Link Button: One-click jump to the project management system, remote repository, and test environment address. In S5, after sending a test request email to testers, test reports are obtained through actual testing, and problems in the target business system can be found. The problems are sent to the project management system for aggregation, and then a reminder to fix the defects is sent through the preset messaging software, so that developers know that there are bugs in the code they have written and that they need to be fixed. When the bugs are not serious, the code can be deployed and run to meet the required business requirements. After deployment, developers fix the bugs in sync.

[0046] When the risk checklist contains known defects, the service checks if the checklist contains preset keywords. If so, the intermediate service immediately calls the JIRA API to create a subtask for the business requirement, setting the subtask type to "Repair Defect," and automatically assigns it to the responsible person for the corresponding module. This eliminates the need for manual submission of orders, synchronizing the defect to be repaired to the project management system and achieving defect closure (similar to the previous method, a reminder to repair the defect is sent via a preset messaging software). Following the same handling method as the business requirement, after the subtask is created, its unique ID is written back to the GitLabMR description, forming a two-way traceability. For unknown defects, only manual handling is required.

[0047] Reference Figure 3 The deployment topology diagram shown illustrates how the browser (JIRA page) interacts with the middleware services via HTTPS JSON. The browser plugin is used to obtain business requirements, render buttons and pop-ups. The middleware services include AI large model service, GitLab service, and notification service. The interface path for calling the AI ​​large model is / api / analyze, the interface path for calling GitLab is / api / createBranch, the interface path for receiving push events is / webhook / gitlab, and the interface path for various notification services specifically orchestrating and dispatching the testing process is / api / createTestNotice.

[0048] Furthermore, referring to Figure 4 The flowchart illustrating the AI ​​large-scale model fine-tuning method demonstrates the following steps for fine-tuning the AI ​​large-scale model based on the modified test submission email: S6: When it is detected that the test email has been edited, obtain the differences in the email content; S7: When it is identified that the email difference content contains revised content of the code change analysis results and the revised content is valid, a fine-tuning sample is generated based on the revised content; S8: Based on the fine-tuning samples, fine-tune the AI ​​large model; S9: When new code differences are obtained, the fine-tuned AI model is invoked to perform semantic analysis on the new code differences.

[0049] The AI ​​large-scale model uses CodeT5-base (a 220M-parameter Encoder-Decoder Transformer architecture) as its base model and is deployed privately. This private deployment includes model lightweighting, multi-task joint optimization, and representation enhancement. Model lightweighting involves fully freezing the backbone network, inserting only dual LoRA adapters (rank=8, α=16), resulting in trainable parameters accounting for only 0.7% of the total parameters. On a single RTX-3090 card, fine-tuning for one epoch takes ≤18 minutes, avoiding catastrophic forgetting. Multi-task joint optimization involves designing the total loss function as follows: in, This represents the total loss function of a large AI model. This represents the subtask index, indicating different task types that the model optimizes in parallel. Indicates the first The homoscedasticity uncertainty weights of each subtask, i.e., the weights of the AI ​​large model for the first subtask. The quantification of the prediction credibility of each sub-task is automatically learned by the large AI model, rather than being set manually. Indicates the first Dynamic weighting coefficients for each subtask; Indicates the first The loss value of each sub-task; Indicates a regularization term to prevent Approaching 0 results in unbounded weights, ensuring gradient balance.

[0050] The subtasks include four categories, namely, business domain classification loss. Change type classification loss Risk entity labeling loss And test prompts generate loss Business domain classification loss is used to identify the business domain to which the code change belongs (e.g., payment, order, user); change type classification loss is used to determine the type of code change (e.g., addition, deletion, modification, refactoring); risk entity labeling loss is used to locate risk entities in the code change (e.g., high-risk functions, core logic lines); and test prompt generation loss is used to optimize the generation quality of test prompts and checklists.

[0051] In S6, within the DOM node of the test email preview box, an invisible AI-generated segment tag (ai-seg, formatted as follows) is embedded for the code change analysis result paragraph.<ai-segid="p_i"> Establish DOM-level responsibility anchors to monitor only the areas with the specified tag. Listen for character-level content changes within the preview box using the MutationObserver API, recording the differences between the original and edited content (i.e., email differences, denoted as Δ) in real time. Generate a character-level difference log, binding Δ with the corresponding ai-seg tag ID, email unique identifier, and edit timestamp, and temporarily storing it in a local cache to provide original evidence for subsequent verification.

[0052] In S7, the difference content is filtered based on the ai-seg tag, retaining the revised content of the code change analysis result paragraph, and excluding non-model output content such as developer signatures, copies, and formatting notes. A dual-gated validation method is used to verify the validity of revisions: First, a symbolic conflict check is performed between the source code corresponding to the revision and the Abstract Syntax Tree (AST) of the remote repository. If the code change logic described in the revision, the related functions / variables, etc., are completely consistent with the actual code changes reflected in the AST difference, then the revision is considered valid. If there are logical contradictions, symbolic errors, or descriptions lacking corresponding code change support, then it is considered invalid. Second, a sentence-level BERT model (SBERT) is used to calculate the semantic similarity between the revision and the project management system's acceptance criteria. Only revisions with a semantic similarity greater than or equal to a preset similarity value (e.g., 0.85) are retained. If the semantic similarity is less than the preset similarity value, the revision is considered to have a business logic deviation from the acceptance criteria and is therefore invalid. This ensures that the final retained revisions both conform to the actual code changes and are consistent with business acceptance requirements. For valid differences that pass the dual-gated validation, a fine-tuning sample T=(p) in triplet format is generated. i ,p i ′,F diff ), where p i p represents the code change analysis result of the i-th segment output by the large AI model. i ′ corresponds to p i The paragraph was revised by the developers, F diff For the difference features that pass the validity verification, the fine-tuned samples are written into the sample pool; the revisions that fail the verification are marked as negative samples and stored in the sample pool simultaneously, but negative samples are stored separately.

[0053] In S8, when the sample pool capacity is greater than or equal to the preset capacity (e.g., |D) daily When |≥32), the LoRA (Low-Rank Adaptation) incremental update process is triggered, using a preset dual LoRA adapter (rank=8, α=16), with 1 epoch as the training round. Contrastive learning margin-loss is used to incorporate negative samples to prevent overfitting. After training, LoRA weights are merged using EMA (Exponential Moving Average), and hot-swap of model weights is completed within 24 hours without restarting the inference container, achieving zero-downtime updates. The sample source, training parameters, and performance metrics for each fine-tuning are recorded, generating a version snapshot that supports rollback to historical versions.

[0054] In S9, when a remote repository receives a new code push and generates code diff content, the intermediate service obtains the diff content through WebHook and automatically calls the latest fine-tuned private AI big model. During the model inference stage, the ONNXRuntime (an inference engine) framework and INT8 (8-bit integer) quantization are used to compress the AI ​​big model size by 48%, reduce the CPU single-core inference latency to 168ms, and achieve a QPS (queries per second) of ≥120, meeting the remote repository WebHook 3-second timeout constraint. Based on the fine-tuned private knowledge, the AI ​​big model outputs new code change analysis results. It also supports hot-swapping of new LoRA adapters such as "security vulnerabilities" and "performance degradation" to horizontally expand analysis capabilities.

[0055] The AI ​​model employs a total loss function at the main task layer for foundational training and routine fine-tuning. At the sample layer, boundary loss is used as an auxiliary loss for contrastive learning, specifically handling the distinction between valid and invalid revision samples. This prevents the model from overfitting to incorrect human revisions, representing incremental fine-tuning driven by email differences. The combination of these two loss functions ensures both the model's fundamental capabilities and its ability to accurately learn valid human revisions, thus improving model adaptability.

[0056] Further: Refer to Figure 5 The flowchart illustrating an iterative testing method based on test reports demonstrates the following steps for conducting iterative testing of a target business system by evaluating test reports: S10: Obtain the priority of the target business system issues contained in the test report; S11: When the priority of the target business system problem is greater than the preset level, new business requirements are obtained based on the target business system problem; S12: Based on the new business requirements, the target business system is tested again.

[0057] In sections S10-S12, an evaluation model is designed to automatically prioritize issues in the target business system reported in the test report. Alternatively, testers can comprehensively score these issues based on dimensions such as impact scope, functional importance, crash probability, data risk, and ease of reproduction, assigning priority levels and obtaining the final priority for each target business system issue. This final priority is then compared with a preset high-risk level. When a target business system issue is determined to have a higher priority than the preset level, new business requirements are automatically extracted and generated based on the issue's symptoms, impact scope, reproduction steps, and related business information. These requirements are then synchronized to the project management system to form traceable and executable requirement entries. When a target business system issue is determined to have a lower priority than the preset level, no new business requirements need to be generated. Based on this new business requirement, the system restarts the regression testing and key verification process for the target business system, completing the closed-loop processing and quality assurance of high-priority issues.

[0058] Reference Figure 6 This disclosure provides an automated testing system based on requirement semantic analysis, including: Create module 101 to obtain business requirements and create a code management branch based on those requirements; Modification module 102 is used to modify the target business code based on the business requirements and the code management branch, and obtain the code difference content after the modification is completed; Analysis module 103 is used to call an AI big model to perform semantic analysis on the code differences and generate code change analysis results; The generation module 104 is used to generate a test submission email based on the code change analysis results. The testing module 105 is used to test the target business system based on the test request email and obtain a test report.

[0059] The various variations and specific examples of the automatic testing method based on requirement semantic analysis provided above are also applicable to the automatic testing system based on requirement semantic analysis provided in this disclosure. Through the foregoing detailed description of the automatic testing method based on requirement semantic analysis, those skilled in the art can clearly understand the implementation method of the automatic testing system based on requirement semantic analysis. For the sake of brevity, it will not be described in detail here.

[0060] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0061] The processor may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the automated testing method based on requirement semantic analysis of the foregoing embodiments of this disclosure.

[0062] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0063] like Figure 7 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 7 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0064] like Figure 7 As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0065] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 7 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0066] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from ROM. When the computer program is executed by a processor, all or part of the steps of the automated testing method based on requirement semantic analysis according to embodiments of this disclosure are performed.

[0067] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0068] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the automated testing methods based on requirement semantic analysis described in the foregoing embodiments of the present disclosure are performed.

[0069] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0070] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0071] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0072] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0073] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0074] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0075] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0076] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0077] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.< / type> < / type>

Claims

1. An automated testing method based on requirement semantic analysis, characterized in that, include: Obtain business requirements, and create code management branches based on those requirements; Based on the business requirements and the code management branch, the target business code is modified, and after the modification is completed, the code difference content is obtained. The AI ​​model is invoked to perform semantic analysis on the code differences, generating code change analysis results. Based on the code change analysis results, a test submission email is generated; Based on the test request email, the target business system is tested, and a test report is obtained.

2. The automatic testing method based on requirement semantic analysis according to claim 1, characterized in that, The process of obtaining business requirements and creating code management branches based on those requirements includes: A browser plugin is embedded in the project management system page. When business requirements are entered into the project management system, the browser plugin is used to extract the business requirements from the project management system page, assign a unique number and category identifier to the business requirements, and send a branch creation request to the remote repository interface. Based on the branch creation request, a code management branch is created using the remote repository, and a branch identifier for the code management branch is constructed based on the unique number and the category identifier.

3. The automatic testing method based on requirement semantic analysis according to claim 2, characterized in that, The code change analysis results include at least one of the following: code save record summary, risk level, risk checklist, list of affected business modules, and risk warning; Key content from the code change analysis results is selected and synchronously written to the remote repository merge request description, browser plugin local cache, and project management system activity log.

4. The automatic testing method based on requirement semantic analysis according to claim 3, characterized in that, The step of generating a test submission email based on the code change analysis results includes: A test submission button is embedded in the page of the project management system. When the test submission button is triggered, the unique number of the business requirement, the branch identifier of the code management branch, and the description of the remote repository merge request are encapsulated into JSON format data. Based on the JSON format data and the code change analysis results, the test email template variables are populated, and a test email is generated and sent.

5. The automatic testing method based on requirement semantic analysis according to claim 3, characterized in that, Also includes: When the risk level exceeds the preset level, a message reminder is pushed to the developer's terminal; The content of the test submission email is subject to risk assessment. When the assessment result is high risk, a message reminder is sent to the tester's terminal.

6. The automatic testing method based on requirement semantic analysis according to claim 1, characterized in that, Also includes: When it is detected that the test email has been edited, the differences in the email content are obtained; When it is identified that the email difference content contains revised content based on the code change analysis results and that the revised content is valid, a fine-tuning sample is generated based on the revised content; Based on the fine-tuning samples, the large AI model is fine-tuned; When new code differences are obtained, the finely tuned AI model is invoked to perform semantic analysis on the new code differences.

7. The automatic testing method based on requirement semantic analysis according to claim 1, characterized in that, Also includes: Prioritize the target business system issues contained in the test report; When the priority of the target business system problem is greater than the preset level, new business requirements are obtained based on the target business system problem; Based on the new business requirements, the target business system was tested again.

8. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the automated testing method based on requirement semantic analysis as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the automated testing method based on requirement semantic analysis as described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the automated testing method based on requirement semantic analysis as described in any one of claims 1-7.

Images