SDN controller test case generation method and device and storage medium

Abstract

The invention discloses an SDN controller test case generation method and device and a storage medium, relates to the field of communication, is used for improving the generation efficiency of an SDN controller test case, and comprises the following steps: obtaining a demand document and a network topological graph; the network topological graph comprises mapping equipment, mapping equipment interfaces and a connection relation between the mapping equipment interfaces; the demand document comprises demand content for testing the stability of the SDN controller; determining a mapping path corresponding to the demand document based on the demand document and the network topological graph; and generating a first test case according to the demand document, the mapping path corresponding to the demand document and the target large language model. The method and the device are applied to the process of generating the test case of the SDN controller.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a method, device, and storage medium for generating test cases for an SDN controller. Background Art

[0002] With the rapid development of network technologies and the wide application of the Software Defined Networking (SDN) architecture, the ways of network management and control have undergone great changes. By separating the control layer from the data layer, SDN achieves high flexibility and programmability of the network, enabling the network to show stronger adaptability in dynamic scheduling and configuration. In an SDN environment, the stability of the SDN controller is particularly important. If the controller fails and causes the network to be disconnected, users will be unable to use network services normally, which will not only have a great impact on the user experience but also may cause irreversible damage to the reputation of the operator and customer trust.

[0003] Therefore, in order to ensure the reliability and security of the network system, high-quality software testing of the SDN controller has become a key link. However, in the related art, writing test cases for the SDN controller faces device differences, configuration diversity, and the complexity of application scenarios. In particular, due to significant differences in characteristics and interface definitions between devices of different manufacturers, the design and execution of test cases become more difficult. Therefore, how to improve the generation efficiency of SDN controller test cases is still a technical problem to be solved. Summary of the Invention

[0004] The present application provides a method, device, and storage medium for generating test cases for an SDN controller, which can improve the generation efficiency of SDN controller test cases.

[0005] In a first aspect, the present application provides a method for generating test cases for an SDN controller, including: obtaining a requirements document and a network topology diagram; the network topology diagram includes: mapped devices, mapped device interfaces, and connection relationships between the mapped device interfaces; the requirements document includes requirement content for testing the stability of the SDN controller; determining a mapped path corresponding to the requirements document based on the requirements document and the network topology diagram; and generating a first test case according to the requirements document, the mapped path corresponding to the requirements document, and a target large language model.

[0006] In a possible implementation, a training set is obtained; the training set includes: historical requirement documents for multiple historical periods and corresponding actual test cases for the historical requirement documents; the historical requirement documents in the training set are input into an initial prediction model to determine an initial prediction result; based on the initial prediction result and the actual test cases, the loss value of the initial prediction model is determined, and when the loss value does not meet the preset condition, the model parameters of the initial prediction model are adjusted based on the loss value, and iterative training is continued based on the training set until the model convergence condition is met to obtain a target large language model.

[0007] In a possible implementation, sonar scanning is performed on the first test case to determine the first coverage rate of the first test case; the coverage rate is used to characterize the coverage degree of the test case for the software; when the first coverage rate is less than the first threshold, the first test case is optimized to obtain a second test case; the second coverage rate of the second test case is greater than or equal to the first threshold.

[0008] In a possible implementation, the first test case is converted into unit test code through the target large language model; the unit test code is scanned using a static analysis tool to determine the areas of the software that are not covered after the unit test code runs the first test case; a second test case is generated based on the areas of the software that are not covered and the first test case.

[0009] In a possible implementation, the requirement document includes at least one of the following: requirement name, requirement description, function description, and requirement objective.

[0010] In a possible implementation, the test case includes at least one of the following: test case name, test steps, request method, test case description, and prediction result.

[0011] In a possible implementation, the training set includes: device information, historical test data, traffic scenario data, and fault scenario data.

[0012] In a second aspect, the present application provides an SDN controller test case generation device, including: an acquisition unit for acquiring a requirement document and a network topology diagram; the network topology diagram includes: mapped devices, mapped device interfaces, and connection relationships between the mapped device interfaces; the requirement document includes requirement content for testing the stability of the SDN controller; a processing unit for determining a mapped path corresponding to the requirement document based on the requirement document and the network topology diagram; the processing unit for generating a first test case according to the requirement document, the mapped path corresponding to the requirement document, and the target large language model.

[0013] In a possible implementation, the acquisition unit is further configured to acquire a training set; the training set includes: historical requirement documents for multiple historical periods and actual test cases corresponding to the historical requirement documents; the processing unit is further configured to input the historical requirement documents in the training set into an initial prediction model to determine an initial prediction result; the processing unit is further configured to determine a loss value of the initial prediction model based on the initial prediction result and the actual test cases, and in the case where the loss value does not meet a preset condition, adjust the model parameters of the initial prediction model based on the loss value and continue iterative training based on the training set until a model convergence condition is met to obtain a target large language model.

[0014] In a possible implementation, the processing unit is further configured to perform a sonar scan on the first test case to determine a first coverage rate of the first test case; the coverage rate is used to characterize the coverage degree of the test case for the software; the processing unit is further configured to optimize the first test case to obtain a second test case in the case where the first coverage rate is less than a first threshold; the second coverage rate of the second test case is greater than or equal to the first threshold.

[0015] In a possible implementation, the processing unit is further configured to convert the first test case into unit test code through the target large language model; the processing unit is further configured to use a static analysis tool to scan the unit test code to determine an area of the software that is not covered after the unit test code runs the first test case; the processing unit is further configured to generate a second test case based on the area of the software that is not covered and the first test case.

[0016] In a possible implementation, the requirement document includes at least one of the following: requirement name, requirement description, function description, and requirement objective.

[0017] In a possible implementation, the test case includes at least one of the following: test case name, test steps, request method, test case description, and prediction result.

[0018] In a possible implementation, the training set includes: device information, historical test data, traffic scenario data, and fault scenario data.

[0019] In a third aspect, the present application provides a computer-readable storage medium storing one or more programs, where the one or more programs include instructions that, when executed by an electronic device of the present application, cause the electronic device to execute the SDN controller test case generation method described in the first aspect and any possible implementation of the first aspect.

[0020] Fourth aspect, the present application provides an electronic device, including: a processor and a memory; wherein, the memory is used to store one or more programs, and the one or more programs include computer execution instructions. When the electronic device runs, the processor executes the computer execution instructions stored in the memory, so that the electronic device executes the SDN controller test case generation method described in the first aspect and any possible implementation manner of the first aspect.

[0021] Fifth aspect, the present application provides a computer program product containing instructions. When the instructions run on a computer, the electronic device of the present application is enabled to execute the SDN controller test case generation method described in the first aspect and any possible implementation manner of the first aspect.

[0022] Sixth aspect, the present application provides a chip system, which is applied to an SDN controller test case generation device; the chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the memory of the SDN controller test case generation device and send signals to the processors, and the signals include computer instructions stored in the memory. When the processors execute the computer instructions, the SDN controller test case generation device executes the SDN controller test case generation method according to the first aspect and any possible design manner thereof.

[0023] In the present application, the name of the above-mentioned SDN controller test case generation device does not constitute a limitation on the device or functional unit itself. In actual implementation, these devices or functional units may appear under other names. As long as the functions of each device or functional unit are similar to those of the present application, they all fall within the scope of the claims of the present application and their equivalent technologies.

[0024] Based on the above technical solutions, in the present application, it is considered that the network topology diagram includes: mapping devices, mapping device interfaces, and the connection relationships between the mapping device interfaces. The SDN controller test case generation device can determine the mapping path corresponding to the requirement document based on the requirement document and the network topology diagram. Since the mapping path corresponding to the requirement document can represent the mapping devices required for the test case and the order of the mapping devices that the test case tests in sequence, therefore, the SDN controller test case generation device can quickly and accurately determine the test steps through the target large language model for the requirement document and the mapping path corresponding to the requirement document, improving the efficiency of generating the first test case. Description of the Drawings

[0025] Figure 1 It is a schematic diagram of the architecture of an SDN controller test case generation system provided by an embodiment of the present application;

[0026] Figure 2Schematic diagram of a structure of an SDN controller test case generation device provided by an embodiment of the present application;

[0027] Figure 3 Schematic diagram of a process of a method for generating an SDN controller test case provided by an embodiment of the present application;

[0028] Figure 4 Schematic diagram of a network topology provided by an embodiment of the present application;

[0029] Figure 5 Schematic diagram of a process of a method for obtaining a training set provided by an embodiment of the present application;

[0030] Figure 6 Schematic diagram of a process of another method for generating an SDN controller test case provided by an embodiment of the present application;

[0031] Figure 7 Schematic diagram of a process of another method for generating an SDN controller test case provided by an embodiment of the present application;

[0032] Figure 8 Schematic diagram of the sonar scan result of an SDN controller test case generation device provided by an embodiment of the present application for a first test case;

[0033] Figure 9 Schematic diagram of a process of another method for generating an SDN controller test case provided by an embodiment of the present application;

[0034] Figure 10 Schematic diagram of a process of another method for generating an SDN controller test case provided by an embodiment of the present application;

[0035] Figure 11 Schematic diagram of a structure of another SDN controller test case generation device provided by an embodiment of the present application. Detailed implementation manners

[0036] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0037] In this article, the character " / " generally indicates an "or" relationship between the associated objects before and after. For example, A / B can be understood as A or B.

[0038] The terms "first" and "second" in the description and claims of this application are used to distinguish different objects, rather than to describe a specific order of the objects. For example, the first edge service node and the second edge service node are used to distinguish different edge service nodes, rather than to describe the characteristic order of the edge service nodes.

[0039] In addition, the terms "comprising" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include other unlisted steps or units, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0040] In addition, in the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplarily" or "for example" in this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplarily" or "for example" is intended to present concepts in a specific manner.

[0041] With the rapid development of network technology and the wide application of the Software Defined Networking (SDN) architecture, the way of network management and control has undergone great changes. By separating the control layer from the data layer, SDN realizes the high flexibility and programmability of the network, enabling the network to show stronger adaptability in dynamic scheduling and configuration. In the SDN environment, the stability of the SDN controller is particularly important. If the controller fails and causes the network to disconnect, users will not be able to use network services normally, which will not only have a great impact on the user experience, but may also cause irreversible damage to the reputation of the operator and the customer trust.

[0042] Therefore, in order to ensure the reliability and security of the network system, high-quality software testing of the SDN controller has become a key link. However, in the related technology, the writing of SDN controller test cases faces device differences, configuration diversity and application scenario complexity. Especially because there are significant differences in characteristics and interface definitions among devices of different manufacturers, the design and execution of test cases become more difficult. Therefore, how to improve the generation efficiency of SDN controller test cases is still a technical problem to be solved.

[0043] Based on the above technical solution, in this application, it is considered that the network topology diagram includes: mapping devices, mapping device interfaces, and the connection relationships between the mapping device interfaces. The SDN controller test case generation device can determine the mapping path corresponding to the requirements document based on the requirements document and the network topology diagram. Since the mapping path corresponding to the requirements document can represent the mapping devices required for the test case and the order of the mapping devices that the test case tests in sequence, therefore, the SDN controller test case generation device can quickly and accurately determine the test steps through the target large language model for the requirements document and the mapping path corresponding to the requirements document, improving the efficiency of generating the first test case.

[0044] Exemplarily, as Figure 1 shown, Figure 1 FIG. 10 is a schematic structural diagram of an SDN controller test case generation system provided by this application. The SDN controller test case generation system 10 includes: a data acquisition device 101 and an SDN controller test case generation device 102.

[0045] The data acquisition device 101 obtains the requirements document and the network topology diagram.

[0046] The SDN controller test case generation device 102 obtains the requirements document and the network topology diagram through the data acquisition device 101, and determines the mapping path corresponding to the requirements document based on the requirements document and the network topology diagram. In this way, it is convenient to generate the first test case according to the requirements document, the mapping path corresponding to the requirements document, and the target large language model.

[0047] Among them, the target large language model can be set in the SDN controller test case generation device 102, or can be set in the processing device connected to the SDN controller test case generation device 102.

[0048] Optionally, the physical device of the data acquisition device 101 is a terminal, and the physical device of the SDN controller test case generation device 102 is a server.

[0049] Optionally, the above terminal can be a device that provides voice and / or data connectivity to the user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or referred to as a "cellular" phone) and a computer with a mobile terminal, or can also be a portable, pocket-sized, handheld, computer-integrated or vehicle-mounted mobile device, which exchanges language and / or data with the wireless access network. For example, a mobile phone, a tablet computer, a notebook computer, a netbook, a personal digital assistant (PDA).

[0050] Optionally, the above server may be one server in a server cluster (composed of multiple servers), or a chip in the server, or a system on chip in the server, or may be implemented by a virtual machine (VM) deployed on a physical machine. The embodiments of the present application do not make any limitations in this regard.

[0051] The embodiments of the present application provide an SDN controller test case generation device for executing the SDN controller test case generation system provided by the embodiments of the present application. Figure 2 It is a schematic structural diagram of an SDN controller test case generation device provided by the embodiments of the present application. As Figure 2 shown, the SDN controller test case generation device 200 includes at least one processor 201, a communication line 202, and at least one communication interface 204, and may further include a memory 203. Among them, the processor 201, the memory 203, and the communication interface 204 can be connected through the communication line 202.

[0052] The processor 201 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. For example: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0053] The communication line 202 may include a path for transmitting information between the above components.

[0054] The communication interface 204 is used to communicate with other devices or communication networks, and any transceiver-like device can be used, such as Ethernet, radio access network (RAN), WLAN, etc.

[0055] The memory 203 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to include or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0056] In a possible design, the memory 203 can exist independently of the processor 201, that is, the memory 203 can be an external memory of the processor 201. At this time, the memory 203 can be connected to the processor 201 through the communication line 202, used to store execution instructions or application program codes, and be controlled by the processor 201 to execute, so as to implement the SDN controller test case generation method provided in the following embodiments of the present application. In another possible design, the memory 203 can also be integrated with the processor 201, that is, the memory 203 can be an internal memory of the processor 201. For example, the memory 203 is a cache, which can be used to temporarily store some data and instruction information, etc.

[0057] As an implementable manner, the processor 201 can include one or more CPUs, such as Figure 2 CPU0 and CPU1 in Figure 2 . As another implementable manner, the SDN controller test case generation device 200 can include multiple processors, such as

[0058] the processor 201 and the processor 207 in Figure 3 . As yet another implementable manner, the SDN controller test case generation device 200 can further include an output device 205 and an input device 206.

[0058] Hereinafter, a detailed description will be given of an SDN controller test case generation method provided in an embodiment of the present application in conjunction with the attached Figure 3 drawings. As shown in Figure 3 , the SDN controller test case generation method includes S301 - S303.

[0059] S301. The SDN controller test case generation device obtains a requirements document and a network topology diagram.

[0060] Among them, the network topology diagram includes: a mapping device, mapping device interfaces, and the connection relationships between the mapping device interfaces; the requirements document is used to test the stability of the SDN controller.

[0061] Optionally, the requirements document includes at least one of the following: requirement name, requirement description, function description, and requirement objective.

[0062] Exemplarily, as Figure 4 shown, the network topology diagram includes: H3C-R2, ZTE-R1, HW-PE1, H3C-R4, a damage instrument, and a current injector. In the network topology diagram, the interface Ten 2 / 2 / 2 of H3C-R2 is connected to the interface 2 / 8 of the current injector, H3C-R2 is connected to the damage instrument, the interface Ten 2 / 2 / 1 of H3C-R2 is connected to the interface XGE 0 / 0 / 0 / 1 of ZTE-R1, the interface Ten2 / 4 / 4 of H3C-R2 is connected to the interface Gi 1 / 0 / 2 of HW-PE1, and the interface Ten 2 / 1 / 15 of H3C-R2 is connected to GO BGP; ZTE-R1 is connected to the damage instrument, the interface cgei 0 / 0 / 1 / 1 of ZTE-R1 is connected to the interface 100GE 5 / 0 / 1 of HW-PE1, the interface cgei 0 / 0 / 1 / 2 of ZTE-R1 is connected to the interface 100G3 / 1 / 2 of H3C-R4, and the interface XGE 0 / 0 / 0 / 6 of ZTE-R1 is connected to the interface 1 / 13 of the current injector; the interface Gi 1 / 1 / 0 of HW-PE1 is connected to the interface 2 / 6 of the current injector, and the interface 100G 5 / 0 / 0 of HW-PE1 is connected to the interface 100G3 / 1 / 1 of H3C-R4; the interface 100G3 / 1 / 1 of H3C-R4 is connected to the interface 1 / 4 of the current injector.

[0063] S302. The SDN controller test case generation device determines the mapping path corresponding to the requirements document based on the requirements document and the network topology diagram.

[0064] In a possible implementation manner, the SDN controller test case generation device associates the requirements document with the corresponding interfaces, and associates the requirements document and the topology to determine the mapping path corresponding to the requirements document.

[0065] Exemplarily, the requirements document description is: simulate damage, create a tunnel, and obtain the routing calculation result.

[0066] Associate the requirements with the corresponding interfaces:

[0067]

[0068] Associate the requirements document and the topology:

[0069]

[0070] The final generated result is as follows:

[0071] Interface: CreateTunnel

[0072] URL: / api / tunnel / create, Request method: POST

[0073] Mapped devices: ['ZTE - R1', 'H3C - R2', 'HW - PE1', 'H3C - R4']

[0074] Mapped path: ['ZTE - R1 --> Damage instrument --> H3C - R2 --> HW - PE1 --> H3C - R4']

[0075] Mapping requirements: ['Simulate damage, create tunnel, obtain routing calculation results'].

[0076] S303. The SDN controller test case generation device generates the first test case according to the requirements document, the mapped path corresponding to the requirements document, and the target large language model.

[0077] Among them, the test case includes at least one of the following: test case name, test steps, request method, test case description, predicted result.

[0078] In a possible implementation, the SDN controller test case generation device writes the first prompt, and the target large model generates the first test case according to the interface document, the requirements document, and the mapped path corresponding to the requirements document.

[0079]

[0080]

[0081] Optionally, the target language model is baichuan2 - 13b - chat.

[0082] It should be noted that before using the target language model, the SDN controller test case generation device prepares the training script, downloads the baichuan2 - main project, based on the fine - tuning script in the project, modifies the model path in the file to point to the local stored baichuan2 - 13b - chat model weight path; modifies the training script to obtain json parameter information to adapt to the cleaned data format; adds a loss function to facilitate viewing the loss value during the training process. The SDN controller test case generation device sets the initial learning rate, batch size, and training cycle parameters for reasonable training optimization.

[0083] Optionally, the loss function is a non - negative real - valued function used to measure the degree of inconsistency between the predicted values and the true values of the model. Generally, the smaller the loss value, the better the accuracy of the model. Learning rate: Controls the magnitude of the weight update in each iteration of the model. The learning rate determines how far the model should move towards the minimum of the loss function in each step of the optimization process. An overly high learning rate may cause the model to skip the optimal solution during training, resulting in non - convergence and even unstable training. An overly low learning rate will cause the model to converge too slowly, increasing the training time. The batch size is the number of data samples passed to the model for training each time. The number of training epochs is the number of times the entire training dataset is used for training once. When the number of training epochs is too small, the model may not fully learn the patterns in the data, resulting in underfitting and poor performance; when the number of training epochs is too large, it may lead to overfitting of the model to the training data.

[0084] Exemplarily, as shown in Table 1 above, the test steps of test case TC - 001 are as follows: 1. The VPN is successfully created. 2. Use a damage instrument to simulate a 100 - ms delay damage. 3. Create an r1 - r2 tunnel and configure a 10 - ms delay. 4. Obtain the routing calculation data. The test interface is / api / tunnel, and the request method is Post. The test case description is to create an srv6 tunnel, select a route based on delay constraints, and the request body is "

[0085] "tunnel": [{"tunnel - name": "r1", "tunnel - id": "r1"}, {"tunnel - name":

[0086] Exemplarily, as Figure 5 shown, the SDN controller test case generation device uploads a first file. The first file includes the SDN controller document and the network topology diagram, and determines whether the first file is successfully uploaded. In the case where the first file upload fails, the first file is re - uploaded. In the case where the first file is successfully uploaded, the first file data is automatically read, a second prompt is written, and the first test case is generated according to the second prompt. In the case where the first file upload fails, the SDN controller test case generation device re - uploads the first file.

[0087] The above solution brings at least the following beneficial effects: In this application, it is considered that the network topology diagram includes: mapping devices, mapping device interfaces, and the connection relationships between the mapping device interfaces. The SDN controller test case generation device can determine the mapping path corresponding to the requirement document based on the requirement document and the network topology diagram. Since the mapping path corresponding to the requirement document can represent the mapping devices required for the test case and the order of the mapping devices to be tested by the test case in sequence, therefore, the SDN controller test case generation device can quickly and accurately determine the test steps through the target large language model for the requirement document and the mapping path corresponding to the requirement document, improving the efficiency of generating the first test case.

[0088] Combined Figure 3 , such as Figure 6 shown, the target large language model can be constructed through an electronic device, such as Figure 6 shown, the target large language model is trained in the following manner:

[0089] S601. The SDN controller test case generation device obtains a training set.

[0090] Among them, the training set includes: historical requirement documents of multiple historical periods and actual test cases corresponding to the historical requirement documents.

[0091] Optionally, the training set further includes: device information, historical test data, traffic scenario data, and fault scenario data. The traffic scenario data includes normal traffic, burst traffic, and abnormal traffic; the fault scenario data includes link jitter, packet loss, and latency.

[0092] In a realizable manner, the target large language model can be constructed through an electronic device. The electronic device needs to train an initial prediction model based on the training set and continuously adjust the model parameters of the initial prediction model according to the initial prediction result and the training set data to obtain the target large language model. Therefore, the electronic device needs to obtain a training set including historical requirement documents of multiple historical periods and actual test cases corresponding to the historical requirement documents.

[0093] In another possible implementation manner, the SDN controller test case generation device obtains SDN controller document data and writes a cleaning script to clean the SDN controller document data to obtain a training set.

[0094] It should be explained that the cleaning script is {

[0095]

[0096]

[0097] It should be noted that the training script distinguishes the source of the input data by user ID and assistant ID. from:human is used to represent the user, Value is used to represent the content of the user's question, from:assistant is used to represent the target language model, and value is used to represent the content of the target language model's response.

[0098] S602. The SDN controller test case generation device inputs the historical requirement documents in the training set into the initial prediction model to determine the initial prediction result.

[0099] In a feasible way, the electronic device can input the service information in the training set into the initial prediction model and determine the initial prediction result, so that the electronic device adjusts the model parameters of the initial prediction model based on the initial prediction result and the actual test cases in the training set to obtain the target large language model.

[0100] S603. The SDN controller test case generation device determines the loss value of the initial prediction model based on the initial prediction result and the actual test cases, and when the loss value does not meet the preset conditions, adjusts the model parameters of the initial prediction model based on the loss value and continues to perform iterative training based on the training set until the model convergence condition is met to obtain the target large language model.

[0101] In a feasible way, the electronic device adjusts the model parameters of the initial prediction model based on the initial prediction result and the actual test cases to obtain the target large language model.

[0102] Optionally, the electronic device can build the target large language model based on a neural network.

[0103] Optionally, a neural network is a computational model that mimics the connection and information processing method between neurons in a biological brain.

[0104] Optionally, a neural network consists of a large number of interconnected nodes (which can also be called neurons), and these nodes are arranged in a certain hierarchical structure (input layer, hidden layer, output layer). Information is transmitted and processed through the connections between neurons. Each neuron receives input signals from other neurons and calculates the input through an activation function to generate an output signal that is transmitted to the subsequent neurons.

[0105] Optionally, the electronic device can build a loss function based on multiple loss values.

[0106] Optionally, the loss function includes at least one of the mean square error function, mean absolute error function, cross-entropy loss function, binary cross-entropy loss function, hinge loss function, log-likelihood loss function, and Kullback-Leibler (KL) divergence function.

[0107] It should be noted that the SDN controller test case generation device fine-tunes and trains the target language model in the LoRA manner, and the main parameters for training are as follows:

[0108]

[0109]

[0110] Among them, the LoRA training principle: LoRA utilizes the data corresponding to the downstream task and only trains the newly added part of the parameters to adapt to the downstream task.

[0111] It can be understood that after training the new parameters, the SDN controller test case generation device uses the method of recombining parameters to merge the new parameters and the old model parameters, so that it can achieve the effect of fine-tuning the entire model on the new task without increasing the inference time during inference, reducing the consumption of server resources.

[0112] The above solution brings at least the following beneficial effects: In the embodiments of the present application, the SDN controller test case generation device improves the coverage of the first test case based on device information, historical test data, traffic scenario data, and fault scenario data, thereby enhancing the comprehensiveness and effectiveness of the test. Through in-depth analysis of historical project data by fine-tuning the target language model, common test scenarios are learned, providing practical references and templates for the generation of test cases for new projects, effectively shortening the development cycle, and improving the test efficiency.

[0113] In a possible implementation manner, in combination with Figure 3 , as Figure 7 shown, after the SDN controller test case generation device generates the first test case according to the requirements document, the mapping path corresponding to the requirements document, and the target large language model in S303, the SDN controller test case generation device optimizes the first test case to obtain the second test case. The process of the SDN controller test case generation device optimizing the first test case to obtain the second test case can be specifically implemented through the following S701-S702.

[0114] S701. The SDN controller test case generation device performs a sonar scan on the first test case to determine the first coverage rate of the first test case.

[0115] Among them, the coverage rate is used to characterize the coverage degree of the test case for the software.

[0116] Optionally, the coverage rate includes at least one of the following: requirements coverage rate, code coverage rate, function coverage rate, and scenario coverage rate.

[0117] In one possible implementation, the SDN controller test case generation device performs a sonar scan on the first test case to determine the first coverage rate of the first test case, and consults the requirements document to determine whether the first test case covers all scenarios.

[0118] Exemplarily, as Figure 8 shown, the results of the sonar scan performed by the SDN controller test case generation device on the first test case include: the coverage rate is 79.3%, the newly covered behaviors are 164, the uncovered behaviors are 34, the covered behaviors are 79.3%, the newly coverable branches are 0, and the uncovered conditions are 0.

[0119] S702. When the first coverage rate is less than the first threshold, the SDN controller test case generation device optimizes the first test case to obtain a second test case.

[0120] Among them, the second SDN controller coverage rate of the second test case is greater than or equal to the first threshold.

[0121] In one possible implementation, when the first coverage rate is less than the first threshold and the first test case does not cover all scenarios, the SDN controller test case generation device optimizes the first test case to obtain a second test case.

[0122] Optionally, the first threshold is 80%.

[0123] The above solution brings at least the following beneficial effects: In the embodiments of the present application, when the first coverage rate is less than the first threshold and the first test case does not cover all scenarios, the adaptability of the first test case is poor, and the SDN controller test case generation device can adjust the first test case until a second test case that meets the coverage rate requirement is obtained.

[0124] In one possible implementation, in combination with Figure 7 , as Figure 9 shown, the process of the SDN controller test case generation device optimizing the first test case to obtain a second test case in S702 can be specifically implemented through the following S901 - S903.

[0125] S901. The SDN controller test case generation device converts the first test case into unit test code through a target large language model.

[0126] In one possible implementation, the SDN controller test case generation device writes a third prompt word, uses the target large language model to convert the first test case into unit test code, and uploads the unit test code to the project code.

[0127] S902. The SDN controller test case generation device uses a static analysis tool to scan the unit test code and determine the areas of the software that are not covered after the unit test code runs the first test case.

[0128] In a possible implementation, the SDN controller test case generation device uses coverage to execute unit tests: python3 -m coverage run manage.py test–keepdb. When the unit test is executed successfully, the SDN controller test case generation device uses a static analysis tool to scan the unit test code and determine the areas of the software corresponding to the unit test code that are not covered.

[0129] Optionally, the static analysis tool includes SonarQube.

[0130] It should be explained that when the unit test is executed successfully, the SDN controller test case generation device uses coverage to execute unit tests again: python3 -m coverage run manage.py test–keepdb.

[0131] It can be understood that the SDN controller test case generation device uploads the second file to the knowledge base. The second file includes the first test case, the sonar scan result, the requirements document, and the interface document. The SDN controller test case generation device determines whether the second file is uploaded successfully. If the second file is uploaded successfully, it reads the content of the second file and saves the execution information of the first test case, the requirement information, the interface information, and the code coverage situation into variables. If the second file upload fails, the SDN controller test case generation device re-uploads the second file.

[0132] S903. The SDN controller test case generation device generates a second test case based on the areas of the software that are not covered and the first test case.

[0133] In a possible implementation, the SDN controller test case generation device writes a fourth prompt word to prompt the target language model to identify the test case and the execution result, determine the failure reason, the uncovered code module, and the corresponding line numbers. The SDN controller test case generation device adjusts the first test case based on the failure reason, the uncovered code module, and the corresponding line numbers to generate a second test case.

[0134] Optionally, the second test case covers all scenarios.

[0135] It should be explained that the SDN controller test case generation device calls the agent tool to convert the generated test case into an excel file and send an email.

[0136] Exemplarily, as Figure 10 shown, the SDN controller test case generation device downloads an initial language model, prepares a fine-tuning script, and collects SDN controller test documents. In this way, the SDN controller test case generation device can clean the SDN controller test document data, and then fine-tune the initial language model to obtain a trained target language model. The SDN controller test case generation device determines whether the answer result of the target language model meets the requirements. If the answer result meets the requirements, the target language model is used to generate a first test case. The SDN controller test case generation device uses the AGENT tool to save and email the first test case, and determines whether the first test case is comprehensive. If the first test case is not comprehensive, the first test case is optimized and feedback is provided.

[0137] The above solution has at least the following beneficial effects: In the embodiments of the present application, the SDN controller test case generation device can adjust the first test case purposefully according to the failure reason, the uncovered code module and the corresponding line number. In this way, the applicability of the adjusted second test case can be more comprehensive, and the stability of the SDN controller can be accurately tested.

[0138] In the embodiments of the present application, the SDN controller test case generation device can be divided into function modules or functional units according to the above method examples. For example, each function can correspond to a function module or a functional unit, or two or more functions can be integrated into a processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module or a functional unit. Among them, the division of modules or units in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0139] Exemplarily, as Figure 11 shown, it is a possible structural schematic diagram of an SDN controller test case generation device involved in the embodiments of the present application. The SDN controller test case generation device 110 includes: an acquisition unit 1101 and a processing unit 1102.

[0140] The acquisition unit 1101 is used to acquire a requirement document and a network topology diagram; the network topology diagram includes: mapping devices, mapping device interfaces, and the connection relationships between the mapping device interfaces; the requirement document includes requirement content for testing the stability of the SDN controller; the processing unit 1102 is used to determine a mapping path corresponding to the requirement document based on the requirement document and the network topology diagram; the processing unit 1102 is used to generate a first test case according to the requirement document, the mapping path corresponding to the requirement document, and a target large language model.

[0141] Optionally, the obtaining unit 1101 is further configured to obtain a training set, where the training set includes historical requirement documents for multiple historical periods and actual test cases corresponding to the historical requirement documents; the processing unit 1102 is further configured to input the historical requirement documents in the training set into an initial prediction model to determine an initial prediction result; the processing unit 1102 is further configured to determine a loss value of the initial prediction model based on the initial prediction result and the actual test cases, and in the case where the loss value does not meet a preset condition, adjust the model parameters of the initial prediction model based on the loss value and continue iterative training based on the training set until a model convergence condition is met to obtain a target large language model.

[0142] Optionally, the processing unit 1102 is further configured to perform a sonar scan on the first test case to determine a first coverage rate of the first test case; the coverage rate is used to characterize the coverage degree of the test case for the software; the processing unit 1102 is further configured to optimize the first test case to obtain a second test case in the case where the first coverage rate is less than a first threshold; the second coverage rate of the second test case is greater than or equal to the first threshold.

[0143] Optionally, the processing unit 1102 is further configured to convert the first test case into unit test code through the target large language model; the processing unit 1102 is further configured to use a static analysis tool to scan the unit test code to determine an area of the software that is not covered after the unit test code runs the first test case; the processing unit 1102 is further configured to generate a second test case according to the area of the software that is not covered and the first test case.

[0144] Optionally, the requirement document includes at least one of the following: requirement name, requirement description, function description, and requirement objective.

[0145] Optionally, the test case includes at least one of the following: test case name, test steps, request method, test case description, prediction result.

[0146] Optionally, the training set includes: device information, historical test data, traffic scenario data, and fault scenario data.

[0147] An embodiment of this application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a computer program or instruction to implement the SDN controller test case generation method in the foregoing method embodiment.

[0148] An embodiment of this application provides a computer program product including instructions, which when running on a computer, cause the computer to execute the switching method of an industrial terminal in the foregoing method embodiment.

[0149] Among them, a computer-readable storage medium can be, for example, but not limited to, a system, device, or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). In an embodiment of the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, device, or component.

[0150] Since the devices, equipment, computer-readable storage media, and computer program products in the embodiments of the present invention can be applied to the above method, the technical effects that can be obtained can also refer to the method embodiments above, and will not be elaborated herein in the embodiments of the present application.

[0151] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating test cases for a software defined network (SDN) controller, characterized in that: The method comprises: Obtaining a requirement document and a network topology diagram; the network topology diagram includes: a mapping device, the mapping device interface, and a connection relationship between the mapping device interfaces; the requirement document includes requirement content for testing the stability of the SDN controller; Based on the requirement document and the network topology diagram, determining a mapping path corresponding to the requirement document; A first test case is generated according to the requirement document, a mapping path corresponding to the requirement document, and a target large language model.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a training set; the training set includes: historical requirement documents of multiple historical periods and actual test cases corresponding to the historical requirement documents; Inputting the historical demand documents in the training set into an initial prediction model to determine an initial prediction result; Based on the initial prediction result and the actual test case, the loss value of the initial prediction model is determined, and when the loss value does not meet the preset conditions, the model parameters of the initial prediction model are adjusted based on the loss value, and iterative training is continued based on the training set until the model convergence conditions are met to obtain the target large language model.

3. The method according to claim 1, characterized in that After generating the first test case according to the requirement document, the mapping path corresponding to the requirement document and the first target large language model, the method further includes: Performing a sonar scan on the first test case to determine a first coverage rate of the first test case; the coverage rate is used to characterize the coverage degree of the test case on the software; When the first coverage rate is less than a first threshold, the first test case is optimized to obtain a second test case; and a second coverage rate of the second test case is greater than or equal to the first threshold.

4. The method according to claim 3, characterized in that The step of optimizing the first test case to obtain a second test case includes: Converting the first test case into unit test code by using the target large language model; Scan the unit test code using a static analysis tool to determine areas of the software that are not covered after the unit test code runs the first test case; The second test case is generated according to the area not covered by the software and the first test case.

5. The method according to any one of claims 1 to 4, characterized in that: The requirement document includes at least one of the following: requirement name, requirement description, function description and requirement goal.

6. The method according to any one of claims 1 to 4, characterized in that: The test case includes at least one of the following: a test case name, test steps, a request method, a test case description, and a prediction result.

7. The method according to claim 2, characterized in that The training set includes: equipment information, historical test data, traffic scenario data and fault scenario data.

8. A SDN controller test case generation device, characterized in that: include: Acquisition unit and processing unit; The acquisition unit is used to acquire the demand document and the network topology diagram; The network topology diagram includes: mapping devices, mapping device interfaces, and connection relationships between the mapping device interfaces; the requirement document includes requirement content for testing the stability of the SDN controller; The processing unit is used to determine a mapping path corresponding to the requirement document based on the requirement document and the network topology diagram; The processing unit is used to generate a first test case according to the requirement document, the mapping path corresponding to the requirement document and the target large language model.

9. A SDN controller test case generation device, characterized in that: include: A processor and a memory; wherein the memory is used to store computer-executable instructions, and when the SDN controller test case generation device is running, the processor executes the computer-executable instructions stored in the memory, so that the SDN controller test case generation device performs the SDN controller test case generation method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions, which, when executed by the SDN controller test case generation device, enable the computer to execute the SDN controller test case generation method according to any one of claims 1 to 6.

Images