Network device intelligent testing method and device, equipment and medium

Abstract

This disclosure proposes an intelligent testing method, apparatus, device, and medium for network devices. The intelligent testing method includes: determining target real data for the network device; determining initial generated data based on the target real data; obtaining the distribution characteristics of the initial generated data through a first model, and generating intermediate generated data based on the distribution characteristics; processing the target real data and intermediate generated data through a second model to obtain target generated data, wherein the second model is used to model and learn the distribution differences between the target real data and the intermediate generated data; and performing intelligent testing on the network device based on the target real data and target generated data. This addresses the technical problem in existing technologies where network device test results are not comprehensive enough and the test results are unsatisfactory.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to an intelligent testing method, apparatus, device and medium for network devices. Background Technology

[0002] Intelligent testing of network equipment (such as base stations) refers to conducting various tests on key performance indicators (KPIs) and other test content for fixed communication scenarios to determine whether the network equipment meets the communication quality indicators inherent in a specific intelligent scenario. With the diversified development of network service applications, wireless network intelligence is being further promoted as a crucial core capability for improving network quality and efficiency and solving increasingly complex network tasks.

[0003] In related technologies, the testing of the intelligence level of network devices is usually conducted within a predetermined "range" to determine the performance level of the network device's intelligence capabilities.

[0004] In this approach, the test results for network devices are not comprehensive enough, and the test results are unsatisfactory. Summary of the Invention

[0005] This disclosure aims to at least partially address one of the technical problems in the related art.

[0006] Therefore, this disclosure proposes an intelligent testing method, apparatus, electronic device, non-transitory computer-readable storage medium storing computer instructions, and computer program product for network devices, in order to improve the comprehensiveness of test results for network devices and enhance test effectiveness.

[0007] The first aspect of this disclosure proposes an intelligent testing method for network devices, comprising: determining target real data of the network device; determining initial generated data based on the target real data; obtaining distribution characteristics of the initial generated data through a first model, and generating intermediate generated data based on the distribution characteristics; processing the target real data and intermediate generated data through a second model to obtain target generated data, wherein the second model is used to model and learn the distribution differences between the target real data and the intermediate generated data; and performing intelligent testing on the network device based on the target real data and target generated data.

[0008] A second aspect of this disclosure provides an intelligent testing apparatus for network devices, comprising: a first determining module for determining target real data of the network device; a second determining module for determining initial generated data based on the target real data; a first acquiring module for acquiring distribution characteristics of the initial generated data through a first model and generating intermediate generated data based on the distribution characteristics; a second acquiring module for processing the target real data and intermediate generated data through a second model to obtain target generated data, wherein the second model is used to model and learn the distribution differences between the target real data and the intermediate generated data; and a testing module for performing intelligent testing on the network device based on the target real data and the target generated data.

[0009] A third aspect of this disclosure provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described above.

[0010] A fourth aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method described above.

[0011] A fifth aspect of this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the method described above.

[0012] The network device intelligent testing method, apparatus, electronic device, non-transitory computer-readable storage medium storing computer instructions, and computer program product disclosed herein determine the target real data of the network device, and based on the target real data, determine the initial generated data. A first model is used to obtain the distribution characteristics of the initial generated data, and intermediate generated data is generated based on the distribution characteristics. A second model processes the target real data and the intermediate generated data to obtain the target generated data. The second model is used to model and learn the distribution differences between the target real data and the intermediate generated data, and to perform intelligent testing on the network device based on the target real data and the target generated data. This greatly enriches the target real data, and since the target generated data is generated based on the target real data, it has a strong correlation with the target real data and is richer than the target real data. Therefore, it can achieve generalized testing of the intelligent capabilities of network devices, thereby greatly improving the comprehensiveness of the test results and enhancing the test effectiveness.

[0013] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0014] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 This is a flowchart illustrating an embodiment of the intelligent testing method for network devices proposed in this disclosure;

[0016] Figure 2 This is a flowchart illustrating an intelligent testing method for network devices according to another embodiment of this disclosure;

[0017] Figure 3 This is a schematic diagram of a test scenario in an embodiment of this disclosure;

[0018] Figure 4 This is a schematic diagram illustrating the process by which the intelligent testing tool generates generalizable generated data in this embodiment of the disclosure.

[0019] Figure 5 This is a schematic diagram of the model in the generative-adversarial mechanism in this embodiment of the disclosure;

[0020] Figure 6 This is a schematic diagram of the structure of an intelligent testing device for network devices according to an embodiment of this disclosure;

[0021] Figure 7 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0022] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0023] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0024] Figure 1 This is a flowchart illustrating an embodiment of the intelligent testing method for network devices proposed in this disclosure.

[0025] This embodiment illustrates the use of a network device intelligent testing method configured within a network device intelligent testing device. In this embodiment, the network device intelligent testing method can be configured within the network device intelligent testing device, which can be located in a server or an electronic device, without limitation.

[0026] It should be noted that the execution entity of the embodiments disclosed herein may be, in hardware, a central processing unit (CPU) in a server or electronic device, and in software, a related background service in a server or electronic device, without limitation.

[0027] like Figure 1 As shown, the intelligent testing method for this network device includes:

[0028] S101: Determine the target real data of the network device.

[0029] The target real data refers to the actual data generated by network devices when providing communication services. Target real data can include, for example, data related to network standard, coverage type, station type, and frequency band, without limitation. Optionally, in some embodiments, the target real data can also be obtained by processing original real data, which can also be called initial real data. That is to say, initial real data is the unprocessed real data generated by network devices when providing communication services. The initial real data can be processed (e.g., encoding, deduplication, linear fitting, etc.) to obtain the target real data.

[0030] Optionally, in some embodiments, the process of determining the target real data of the network device may involve collecting initial real data of the network device, wherein the initial real data includes at least one sample feature, encoding the at least one sample feature in the initial real data to obtain at least one encoded feature, and determining the target real data based on the at least one encoded feature. This effectively reduces redundant information in the initial real data, saves data volume, reduces data processing costs required for testing, and improves testing efficiency while ensuring test effectiveness.

[0031] Optionally, in some embodiments, the above-described encoding method can be a one-hot encoding method. For example, one or more sample features can be extracted from the initial real data, and each sample feature can be encoded using a one-hot encoding method. The encoded features can be called encoded features, thereby obtaining one or more encoded features. Then, the one or more encoded features are further processed to obtain the target real data. For example, one or more encoded features can be concatenated to obtain the target real data, and there is no limitation on this.

[0032] Optionally, in some embodiments, each sample feature in the initial real data may correspond to at least one dimension. The "dimension" may be one or more of the following: network standard, coverage type, station type, frequency band, and service type. The feature values ​​of different sample features under different dimensions are different, and the feature values ​​of different sample features under the same dimension may be the same or different, without limitation.

[0033] Optionally, in some embodiments, the aforementioned "dimensions" can be customized by testers based on actual testing needs, and there are no restrictions on this.

[0034] S102: Determine the initial generated data based on the target's actual data.

[0035] The initial generated data can refer to the data obtained by expanding the target real data. The initial generated data is obtained by feature learning and expansion of one or more coded features in the target real data.

[0036] Optionally, in some embodiments, after determining the target real data of the network device, the target real data can be processed to obtain initial generated data.

[0037] Optionally, in some embodiments, in the process of determining the initial generated data based on the target real data, the initial generated data may be obtained by linearly fitting at least one encoded feature in the target real data. This effectively improves data interpretability, and since the linear fitting generates the initial generated data based on the low-dimensional encoded features of the target real data, it effectively ensures that the mean and variance of the distribution between the initial generated data and the target real data are consistent, and prevents the introduction of bias or noise during the generation process, thereby improving the reliability of the initial generated data.

[0038] For example, a linear model can be used to learn the distribution pattern of one or more encoded features of the target real data, and based on the learned distribution pattern, initial generated data that conforms to the distribution pattern can be generated, without any restrictions.

[0039] S103: Obtain the distribution characteristics of the initial generated data through the first model, and generate intermediate generated data based on the distribution characteristics.

[0040] Optionally, in some embodiments, after determining the target real data of the network device and determining the initial generated data based on the target real data, the distribution characteristics of the initial generated data can be obtained through a first model, and intermediate generated data can be generated based on the distribution characteristics. The first model is used to perform secondary generation processing based on the initial generated data, and the resulting real data can be referred to as the intermediate generated data. The first model can be, for example, a neural network model. That is to say, the process of "determining the initial generated data based on the target real data" can be regarded as the first generation process, while the generation of intermediate generated data based on the initial generated data through the first model can be regarded as the second generation process.

[0041] Optionally, in some embodiments, intelligent testing of network devices can be achieved based on a generative adversarial mechanism. In this case, the first model can be, for example, the generative model in the generative adversarial mechanism. The distribution characteristics of the initial generated data can be obtained through the generative model, and intermediate generated data can be generated based on the distribution characteristics.

[0042] S104: The target real data and intermediate generated data are processed by the second model to obtain the target generated data. The second model is used to model and learn the distribution differences between the target real data and the intermediate generated data.

[0043] Optionally, in some embodiments, after obtaining the distribution characteristics of the initial generated data through the first model and generating intermediate generated data based on the distribution characteristics, the distribution difference between the target real data and the intermediate generated data can be modeled and learned based on the second model to learn the distribution difference, and the target generated data can be further modeled based on the "distribution difference".

[0044] Optionally, in some embodiments, intelligent testing of network devices can be achieved based on the generative adversarial mechanism. In this case, the second model can be, for example, the discriminative model in the generative adversarial mechanism, which can process the target real data and intermediate generated data to obtain the target generated data.

[0045] Optionally, in some embodiments, the second model is used to model the distribution difference between the target real data and the intermediate generated data. The objective of the modeling learning can be, for example, to minimize the difference between the distribution of the target generated data and the distribution of the target real data. This difference is the aforementioned distribution difference, which can also be referred to as KL divergence (i.e., relative entropy, information divergence).

[0046] Optionally, in some embodiments, the modeling and analysis processes of the first model and the second model described above can be performed simultaneously. For example, the first model and the second model can be jointly trained in multiple rounds of iteration, and the weights of the first model and / or the second model can be adjusted in each round of iteration until the first model and the second model converge to obtain the target generated data. For details, please refer to the following embodiments.

[0047] S105: Perform intelligent testing on network devices based on the target's real data and target-generated data.

[0048] Optionally, in some embodiments, after obtaining the target generated data, intelligent testing of the network device can be performed based on the target real data and the target generated data. This greatly enriches the target real data, and since the target generated data is generated based on the target real data, it has a strong correlation with the target real data and is richer than the target real data. Therefore, it can achieve generalized testing of the network device's intelligent capabilities, thereby greatly improving the comprehensiveness of the test results and enhancing the test effectiveness.

[0049] Optionally, in some embodiments, the intelligent testing method for network devices in this disclosure can be applied to an intelligent testing tool. An intelligent testing interface can be established between the intelligent testing tool and the network device. Target real data and target generated data can be sent to the network device through this intelligent testing interface. The target real data and target generated data are used to perform intelligent testing on the network device. This enables timely expansion, optimization, and supplementation of the target real data for the network device, and by promptly feeding back the target real data and target generated data to the network device, it supports subsequent timely intelligent testing of the network device, improving testing efficiency.

[0050] In this embodiment, target real data for the network device is determined, and initial generated data is determined based on the target real data. A first model is used to obtain the distribution characteristics of the initial generated data, and intermediate generated data is generated based on these distribution characteristics. A second model processes the target real data and the intermediate generated data to obtain target generated data. The second model is used to model and learn the distribution differences between the target real data and the intermediate generated data, and to perform intelligent testing on the network device based on the target real data and target generated data. This greatly enriches the target real data, and since the target generated data is generated based on the target real data, it has a strong correlation with the target real data and is richer than the target real data. Therefore, it can achieve generalized testing of the network device's intelligent capabilities, thereby greatly improving the comprehensiveness of the test results and enhancing the test effectiveness.

[0051] Figure 2 This is a flowchart illustrating an intelligent testing method for network devices proposed in another embodiment of this disclosure.

[0052] This embodiment illustrates the use of a network device intelligent testing method configured within a network device intelligent testing device. In this embodiment, the network device intelligent testing method can be configured within the network device intelligent testing device, which can be located in a server or an electronic device, without limitation.

[0053] It should be noted that the execution entity of the embodiments disclosed herein may be, in hardware, a central processing unit (CPU) in a server or electronic device, and in software, a related background service in a server or electronic device, without limitation.

[0054] like Figure 2 As shown, the intelligent testing method for this network device includes:

[0055] S201: Determine the target real data for network devices.

[0056] S202: Determine the initial generated data based on the target's actual data.

[0057] For a detailed description of S201-S202, please refer to the above embodiments, which will not be repeated here.

[0058] S203: In the current iteration of training, the distribution characteristics of the initial generated data are obtained by the first model based on the current weights of the first model, and the distribution characteristics are processed to obtain the first generated data.

[0059] Optionally, in some embodiments, the first model and / or the second model can be trained iteratively multiple times. In the current iteration training process, S203, S204, S206, and S207 can be executed once, or S203, S205, S206, and S208 can be executed once, until the first model and the second model converge. The network device is then intelligently tested based on the intermediate generated data output by the first model and the target generated data output by the second model.

[0060] In the current iteration of training, the weights of the first model can be referred to as the current first model weights. The first model can learn the distribution characteristics of the initial generated data under the first model weights, and generate corresponding generated data based on the learned distribution characteristics. This "corresponding generated data" can be referred to as the first generated data.

[0061] In other words, the first model can be trained in multiple rounds of iterative training. If the first generated data can be obtained in the previous round of iterative training, it can be determined whether the first model has converged after the previous round of iterative training based on the first generated data, and whether to carry out the next round of iterative training based on the result of convergence.

[0062] S204: If the convergence of the first model is determined based on the first generated data, the first generated data is determined as intermediate generated data.

[0063] Optionally, in some embodiments, during the process of determining the convergence of the first model based on the first generated data, the first generated data can be provided to the second model simultaneously. The second model can process the target real data and the first generated data to obtain the second generated data, and determine the distribution difference between the target real data and the second generated data. If the distribution difference satisfies a difference condition, the first model is determined to have converged. That is, the training process of the first model and the training process of the second model can be mutually constrained. During each iteration of training the first model, the convergence of the first model can be determined simultaneously based on the output result of the second model (e.g., the second generated data obtained based on the first generated data). Furthermore, a difference condition can be preset (e.g., maximizing or minimizing the distribution difference). If the distribution difference between the target real data and the second generated data satisfies the preset difference condition, the first model is determined to have converged, and the first generated data output by the first model in this iteration is determined as intermediate generated data.

[0064] S205: If it is determined that the first model has not converged based on the first generated data, the weights of the first model are adjusted, and the first model is trained in the next iteration based on the adjusted weights, until the first model is determined to have converged based on the generated data output by the first model, and the output generated data is determined as intermediate generated data.

[0065] Optionally, in some embodiments, if it is determined from the first generated data that the first model has not converged, the weights of the first model can be adjusted, and the first model can be trained in the next iteration based on the adjusted weights, until it is determined from the generated data output by the first model that the first model has converged, and the output generated data is determined as intermediate generated data. The method for analyzing whether the first model has converged in subsequent iterations can be found in the method for determining whether the first model has converged in the current iteration, and will not be elaborated further here.

[0066] S206: In the current iteration of training, the second model obtains the current distribution difference between the target real data and the first generated data based on the current weights of the second model, and generates the second generated data based on the current distribution difference.

[0067] In the current iteration of training, the weights of the second model can be referred to as the current weights of the second model.

[0068] In addition, during the current iteration of training, the distribution characteristics of the initial generated data can be obtained by the first model based on the current weights of the first model, and the distribution characteristics can be processed to obtain the first generated data. Then, the first generated data can be provided to the second model to support the training steps of the second model during the current iteration of training. The second model can model and learn the distribution difference between the target real data and the first generated data based on the current weights of the second model to obtain the current distribution difference, and obtain the second generated data based on the current distribution difference.

[0069] Optionally, in some embodiments, during the current iteration of training, the second generated data output by the second model can be used not only to determine whether the first model has converged, but also to determine whether the second model has converged, as detailed in the following description.

[0070] S207: If the convergence of the second model is determined based on the target real data and the second generated data, the second generated data shall be determined as the target generated data.

[0071] Optionally, in some embodiments, in the process of determining the convergence of the second model based on the target real data and the second generated data, the distribution difference between the target real data and the second generated data may be determined, and the convergence of the second model may be determined when the distribution difference satisfies the difference condition.

[0072] S208: If it is determined that the second model has not converged based on the target real data and the second generated data, the weights of the second model are adjusted, and the second model is trained in the next iteration based on the adjusted weights, until the second model is determined to have converged based on the generated data output by the second model, and the output generated data is determined as the target generated data.

[0073] Optionally, in some embodiments, if it is determined that the second model has not converged based on the target real data and the second generated data, the weights of the second model can be adjusted, and the second model can be trained in the next iteration based on the adjusted weights, until the second model is determined to have converged based on the second generated data output by the second model, and the output generated data is determined as the target generated data. The method for analyzing whether the second model has converged in subsequent iterations can be found in the method for determining whether the second model has converged in the current iteration, and will not be elaborated here.

[0074] S209: Perform intelligent testing on network devices based on the target's real data and target-generated data.

[0075] Optionally, in some embodiments, after obtaining the target generated data, intelligent testing of the network device can be performed based on the target real data and the target generated data. This greatly enriches the target real data, and since the target generated data is generated based on the target real data, it has a strong correlation with the target real data and is richer than the target real data. Therefore, it can achieve generalized testing of the network device's intelligent capabilities, thereby greatly improving the comprehensiveness of the test results and enhancing the test effectiveness.

[0076] Optionally, in some embodiments, the intelligent testing method for network devices in this disclosure can be applied to an intelligent testing tool. An intelligent testing interface can be established between the intelligent testing tool and the network device. Target real data and target generated data can be sent to the network device through this intelligent testing interface. The target real data and target generated data are used to perform intelligent testing on the network device. This enables timely expansion, optimization, and supplementation of the target real data for the network device, and by promptly feeding back the target real data and target generated data to the network device, it supports subsequent timely intelligent testing of the network device, improving testing efficiency.

[0077] In this embodiment, the target real data is greatly enriched, and the target generated data is generated based on the target real data. Therefore, the target generated data has a strong correlation with the target real data and is richer than the target real data. Thus, it is possible to perform generalized testing on the intelligent capabilities of network devices, thereby greatly improving the comprehensiveness of the test results and the test effect.

[0078] Examples of the above embodiments can be illustrated as follows:

[0079] An example is given using network equipment as the base station and the execution subject of intelligent testing methods for network equipment as the testing tool (such as the intelligent testing tool mentioned above).

[0080] The testing method provided in this embodiment introduces a generative adversarial mechanism, enabling the testing tool to generate more test samples based on the initial real data (an optional example of the target generated data mentioned above). These samples are related to the initial real data and can achieve diversity in the test samples. In turn, it provides an evaluation result on the generalization of intelligent capabilities and locates the defects of intelligent capabilities based on the evaluation result.

[0081] like Figure 3 As shown, Figure 3This is a schematic diagram of a test scenario in an embodiment of this disclosure. The base station can provide communication services to the UE (User Equipment) through the Operations Management Center (OMC). Intelligent testing tools can be used to perform intelligent testing on the base station. The intelligent testing tools and the base station can connect via an intelligent testing interface (Tapi) to transmit data.

[0082] As above Figure 3 As shown, the intelligent testing tool can connect to the base station through the intelligent testing interface (Tapi) to obtain the live network data of the base station under test, which serves as the real dataset in the model below (an optional example of the initial real data mentioned above). This intelligent testing interface (Tapi) is used to transmit live network data from the base station side (the specific data type is defined by the tester, such as network standard, coverage type, station type, frequency band, etc.) and generalizable generated data produced by the intelligent testing tool (an optional example of the target generated data mentioned above). The instantiated interface is shown in Table 1 below:

[0083] Table 1

[0084]

[0085] Specifically, the intelligent testing tool defines service assurance characteristics in five dimensions: network standard, coverage type, site type, frequency band, and service type. Network standard includes: whether it is a 5G base station or a 4G base station; coverage type includes: whether it is in urban areas, scenic areas, or suburban areas; site type includes: whether it is a macro base station, an emergency base station, an indoor distributed antenna system (DAS), or a small base station; frequency band includes: whether it is 700MHz, 2.6GHz, or 4.9GHz; and service type includes: whether it is short video, live streaming, or long video.

[0086] In this embodiment of the disclosure, the specific process by which the intelligent testing tool generates generalizable generated data (an optional example of the aforementioned target generated data) can be as follows: Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the process by which the intelligent testing tool in this embodiment generates generalizable generated data. Figure 4The real dataset in the above can be an optional example of the initial real data, the initial generated data can be an optional example of the initial generated data, the generating model can be an optional example of the first model, the discriminant model can be an optional example of the second model, the weights of the generating model can be an optional example of the weights of the first model, the weights of the discriminant model can be an optional example of the weights of the second model, and the "new generated data" can be an optional example of the intermediate generated data.

[0087] Step 1: Perform one-hot encoding on each sample feature in the acquired real dataset (an optional example of the initial real data mentioned above), thereby obtaining the encoded real dataset (the encoded real dataset is an optional example of the encoded initial real data mentioned above). After performing one-hot encoding on the sample features in the real dataset, the real dataset X = {x1, x2, ..., x...} is obtained. n}, where n is the number of data points and n is a positive integer. The initial generated data Z = {z1, z2, ..., zn} is obtained by linear fitting the real dataset. n (An optional example of the initial generated data described above). Where, x i The meaning is a sample of service assurance characteristics in the current network environment, consisting of the above five dimensions, such as: network standard is 5G, coverage type is urban area, site type is macro site, frequency band is 700M, and service type is long video. i The meaning is to generate samples of business assurance features composed of the above four dimensions, such as: network standard is TD-LTE, coverage type is suburban county, station type is emergency station, frequency band is 700M, and business type is live broadcast.

[0088] Step 2: Establish the generative and discriminative models in the generative-adversarial mechanism (e.g., Figure 5 As shown, Figure 5 This is a schematic diagram of the generative-adversarial mechanism in this embodiment of the present disclosure. Both the generative and discriminative models use neural networks with three hidden layers, each containing four neurons. The input and output layers of the generative model have the same number of neurons. In the discriminative model, the input layer includes both real and generated data, and the output layer has only one neuron, outputting 0 / 1. The weights θ of the neural networks in the generative and discriminative models are initialized. g and θ d .

[0089] Step 3: Generate the initial data Z = {z1, z2, ..., z} n The input is fed into the generative model, and its output is the new generated data. (An optional example of the first generated data mentioned above). At this point... This can be understood as improved generated data; through weight adjustments, the generated data is continuously optimized. For example: z i The generated data, representing a 5G network standard, suburban coverage, emergency station type, 700MHz frequency band, and long video service type, may be output as generated data with a 5G network standard, suburban coverage, remote station type, 700MHz frequency band, and long video service type.

[0090] Step 4: The generative-adversarial mechanism consists of two phases: generation and adversarial. Step 3 corresponds to the generation phase. In the adversarial phase, the network's objective is to minimize the difference between the generated data distribution and the real data distribution, i.e., the KL divergence. Therefore, the objective function of the discriminant model can be expressed as follows:

[0091]

[0092] Here, G(x) is the output value of the discriminant model for data x. For real data, a larger value is better; for generated data, a smaller value is better. Based on this, the objective function is maximized (if the training objective is to maximize the objective function, the above-mentioned difference conditions can be configured accordingly). The weights θ of the neural networks in the discriminant model and the generator model are trained and updated sequentially using gradient ascent. g and θ d θ g As an optional example of the weights in the second model described above, θ d This is an optional example of the weights for the first model mentioned above.

[0093] Step 5: Repeat steps 3 and 4 until the neural networks in both the discriminant and generative models converge.

[0094] Then, the union of the generated dataset (an optional example of the target generated data) obtained based on the above method and the real dataset (an optional example of the target real data) can be transmitted to the base station side through the intelligent test interface (Tapi) to complete the specific test process as test data in subsequent tests.

[0095] Figure 6 This is a schematic diagram of the structure of an intelligent testing device for network devices proposed in an embodiment of this disclosure.

[0096] like Figure 6 As shown, the intelligent testing device 60 for network equipment includes:

[0097] The first determining module 601 is used to determine the target real data of the network device.

[0098] The second determining module 602 is used to determine the initial generated data based on the target real data.

[0099] The first acquisition module 603 is used to acquire the distribution characteristics of the initial generated data through the first model, and generate intermediate generated data based on the distribution characteristics.

[0100] The second acquisition module 604 is used to process the target real data and intermediate generated data through the second model to obtain the target generated data. The second model is used to model and learn the distribution differences between the target real data and the intermediate generated data.

[0101] Test module 605 is used to perform intelligent testing on network devices based on real target data and target generated data.

[0102] It should be noted that the foregoing explanation of the intelligent testing method for network devices also applies to the intelligent testing device for network devices in this embodiment, and will not be repeated here.

[0103] In this embodiment, target real data for the network device is determined, and initial generated data is determined based on the target real data. A first model is used to obtain the distribution characteristics of the initial generated data, and intermediate generated data is generated based on these distribution characteristics. A second model processes the target real data and the intermediate generated data to obtain target generated data. The second model is used to model and learn the distribution differences between the target real data and the intermediate generated data, and to perform intelligent testing on the network device based on the target real data and target generated data. This greatly enriches the target real data, and since the target generated data is generated based on the target real data, it has a strong correlation with the target real data and is richer than the target real data. Therefore, it can achieve generalized testing of the network device's intelligent capabilities, thereby greatly improving the comprehensiveness of the test results and enhancing the test effectiveness.

[0104] Figure 7 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 7 The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0105] like Figure 7 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, memory 28, and bus 18 connecting different system components (including memory 28 and processing unit 16).

[0106] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0107] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0108] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive".

[0109] although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0110] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0111] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0112] The processing unit 16 executes various functional applications and data processing by running programs stored in the memory 28, such as implementing the intelligent testing method for network devices mentioned in the foregoing embodiments.

[0113] To implement the above embodiments, this disclosure also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0114] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0115] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0116] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0118] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0119] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0120] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0121] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0123] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0124] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0125] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0126] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for intelligent testing of network devices, characterized in that, include: Determine the target real data of the network device; Based on the target real data, determine the initial generated data; The distribution characteristics of the initial generated data are obtained through the first model, and intermediate generated data are generated based on the distribution characteristics; The target real data and the intermediate generated data are processed by the second model to obtain target generated data, wherein the second model is used to model and learn the distribution differences between the target real data and the intermediate generated data; as well as The network device is intelligently tested based on the target's actual data and the target's generated data.

2. The method according to claim 1, characterized in that, Determining the target real data of the network device includes: Collect initial real data from the network device, wherein the initial real data includes at least one sample feature; At least one sample feature in the initial real data is encoded to obtain at least one encoded feature; The target real data is determined based on the at least one encoded feature.

3. The method according to claim 2, characterized in that, The step of determining the initial generated data based on the target real data includes: The initial generated data is obtained by linearly fitting at least one encoded feature in the target real data.

4. The method according to claim 1, characterized in that, The step of obtaining the distribution characteristics of the initial generated data through the first model and generating intermediate generated data based on the distribution characteristics includes: In the current iteration of training, the distribution characteristics of the initial generated data are obtained by the first model based on the current weights of the first model, and the distribution characteristics are processed to obtain the first generated data; If the convergence of the first model is determined based on the first generated data, the first generated data is determined as the intermediate generated data; If it is determined that the first model has not converged based on the first generated data, the weights of the first model are adjusted, and the first model is trained in the next iteration based on the adjusted first model weights, until it is determined that the first model has converged based on the generated data output by the first model, and the output generated data is determined as the intermediate generated data.

5. The method according to claim 4, characterized in that, The step of determining the convergence of the first model based on the first generated data includes: The target real data and the first generated data are processed according to the second model to obtain the second generated data; Determine the distribution differences between the target real data and the second generated data; If the distribution difference satisfies the difference condition, the first model is determined to be converged.

6. The method according to claim 5, characterized in that, The process of processing the target real data and the intermediate generated data through the second model to obtain the target generated data includes: During the current iteration of training, the second model obtains the current distribution difference between the target real data and the first generated data based on the current weights of the second model, and generates the second generated data based on the current distribution difference; If the convergence of the second model is determined based on the target real data and the second generated data, the second generated data is determined as the target generated data; If the second model fails to converge based on the target real data and the second generated data, the weights of the second model are adjusted, and the second model is trained in the next iteration based on the adjusted weights, until the second model is determined to have converged based on the generated data output by the second model, and the output generated data is determined as the target generated data.

7. The method according to claim 6, characterized in that, The step of determining the convergence of the second model based on the target real data and the second generated data includes: Determine the distribution differences between the target real data and the second generated data; If the distribution difference satisfies the difference condition, the second model is determined to be convergent.

8. The method according to claim 1, characterized in that, The intelligent testing of the network device based on the target's real data and the target's generated data includes: The target real data and the target generated data are sent to the network device through an intelligent testing interface between the network device and the network device. The target real data and the target generated data are used to perform intelligent testing on the network device.

9. An intelligent testing device for network equipment, characterized in that, include: The first determining module is used to determine the target real data of the network device; The second determining module is used to determine the initial generated data based on the target real data; The first acquisition module is used to acquire the distribution characteristics of the initial generated data through the first model, and generate intermediate generated data based on the distribution characteristics; The second acquisition module is used to process the target real data and the intermediate generated data through the second model to obtain target generated data, wherein the second model is used to model and learn the distribution differences between the target real data and the intermediate generated data; The testing module is used to perform intelligent testing on the network device based on the target's real data and the target's generated data.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images