Network analysis system, network analysis method, and network analysis program

Abstract

Conduct analysis of specific communications networks to generate tailored advice. [Solution] One aspect of the present invention is a network analysis system comprising: an input unit that inputs performance data of a target network device; a processing unit that generates processed data by processing the performance data based on the performance data and a threshold; an analysis unit that inputs the processed data into a first machine learning model trained using a training dataset including network configuration data of the network device, the processed data, and analysis results indicating past trends and future trends, and acquires the analysis results output from the first machine learning model; and a generation unit that inputs the analysis results acquired by the analysis unit into a second machine learning model trained using training data including the analysis results and information indicating a response policy, and generates advice information based on response policy information indicating the response policy output from the second machine learning model.

Description

[Technical Field]

[0001] The present invention relates to a network analysis system, a network analysis method, and a network analysis program. [Background technology]

[0002] Techniques for analyzing communication networks and dealing with changes in the communication networks are known. For example, as described in Patent Document 1, this type of technique predicts network load for a prediction period from a reference point to a predetermined time after the reference point based on performance index values ​​related to the communication network before the reference point, and determines whether the magnitude of the predicted network load value for the prediction period satisfies a predetermined condition based on the network load prediction. This technique performs a determination for each of two or more prediction periods that have different reference points and at least a partial overlap, and if a predetermined number or more determinations that the predetermined condition is satisfied are made, scale-out of elements included in the communication network is performed. In other words, the technique described in Patent Document 1 aims to automatically deal with problems that occur in a telecommunications carrier's huge network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2024-004103 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 above is intended to automatically handle large-scale network services provided by telecommunications carriers, and therefore has the problem of being unable to provide detailed advice for specific communication networks such as corporate WANs and LANs. There are also products known that report management data related to network performance, but most of them analyze and score performance data, and the analysis and proposal of what kind of measures are needed must be carried out manually.

[0005] The present invention has been made in consideration of these circumstances, and one of its objects is to provide a network analysis system, a network analysis method, and a network analysis program that can analyze a specific communication network and create detailed advice. [Means for solving the problem]

[0006] One aspect of the present invention is a network analysis system comprising: an input unit that inputs performance data of a target network device; a processing unit that generates processed data by processing the performance data based on the performance data and a threshold; an analysis unit that inputs the processed data to a first machine learning model trained using a training dataset including network configuration data of the network device, the processed data, and analysis results indicating past trends and future trends, and acquires the analysis results output from the first machine learning model; and a generation unit that inputs the analysis results acquired by the analysis unit to a second machine learning model trained using training data including the analysis results and information indicating a response policy, and generates advice information based on response policy information indicating the response policy output from the second machine learning model.

[0007] One aspect of the present invention is a network analysis method including the steps of: inputting performance data of a target network device by a computer; generating processed data by processing the performance data based on the performance data and a threshold; inputting the processed data into a first machine learning model trained with a training dataset including network configuration data of the network device, the processed data, and analysis results indicating past trends and future trends, and obtaining analysis results output from the first machine learning model; and inputting the obtained analysis results into a second machine learning model trained with training data including the analysis results and information indicating a response policy, and generating advice information based on response policy information indicating the response policy output from the second machine learning model.

[0008] One aspect of the present invention is a network analysis program that causes a computer to execute the following steps: inputting performance data of a target network device; generating processed data by processing the performance data based on the performance data and a threshold; inputting the processed data into a first machine learning model trained using a training dataset including network configuration data of the network device, the processed data, and analysis results indicating past trends and future trends, and obtaining analysis results output from the first machine learning model; and inputting the obtained analysis results into a second machine learning model trained using training data including the analysis results and information indicating a response policy, and generating advice information based on response policy information indicating the response policy output from the second machine learning model. [Effects of the Invention]

[0009] According to one aspect of the present invention, analysis can be performed on a specific communication network to generate tailored advice. [Brief explanation of the drawings]

[0010] [Figure 1]1 is a diagram illustrating an example of a configuration of a network system 1 for a company according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of a functional configuration of a network analysis system 120 according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of performance data D11 according to the first embodiment. [Figure 4] FIG. 10 is a diagram showing an example of region designation data D12 according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a threshold value D21 according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of processed data D31 for traffic data in the first embodiment. [Figure 7] FIG. 10 is a diagram showing an example of processed data D32 for CPU memory data in the first embodiment. [Figure 8] FIG. 10 is a diagram showing an example of processed data D33 for error data in the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of generated data D41 for IN traffic data in the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of generated data D41 for OUT traffic data in the first embodiment. [Figure 11] FIG. 10 is a diagram showing an example of generated data D41 for traffic data in the first embodiment. [Figure 12] FIG. 10 is a diagram showing an example of generated data D42 for a CPU according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing an example of generated data D42 for a CPU according to the first embodiment. [Figure 14] FIG. 10 is a diagram showing an example of generated data D42 for a memory in the first embodiment. [Figure 15] FIG. 10 is a diagram showing an example of generated data D42 for a memory in the first embodiment. [Figure 16]FIG. 10 is a diagram showing an example of generated data D43 for error data in the first embodiment. [Figure 17] FIG. 10 is a diagram showing an example of generated data D43 regarding an IF error in the first embodiment. [Figure 18] FIG. 10 is a diagram showing an example of generated data D43 for discards in the first embodiment. [Figure 19] FIG. 2 is a diagram showing an example of a summary document in the first embodiment. [Figure 20] FIG. 2 shows an example of a configuration diagram of a network device 110 according to the first embodiment. [Figure 21] 10 is a flowchart showing an example of a processing procedure of a network analysis system 120 according to the first embodiment. [Figure 22] FIG. 10 is a block diagram illustrating an example of a network analysis system 120 according to a second embodiment. [Figure 23] FIG. 10 is a block diagram illustrating an example of a learning system 400 according to a second embodiment. [Figure 24] FIG. 10 is a diagram showing an example of network configuration data D61 in the second embodiment. [Figure 25] FIG. 13 is a diagram showing an example of customer case trend and needs data D62 in the second embodiment. [Figure 26] FIG. 11 is a diagram showing an example of correspondence patterns and threshold data D71 according to the second embodiment. [Figure 27] FIG. 10 is a diagram showing an example of a learning set D81 in the second embodiment. [Figure 28] FIG. 10 is a diagram showing an example of a learning set D82 in the second embodiment. [Figure 29] FIG. 10 is a diagram showing an example of a learning set D83 in the second embodiment. [Figure 30] 10 is a flowchart showing an example of a processing procedure of a network performance data learning device 410 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a network analysis system, a network analysis method, and a network analysis program according to the present invention will be described with reference to the drawings.

[0012] FIG. 1 is a diagram showing an example of the configuration of a network system 1 for a company according to the first embodiment. The network system 1 includes, for example, a corporate LAN data center 100 and multiple corporate LAN systems 200A, 200B, etc. In the following description, the corporate LAN systems will be collectively referred to simply as "corporate LAN system 200." The corporate LAN data center 100 and the corporate LAN systems 200 are connected via a communications network NW such as a public network that implements a WAN or the like.

[0013] The corporate LAN data center 100 includes, for example, a network device 110 and a network analysis system 120. The network device 110 includes, for example, a router (RT) 110a, a load balancer (LB) 110b, a firewall (FW) 110c, an intrusion detection system (IDS) / intrusion prevention system (IPS) 110d, a layer 2 switch (L2SW) 110e, and a layer 3 switch (L3SW) 110f. The network analysis system 120 is connected to the network device 110 and performs network analysis processing on the network device 110.

[0014] The corporate LAN systems 200 include network devices 210 including, for example, an RT 210a and an L2SW 210b. Each of the corporate LAN systems 200 is located, for example, at a different base.

[0015] FIG. 2 is a block diagram showing an example of the functional configuration of network analysis system 120 according to the first embodiment. The network analysis system 120 includes, for example, a network performance data analysis device 120A, a data storage unit 140, and a learned model storage unit 150. The network performance data analysis device 120A includes, for example, an input unit 122, a processing unit 124, a traffic data trend analysis unit 126, a traffic data trend analysis unit 128, a CPU memory data trend analysis unit 130, a CPU memory data trend analysis unit 132, an error data trend analysis unit 134, an error data trend analysis unit 136, and a report generation unit 138. The input unit 122, the processing unit 124, the traffic data trend analysis unit 126, the traffic data trend analysis unit 128, the CPU memory data trend analysis unit 130, the CPU memory data trend analysis unit 132, the error data trend analysis unit 134, the error data trend analysis unit 136, and the report generation unit 138 are functional units realized by a processor (computer) such as a CPU executing a network analysis program.

[0016] The input unit 122 receives performance data D11 for a predetermined period of a target network device. The input unit 122 may receive part designation data D12 that designates a target network device from among a plurality of network devices. 3 is a diagram showing an example of performance data D11 in the first embodiment. The performance data D11 is data that associates the host name, interface, data acquisition date and time, IN traffic volume, OUT traffic volume, number of errors, number of discards, CPU usage rate, and memory usage rate. Discards indicate that packets are discarded in a network device due to reasons such as exceeding the buffer capacity. 4 is a diagram showing an example of the part designation data D12 in the first embodiment. The part designation data D12 is data indicating a part designated by a customer, and is data that associates, for example, a host name, an interface, and a period.

[0017] The processing unit 124 processes the performance data D11 based on the performance data D11 and a threshold value D21 to generate processed data D31, D32, and D33. The processing unit 124 may process the performance data D11 of the part specified by the part specification data D12 based on the performance data D11, the part specification data D12, and the threshold value D21 to generate processed data.

[0018] 5 is a diagram showing an example of the threshold value D21 in the first embodiment. The threshold value D21 is data indicating a threshold value to be compared with the performance data D11, and is, for example, data associating data types, configuration patterns, and threshold values. The threshold value is a value for determining whether or not the network devices 110, 210 require action, and a value is set for each configuration pattern of the network devices 110, 210 when the traffic volume, CPU usage rate, memory usage rate, number of errors, and number of discards are high and it is determined that action is required. The threshold value may be set to a predetermined value, or may be set to a dynamically calculated value. FIG. 6 is a diagram showing an example of processed data D31 for traffic data in the first embodiment. The processed data D31 is data in which, for example, a host name, an interface, IN or OUT data for the host, the number of times a threshold is exceeded, an average traffic volume, a maximum traffic volume, and the date and time of maximum traffic measurement are associated. In the description of FIG. 6, for example, "a or b Mbps" indicates that the maximum value of "a Mbps" or "b Mbps" is set when the data of FIG. 3 is processed, and "aabb Mbps" indicates that the average value of "aa Mbps" and "bb Mbps" is set when the data of FIG. 3 is processed. In other words, FIG. 6 is a schematic table explaining the contents of the set values, and the following tables are similar. Furthermore, the processed data D31 for the traffic data in the first embodiment may be aggregated data (not shown). For example, it may be data obtained by aggregating performance data D11 for a predetermined period stored in the data storage unit 140, or data obtained by aggregating past performance data D11 (including the predetermined period) stored in the data storage unit 140. Aggregated data required for displaying FIG. 9, which shows an example of generated data D41 for IN traffic data in the first embodiment described later, may be further added. Aggregated data required for displaying FIG. 10, which shows an example of generated data D41 for OUT traffic data in the first embodiment described later, may be further added. Aggregated data required for displaying FIG. 11, which shows an example of generated data D41 for traffic data in the first embodiment described later, may be further added. For example, aggregated data related to part of the information in FIG. 11 (e.g., including the amount of increase / decrease, the number of times the threshold is exceeded, the average value band, the peak value band, and the peak value usage rate, but excluding the trend) may be further added. 7 is a diagram showing an example of processed data D32 for CPU memory data in the first embodiment. The processed data D32 is data in which, for example, a host name, a CPU threshold exceedance count, an average CPU usage rate, a maximum CPU usage rate, a date and time when the maximum CPU usage rate was measured, a memory threshold exceedance count, an average memory usage rate, a maximum memory usage rate, and a date and time when the maximum memory usage rate was measured are associated with each other. Furthermore, the processed data D32 for the CPU memory data in the first embodiment may be aggregated data (not shown). For example, the processed data D32 may be data obtained by aggregating performance data D11 for a predetermined period stored in the data storage unit 140, or data obtained by aggregating past performance data D11 (including the predetermined period) stored in the data storage unit 140. Aggregated data required for displaying FIG. 12, which shows an example of generated data D42 for the CPU in the first embodiment (to be described later), may be further added. Aggregated data required for displaying FIG. 14, which shows an example of generated data D42 for the memory in the first embodiment (to be described later), may be further added. Aggregated data required for displaying FIG. 13, which shows an example of generated data D41 for the CPU in the first embodiment (to be described later), may be further added. Aggregated data required for displaying FIG. 15, which shows an example of generated data D41 for the memory in the first embodiment (to be described later), may be further added. For example, aggregated data related to part of the information in FIGS. 13 and 15 (e.g., including the amount of increase / decrease, the number of times the threshold is exceeded, the average usage rate, the peak usage rate, and the peak date and time, but not including the trend) may be further added. 8 is a diagram showing an example of processed data D33 for error data in the first embodiment. The processed data D32 is data in which, for example, a host name, an interface, the number of times an error occurrence threshold is exceeded, the monthly cumulative number of errors, the maximum number of errors, the date on which the maximum number of errors was measured, the number of times a discard occurrence threshold is exceeded, the monthly cumulative number of discards, the maximum number of discards, and the date on which the maximum number of discards was measured are associated with each other. Furthermore, the processed data D33 for the error data in the first embodiment may be aggregated data (not shown). For example, it may be data obtained by aggregating performance data D11 for a predetermined period stored in the data storage unit 140, or data obtained by aggregating past performance data D11 (including the predetermined period) stored in the data storage unit 140. Aggregated data required for displaying FIG. 16 , which shows an example of generated data D43 for error data in the first embodiment (to be described later), may be further added. Aggregated data required for displaying FIG. 17 , which shows an example of generated data D43 for IF errors in the first embodiment (to be described later), may be further added. Aggregated data required for displaying FIG. 18 , which shows an example of generated data D43 for discards in the first embodiment (to be described later), may be further added. For example, aggregated data related to information other than the trends in FIGS. 17 and 18 (e.g., including the amount of increase / decrease, the number of times that the threshold is exceeded, the monthly cumulative number of occurrences, the number of peak value occurrences, and the peak value date and time, but not including the trend) may be further added.

[0019] The traffic data trend analysis unit 126, the traffic data trend analysis unit 128, the CPU memory data trend analysis unit 130, the CPU memory data trend analysis unit 132, the error data trend analysis unit 134, and the error data trend analysis unit 136 function as an analysis unit that acquires analysis results. The analysis unit inputs the processed data to a first machine learning model that has been trained using a training dataset that includes the network configuration of the network device 110, the processed data, and analysis results that indicate past trends and future trends in network performance, and acquires the analysis results output from the first machine learning model.

[0020] The first machine learning model may include a third machine learning model and a fourth machine learning model. The third machine learning model is trained to output trend information indicating past trends when processed data is input. The fourth machine learning model is trained to output transition information indicating future transitions when processed data is input. Specifically, in the analysis unit of the first embodiment, the first machine learning model includes a trained model for trend analysis for analyzing trends and a trained model for transition analysis for analyzing transitions, and the trained model for trend analysis corresponds to the third machine learning model, and the trained model for transition analysis corresponds to the fourth machine learning model. Specifically, the analysis unit is configured as follows.

[0021] The traffic data trend analysis unit 126 inputs processed data D31 to a trained model M10 (first machine learning model) for traffic data trend analysis, which has been trained using a training dataset including a network configuration, processed data, and analysis results showing past trends in network performance, and acquires the analysis results output from the trained model M10 for traffic data trend analysis. The traffic data trend analysis unit 128 inputs processed data D31 to a trained model M12 (first machine learning model) for traffic data trend analysis, which has been trained using a training dataset including a network configuration, processed data, and analysis results showing future trends, and acquires the analysis results output from the trained model M12 for traffic data trend analysis. The analysis results acquired by the traffic data trend analysis unit 126 and the traffic data trend analysis unit 128 are output to the report generation unit 138 as generated data D41.

[0022] 9 is a diagram showing an example of generated data D41 for IN traffic data in the first embodiment. The generated data D41 for IN traffic data includes charts and graphs showing time-series changes in traffic transition, bandwidth limit value, and average traffic. The time-series changes include at least future time-series changes, and may also include past time-series changes (including a predetermined period). 10 is a diagram showing an example of generated data D41 for OUT traffic data in the first embodiment. The generated data D41 for OUT traffic data includes a chart graph showing time-series changes in traffic transition, thresholds, and average traffic. The time-series changes include at least future time-series changes and may also include past time-series changes (including a predetermined period). 11 is a diagram showing an example of generated data D41 for traffic data in the first embodiment. The generated data D41 for traffic data includes charts and graphs showing one-month trends and three-month trends. The trends include at least future trends and may also include past trends (including a predetermined period). The generated data D41, which is the analysis result output from the traffic data trend analysis unit 126, is the analysis result of trend information obtained by inputting the processed data D31, which is data obtained by processing the performance data D11 for a specified period input from the input unit 122, into a trained model M10 for traffic data trend analysis that has been trained using a learning dataset that includes analysis results showing past trends in network configuration, processed data, and network performance, and may be a chart or graph showing trends (e.g., trends, threshold exceedance status, peak values, etc.) related to traffic data for a period including the specified period (e.g., if the specified period is the current month, three months including the current month). For example, the trend in Figure 11 is not simply a selection from increase, decrease, or no change, but is the result of analyzing trend information obtained by inputting the threshold exceedance and peak value into the trained model M10 for traffic data trend analysis, and the threshold exceedance and peak value can be judged in total from past learning and can be considered an increase only if an increasing trend of a certain level or more is observed, a decrease only if a decreasing trend of a certain level or more is observed, and no change in other cases, and values ​​other than the trend in Figure 11 can be included in the processed data D31. The generated data D41, which is the analysis result output from the traffic data trend analysis unit 128, is the analysis result of trend information obtained by inputting processed data D31, which is data obtained by processing performance data D11 for a predetermined period input from the input unit 122, into a trained model M12 for traffic data trend analysis that has been trained using a learning dataset including analysis results showing future trends in network configuration, processed data, and network performance, and may be a chart / graph showing time series changes that are future trends in traffic data, or a chart / graph showing time series changes that are future and past trends.

[0023] The CPU memory data trend analysis unit 130 inputs processed data D32 into a trained model M16 (first machine learning model) for CPU memory data trend analysis, which has been trained using a training dataset including analysis results showing past trends in network configuration, processed data, and performance, and acquires the analysis results output from the trained model M16 for CPU memory data trend analysis. The CPU memory data trend analysis unit 132 inputs processed data D32 into a trained model M18 (first machine learning model) for CPU memory data trend analysis, which has been trained using a training dataset including analysis results showing future trends in network configuration, processed data, and performance, and acquires the analysis results output from the trained model M18 for CPU memory data trend analysis. The analysis results acquired by the CPU memory data trend analysis unit 130 and the CPU memory data trend analysis unit 132 are output to the report generation unit 138 as generated data D42.

[0024] 12 is a diagram showing an example of generated data D42 about the CPU in the first embodiment. The generated data D42 about the CPU includes charts and graphs showing the transition of CPU utilization, thresholds, and time-series changes in average CPU utilization. 13 is a diagram showing an example of the generated data D42 for the CPU in the first embodiment. The generated data D42 for the CPU includes charts and graphs showing one-month trends and three-month trends. 14 is a diagram showing an example of generated data D42 about the memory in the first embodiment. The generated data D42 about the memory includes chart graphs showing memory usage transitions, thresholds, and time-series changes in average memory usage. 15 is a diagram showing an example of generated data D42 about the memory in the first embodiment. The generated data D42 about the memory includes charts and graphs showing one-month trends and three-month trends. The generated data D42, which is the analysis result output from the CPU memory data trend analysis unit 130, is the analysis result of trend information obtained by inputting the processed data D32, which is data obtained by processing the performance data D11 for a specified period input from the input unit 122, into a trained model M16 for CPU memory data trend analysis that has been trained using a learning dataset that includes analysis results showing past trends in network configuration, processed data, and performance, and may be a chart or graph showing trends (e.g., increase / decrease trends, threshold exceedance status, peak values, etc.) related to CPU and memory data for a period including the specified period (e.g., if the specified period is the current month, three months including the current month). For example, the trends in Figures 13 and 15 are not simply a selection from increase, decrease, or no change, but are the analysis results of trend information obtained by inputting the threshold exceedance and peak value into the trained model M16 for CPU memory data trend analysis, and the threshold exceedance and peak value can be judged in total from past learning and be considered to be an increase only if an increasing trend of a certain level or more is observed, a decrease only if a decreasing trend of a certain level or more is observed, and no change in other cases, and values ​​other than the trends in Figures 13 and 15 can be included in the processed data D31. The generated data D42, which is the analysis result output from the CPU memory data trend analysis unit 132, is the analysis result of trend information obtained by inputting processed data D32, which is data obtained by processing performance data D11 for a predetermined period input from the input unit 122, into a trained model M18 for CPU memory data trend analysis that has been trained using a learning dataset including network configuration, processed data, and analysis results showing future trends, and may be a chart graph showing time series changes that are future trends related to CPU and memory data, or a chart graph showing time series changes that are future and past trends.

[0025] The error data trend analysis unit 134 inputs the processed data D33 into a trained model M20 (first machine learning model) for error data trend analysis, which has been trained using a training dataset including analysis results showing past trends in network configuration, processed data, and performance, and acquires the analysis results output from the trained model M20 for error data trend analysis. The error data transition analysis unit 136 inputs the processed data D33 into a trained model M22 (first machine learning model) for error data trend analysis, which has been trained using a training dataset including analysis results showing future trends in network configuration, processed data, and performance, and acquires the analysis results output from the trained model M22 for error data transition analysis. The analysis results acquired by the error data trend analysis unit 134 and the analysis results acquired by the error data transition analysis unit 136 are output to the report generation unit 138 as generated data D43.

[0026] 16 is a diagram showing an example of generated data D43 for error data in the first embodiment. The generated data D43 for error data includes chart graphs showing IF (interface) errors, discards, and time-series changes in thresholds. 17 is a diagram showing an example of generated data D43 about IF errors in the first embodiment. The generated data D43 about IF errors includes charts and graphs showing one-month trends and three-month trends. 18 is a diagram showing an example of the generated data D43 about discards in the first embodiment. The generated data D43 about discards includes charts and graphs showing one-month trends and three-month trends. The generated data D43, which is the analysis result output from the error data trend analysis unit 134, is the analysis result of trend information obtained by inputting the processed data D33, which is data obtained by processing the performance data D11 for a specified period input from the input unit 122, into a trained model M20 for error data trend analysis that has been trained using a learning dataset that includes analysis results showing past trends in network configuration, processed data, and performance, and may be a chart or graph showing trends (e.g., increase / decrease trends, threshold exceedance status, peak values, etc.) related to the error data for a period including the specified period (e.g., if the specified period is the current month, three months including the current month). For example, the trends in Figures 17 and 18 are not simply a selection from increase, decrease, or no change, but are the analysis results of trend information obtained by inputting the threshold exceedance and peak value into the trained model M20 for error data trend analysis, and the threshold exceedance and peak value can be judged in total from past learning and be considered to be an increase only if an increasing trend of a certain level or more is observed, a decrease only if a decreasing trend of a certain level or more is observed, and no change in other cases, and values ​​other than the trends in Figures 17 and 18 can be included in the processed data D31. The generated data D43, which is the analysis result output from the error data trend analysis unit 136, is the analysis result of trend information obtained by inputting processed data D32, which is data obtained by processing performance data D11 for a predetermined period input from the input unit 122, into a trained model M22 for error data trend analysis that has been trained using a learning dataset including network configuration, processed data, and analysis results showing future trends, and may be a chart graph showing time series changes that are future trends related to error data, or a chart graph showing time series changes that are future and past trends.

[0027] The report generation unit 138 functions as a generation unit that inputs the analysis results acquired by the analysis unit into a second machine learning model trained using learning data including the analysis results and information indicating a response policy, and generates advice information based on response policy information indicating the response policy output from the second machine learning model. The second machine learning model is a trained model M14 for report generation (second machine learning model). The trained model M14 for report generation is trained to output response policy information when trend information output from a trained model for trend analysis (third machine learning model) and transition information output from a trained model for transition analysis (fourth machine learning model) are input. The response policy information is information indicating a response policy for the network device 110. The report generation unit 138 outputs an analysis result report D51 including the advice information. The analysis result report D51 includes a summary document including the response policy, and charts and graphs.

[0028] The data storage unit 140 accumulates the performance data D11, the processed data D31, D32, and D33, the generated data D41, D42, and D43, and the analysis result report D51. The trained model storage unit 150 records parameter data for constructing the trained model described above. The trained model storage unit 150 updates the trained model as the trained model is trained.

[0029] FIG. 19 is a diagram showing an example of a summary document according to the first embodiment. The summary document is output as text data by the trained model for report generation M14, for example, by training a summary document created in the past by the trained model for report generation M14, and is included in the analysis result report D51. The analysis result report D51 may also include the various generated data shown in Figures 9 to 18, and may also include a configuration diagram showing the devices in the target network device 110, as shown in Figure 20. 20 is a diagram showing an example of a configuration diagram of network device 110 according to the embodiment. Part designation data D12 input from input unit 122 is data that designates a part of the configuration diagram of network device 110 shown in FIG.

[0030] 21 is a flowchart showing an example of a processing procedure of the network analysis system 120 in the first embodiment. This processing procedure may be executed every predetermined period (for example, every month). First, the input unit 122 inputs performance data D11 and part designation data D12 (step S100), and stores the input performance data D11 and part designation data D12 in the data storage unit 140 (step S102). The processing unit 124 aggregates the input performance data D11 to extract the maximum and average values ​​of the performance data D11 (step S104). The processed data D31, D32, and D33 may include the values ​​aggregated in step S104.

[0031] The processing unit 124 refers to the performance data D11 and the part designation data D12 and determines whether or not there is a new network device 110 (step S106). If there is a new network device 110 (step S106: NO), a base for the new network device 110 is created (step S108). The base for the new network device 110 is information to be compared in subsequent processing, and is information created based on the performance data D11 and the part designation data D12. The base may include the addition of a necessary table or other information associated with the new network device 110 (for example, information indicating a relationship with the configuration diagram of the network device 110 shown in FIG. 20).

[0032] If there is no new network device 110 (step S106: YES), the processing unit 124 refers to the past performance data D11 and part designation data D12 stored in the data storage unit 140 and compares the currently input performance data D11 and part designation data D12 with the performance data D11 and part designation data D12 from one month ago (step S110). The processing unit 124 calculates the difference (increase or decrease) between the performance data D11 and part designation data D12 of this month and the performance data D11 and part designation data D12 of last month. The processed data D31, D32, and D33 may include the values ​​tallied in steps S110 and S112.

[0033] Next, the processing unit 124 determines whether the calculated difference exceeds the threshold D21 (step S114). If the calculated difference does not exceed the threshold D21 (step S114: NO), it determines whether a region is specified by the region specification data D12 (step S116). If data is not specified by the region specification data D12 (step S116: NO), the processing unit 124 saves the number of times the threshold is exceeded as 0 (step S118) and saves the aggregated data in the data storage unit 140 (step S120).

[0034] If the calculated difference exceeds the threshold D21 (step S114: YES), the number of times the threshold is exceeded is tallied (step S122), and the tallied data is stored in the data storage unit 140 (step S126). At the same time, the configuration information of each component is read (step S124). The configuration information indicates, for example, any one of network configurations, such as a single configuration, a redundant configuration, or a one-sided configuration, that indicates the relationship between a specific component and other components. Next, the processing unit 124 extracts devices related to the specific network device 110, such as redundant devices and connected devices (step S130). A redundant device is, for example, a device that is in a redundant configuration with the specific network device 110. The processed data D31, D32, and D33 may include values ​​tallied in steps S114 to S122 and information collected in steps S124 and S130.

[0035] Next, traffic data trend analysis unit 126, traffic data transition analysis unit 128, CPU memory data trend analysis unit 130, CPU memory data transition analysis unit 132, error data trend analysis unit 134, and error data transition analysis unit 136 use processed data D31, D32, and D33 to create generated data D41, D42, and D43 including graph data and table data (step S132). The created generated data D41, D42, and D43 are stored in data storage unit 140 (step S134).

[0036] Next, the report generation unit 138 generates a response policy using the generated data D41, D42, and D43, and determines the priority of the multiple response policies (step S136). At this time, the report generation unit 138 determines the number of times the threshold value is exceeded corresponding to the response policy, and the higher the number of times the threshold value is exceeded, the higher the priority of the response policy.

[0037] Next, the report generation unit 138 determines the priority of the handling policies reflecting the customer needs (step S138). For example, the report generation unit 138 may increase the priority of the handling policies for the specific parts specified by the part specification data D12. For example, the report generation unit 138 may decrease the priority of the handling policies for parts that are scheduled to be discontinued.

[0038] Next, the report generating unit 138 determines a course of action based on the determination results in steps S136 and S138 (step S140).

[0039] Next, the report generation unit 138 inputs the generated data D41, D42, and D43 and the determined response policy into the trained model M14 for report generation, and creates a summary document of the response policy by outputting text data from the trained model M14 for report generation (step S142).The report generation unit 138 creates a report including the summary document and the data and charts included in the generated data D41, D42, and D43, and outputs the report to, for example, a terminal device of the administrator of the network device 110 (step S144).

[0040] As described above, the network analysis system 120 of the first embodiment generates processed data D31, D32, and D33 by processing the performance data D11 based on the performance data D11 of the target network device 110 and the threshold D21, inputs the processed data to a first machine learning model trained with a training dataset including the network configuration, the processed data, and analysis results indicating past trends and future changes in performance, obtains the analysis results output from the first machine learning model, inputs the analysis results to a second machine learning model trained with training data including the analysis results and information indicating a response policy, and generates advice information based on the response policy information indicating the response policy output from the second machine learning model. This allows the network analysis system 120 to analyze a specific communication network and create detailed advice. Furthermore, according to the network analysis system 120 of the first embodiment, the aggregated performance data D11 can be analyzed based on the network trends of the network devices 110 and 210 to grasp trends and predict trends, and a response policy can be generated from the trends, trends, and customer needs, and advice information can be automatically output. Furthermore, the network analysis system 120 of the first embodiment can grasp changes in the network by analyzing the trends and transitions of the aggregated performance data D11, taking into account trends in networks including LANs and WANs, and can automatically output proposals for countermeasures after identifying the reasons and causes of the changes from predicting trends, transitions, and needs.The network analysis system 120 can also combine multiple AI models to perform detailed analyses and output advice information.

[0041] The second embodiment will be described below. In the description of the second embodiment, the same parts as those in the first embodiment are designated by the same reference numerals. FIG. 22 is a block diagram illustrating an example of a network analysis system 120 according to the second embodiment. The network analysis system 120 of the second embodiment differs from the first embodiment in that a learning system 400 is provided in the corporate LAN data center 100. The following description will focus on the differences.

[0042] FIG. 23 is a block diagram showing an example of a learning system 400 according to the second embodiment. The learning system 400 includes, for example, a network performance data learning device 410, a data storage unit 430, and a trained model storage unit 440. The data storage unit 430 and the trained model storage unit 440 may be shared with the data storage unit 140 and the trained model storage unit 150 described above. The data storage unit 430 stores performance data D11, processed data D31, D32, and D33, generated data D41, D42, and D43, and an analysis result report D51. The learned model storage unit 440 stores a trained model M30 for report generation (second machine learning model) (generative AI), a trained model M32 for traffic data trend analysis, a trained model M34 for traffic data trend analysis, a trained model M36 for CPU memory data trend analysis, a trained model M38 for CPU memory data trend analysis, a trained model M40 for error data trend analysis, and a trained model M42 for error data trend analysis.

[0043] The network performance data learning device 410 includes, for example, an input unit 412, a processing unit 414, a traffic data trend analysis model generation unit 416, a traffic data trend analysis model generation unit 418, a CPU memory data trend analysis model generation unit 420, a CPU memory data trend analysis model generation unit 422, an error data trend analysis model generation unit 424, and an error data trend analysis model generation unit 426. The input unit 412, the processing unit 414, the traffic data trend analysis model generation unit 416, the traffic data trend analysis model generation unit 418, the CPU memory data trend analysis model generation unit 420, the CPU memory data trend analysis model generation unit 422, the error data trend analysis model generation unit 424, the error data trend analysis model generation unit 426, and a report generation model generation unit (not shown) are functional units realized by a processor such as a CPU executing a network analysis program.

[0044] The input unit 412 inputs the performance data D11, the processed data D31, D32, and D33, and the generated data D41, D42, and D43 from the data storage unit 430. The processing unit 414 processes the performance data D11, the processed data D31, D32, and D33, and the generated data D41, D42, and D43 to create learning sets D81, D82, and D83. The learning set D81 is output to the traffic data trend analysis model generation unit 416 and the traffic data trend analysis model generation unit 418. The learning set D82 is output to the CPU memory data trend analysis model generation unit 420 and the CPU memory data trend analysis model generation unit 422. The learning set D83 is output to the error data trend analysis model generation unit 424 and the error data trend analysis model generation unit 426.

[0045] Network configuration data D61, customer project trend and needs data D62, and corresponding pattern data and threshold data D71 may be input to the traffic data trend analysis model generation unit 416, the traffic data trend analysis model generation unit 418, the CPU memory data trend analysis model generation unit 420, the CPU memory data trend analysis model generation unit 422, the error data trend analysis model generation unit 424, and the error data trend analysis model generation unit 426. The data and learning set input to each generation unit are learning data. The network performance data learning device 410 may also include a report generation model generation unit (not shown). The report generation model generation unit may be a generation AI. Network configuration data D61, customer project trend and needs data D62, and response pattern data and threshold data D71 are input to the report generation model generation unit.

[0046] 24 is a diagram showing an example of network configuration data D61 in the second embodiment. The network configuration data D61 is data in which, for example, a host name, a category indicating the host's affiliation, a configuration pattern, a system and state, and a configuration diagram number are associated with each other. 25 is a diagram showing an example of the customer case trend and needs data D62 in the second embodiment. The customer case trend and needs data D62 is data in which, for example, a target scope, needs for determining the priority of a response policy, and trends are associated with each other. 26 is a diagram showing an example of correspondence pattern and threshold data D71 in the second embodiment. The correspondence pattern and threshold data D71 is data that associates, for example, a correspondence pattern number, a data type, whether or not a threshold is exceeded, a medium- to long-term trend, a configuration pattern, and a response policy. The response policy is information that represents a response that takes into account the trend, configuration, and priority, and, to be explained in more detail with reference to FIG. 26, is data for learning a response policy according to the data type, whether or not a threshold is exceeded, a medium- to long-term trend, and a configuration pattern.

[0047] FIG. 27 is a diagram showing an example of a learning set D81 in the second embodiment. The learning set D81 is historical traffic data, including, for example, host names, interfaces, IN or OUT data for hosts, trends from the month before last, the number of times the threshold was exceeded from the month before last, peak traffic from the month before last, trends from last month, the number of times the threshold was exceeded from the month before last, and peak traffic from the month before last. FIG. 27 is data for learning trends based on threshold exceedances and peak values. To explain this in more detail with reference to FIG. 27, this data is data for learning trends from the month before last based on the number of times the threshold was exceeded from the month before last and peak traffic from the month before last, and trends from last month based on the number of times the threshold was exceeded from the previous month and peak traffic from the previous month. The trends may be separately input, may be past analysis results, or may be modified past analysis results.

[0048] The traffic data trend analysis model generation unit 416 acquires the training set D81, network configuration data D61, customer project trend / needs data D62, and corresponding pattern and threshold data D71 as training data. The traffic data trend analysis model generation unit 416 trains the trained model M32 for traffic data trend analysis so as to output traffic data trend information when performance data D10, part specification data D12, and data obtained by processing the threshold D21 are input. Training the trained model M32 for traffic data trend analysis includes changing parameters for defining the processing of the trained model M32 for traffic data trend analysis.

[0049] The traffic data trend analysis model generation unit 418 acquires the training set D81, network configuration data D61, customer project trend / needs data D62, and corresponding pattern and threshold data D71 as training data. The traffic data trend analysis model generation unit 418 trains the trained model M34 for traffic data trend analysis so as to output traffic data trend information when data obtained by processing the performance data D10, the part specification data D12, and the threshold D21 is input. Training the trained model M34 for traffic data trend analysis includes changing parameters for defining the processing of the trained model M34 for traffic data trend analysis.

[0050] FIG. 28 is a diagram illustrating an example of a training set D82 in the second embodiment. The training set D82 contains historical data on CPU usage and memory usage, including, for example, host names, trends from the month before last, the number of times the threshold was exceeded from the month before last, the peak usage rate from the month before last, the trend from last month, the number of times the threshold was exceeded from the month before last, and the peak usage rate from the month before last. FIG. 28 illustrates data for learning trends based on threshold exceedances and peak values. To explain this in more detail with reference to FIG. 28, the data includes the following data: the CPU usage trend from the month before last based on the number of times the CPU usage exceeded the threshold and the peak CPU usage rate from the month before last; the CPU usage trend from the month before last based on the number of times the CPU usage exceeded the threshold and the peak CPU usage rate from the month before last; the memory usage trend from the month before last based on the number of times the memory usage exceeded the threshold and the peak memory usage rate from the month before last; and the memory usage trend from the month before last based on the number of times the memory usage exceeded the threshold and the peak memory usage rate from the month before last. The trends may be separately input, may be past analysis results, or may be modified versions of past analysis results.

[0051] The CPU memory data trend analysis model generation unit 420 acquires the learning set D82, network configuration data D61, customer project trend / needs data D62, and corresponding pattern and threshold data D71 as learning data. The CPU memory data trend analysis model generation unit 420 trains the trained model M36 for CPU memory data trend analysis so as to output trend information for CPU memory data when data obtained by processing performance data D10, part specification data D12, and threshold D21 is input. Training the trained model M36 for CPU memory data trend analysis includes changing parameters for defining the processing of the trained model M36 for CPU memory data trend analysis.

[0052] The CPU memory data trend analysis model generation unit 422 acquires the learning set D82, network configuration data D61, customer project trend and needs data D62, and corresponding pattern and threshold data D71 as learning data. The CPU memory data trend analysis model generation unit 422 trains the trained model M38 for CPU memory data trend analysis so as to output trend information on CPU memory data when data obtained by processing performance data D10, part specification data D12, and threshold D21 is input. Training the trained model M38 for CPU memory data trend analysis includes changing parameters for defining the processing of the trained model M38 for CPU memory data trend analysis.

[0053] FIG. 29 is a diagram illustrating an example of a training set D83 in the second embodiment. The training set D83 contains historical data on the number of errors and the number of discards, including, for example, host names, interfaces, trends from the month before last, the number of times the threshold was exceeded from the month before last, the number of cases at the peak from the month before last, trends from last month, the number of times the threshold was exceeded from the month before last, and the number of cases from last month. FIG. 29 illustrates data for learning trends based on threshold exceedance and peak values. To explain this in more detail with reference to FIG. 29, the data includes the trend in the number of errors from the month before last, based on the number of times the threshold was exceeded from the month before last and the peak number of errors from the month before last; the trend in the number of errors from the month before last, based on the number of times the threshold was exceeded from the month before last and the peak number of errors from the month before last; the trend in the number of discards from the month before last, based on the number of times the threshold was exceeded from the month before last and the peak number of discards from the month before last; and the trend in the number of discards from the month before last, based on the number of times the threshold was exceeded from the month before last and the peak number of discards from the month before last. The trends may be separately input, based on past analysis results, or based on modified past analysis results.

[0054] The error data trend analysis model generation unit 424 acquires the training set D83, network configuration data D61, customer project trend / needs data D62, and corresponding pattern and threshold data D71 as training data. The error data trend analysis model generation unit 424 trains the trained model M40 for error data trend analysis so as to output error data trend information when data obtained by processing performance data D10, part specification data D12, and threshold D21 is input. Training the trained model M40 for error data trend analysis includes changing parameters for defining the processing of the trained model M40 for error data trend analysis.

[0055] The error data trend analysis model generation unit 426 acquires the training set D83, network configuration data D61, customer project trend / needs data D62, and corresponding pattern and threshold data D71 as training data. The error data trend analysis model generation unit 426 trains the trained model M42 for error data trend analysis so as to output error data trend information when performance data D10, part specification data D12, and data obtained by processing the threshold D21 are input. Training the trained model M42 for error data trend analysis includes changing parameters for defining the processing of the trained model M42 for error data trend analysis. The report generation model generation unit may be a generation AI, and acquires network configuration data D61, customer project trend and needs data D62, and corresponding pattern data and threshold data D71 as learning data. The report generation model generation unit trains a trained model M30 for report generation so as to output an analysis result report when the generated data D41, D42, and D43 are input. Training the trained model M30 for report generation includes changing parameters for defining the processing of the trained model M30 for report generation.

[0056] Figure 30 is a flowchart showing an example of a processing procedure of the network performance data learning device 410 in the second embodiment. This processing procedure may be used, for example, after the network analysis system 120 (network performance data analysis device 120A) in the first embodiment performs the processing shown in Figure 21 for each predetermined period (e.g., the current month), by copying the data generated therein and stored in the data storage unit 140 to the data storage unit 430 in Figure 23 or by sharing it. The trained model in the trained model storage unit 440 created by this processing procedure may be used in the processing shown in Figure 21 of the first embodiment by copying it to the trained model storage unit 150 in the first embodiment or by sharing it. First, the input unit 412 inputs the performance data D11, the processed data D31, D32, D33, and the generated data D41, D42, and D43 from the data storage unit 430 (step S200), and the processing unit 414 processes the performance data D11, the processed data D31, D32, D33, and the generated data D41, D42, and D43 to create learning sets D81, D82, and D83 (step S202). The processing may include, for example, calculating the trend of the data from the month before last and the previous month, the number of times the threshold was exceeded, and the peak value, and the data may also be corrected from the input unit 412.

[0057] The processing unit 414 refers to the performance data D11 and determines whether or not a new network device 110 is present (step S204). If a new network device 110 is present (step S204: YES), a base for the new network device 110 is created (step S206). The base for the new network device 110 is information to be compared in subsequent processing, is information created based on the performance data D11, and may include the addition of a necessary table or other information associated with the new network device 110 (for example, information indicating a relationship with the configuration diagram of the network device 110 shown in FIG. 20).

[0058] The processing unit 414 determines whether there is any update information in the network configuration data D61, the customer project trend / needs data D62, and the corresponding pattern and threshold data D71 (step S208). If there is any update information (step S208: YES), the processing unit 414 extracts the updated portion of the data (step S210), performs processing using the updated portion of the data, and updates the trained model using the processed training set and the training set processed in step S202 (step S212). If there is no update information (step S208: NO), the processing unit 414 updates the trained model using the training set processed in step S202 (step S214).

[0059] Next, the network performance data learning device 410 reflects the updated trained model in the trained model storage unit 440 (step S216). This allows the network analysis system 120 to perform processing using the updated trained model.

[0060] As described above, according to the second embodiment, learning data including past processed data, network configuration data, and thresholds generated by the processing unit 414 can be reflected in the first machine learning models (M32, M34, M36, M38, M32, M40, M42), and parameters of the first machine learning model can be learned so that the analysis results of the processed data (progress information and transition information) are output from the first machine learning model; and network configuration data D61, customer project trend / needs data D62, and response pattern data and threshold data D71 can be reflected in the second machine learning model (M30), and parameters of the second machine learning model can be learned so that response policy information is output from the second machine learning model.

[0061] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0062] 1 Network system, 100 Corporate LAN data center, 110, 210 Network equipment, 110 Network equipment, 120 Network analysis system, 120A Network performance data analysis device, 122 Input unit, 124 Processing unit, 126 Traffic data trend analysis unit, 128 Traffic data trend analysis unit, 130 CPU memory data trend analysis unit, 132 CPU memory data trend analysis unit, 134 Error data trend analysis unit, 136 Error data trend analysis unit, 138 Report generation unit, 140 Data storage unit, 150 Learned model storage unit, 200 Corporate LAN system, 210 Network equipment, 302 Input unit, 304 Processing unit, 306 Traffic data trend analysis model generation unit, 308 Traffic data trend analysis model generation unit, 310 CPU memory data trend analysis model generation unit, 312 CPU memory data trend analysis model generation unit, 314 Error data trend analysis model generation unit, 316 Error data trend analysis model generation unit, 330 Data storage unit, 400 learning system, 410 network performance data learning device, 412 input unit, 414 processing unit, 416 traffic data trend analysis model generation unit, 418 traffic data trend analysis model generation unit, 420 CPU memory data trend analysis model generation unit, 422 CPU memory data trend analysis model generation unit, 424 error data trend analysis model generation unit, 426 error data trend analysis model generation unit, 430 data storage unit, 440 trained model storage unit, D11 performance data, D12 part designation data, D21 threshold value, D31, D32, D33 processed data, D41, D42, D43 generated data, D51 analysis result report, D61 network configuration data, D62 customer project trend and needs data, D71 threshold value data, D81, D82, D83 training set, M10 trained model for traffic data trend analysis, M12 Trained model for traffic data trend analysis, M14 Trained model for report generation, M16 Trained model for CPU memory data trend analysis, M18 Trained model for CPU memory data trend analysis, M20 Trained model for error data trend analysis, M22 Trained model for error data trend analysis, M30 Trained model for report generation, M32Trained model for traffic data trend analysis, M34 Trained model for traffic data trend analysis, M36 Trained model for CPU memory data trend analysis, M38 Trained model for CPU memory data trend analysis, M40 Trained model for error data trend analysis, M42 Trained model for error data trend analysis

Claims

1. an input unit for inputting performance data of a target network device; a processing unit that processes the performance data based on the performance data and a threshold value to generate processed data; an analysis unit that inputs the network configuration data and the processed data into a first machine learning model that has been trained using a learning dataset including the network configuration data of the network device, processed data, and analysis results that indicate past trends and future trends, and acquires the analysis results output from the first machine learning model when the network configuration data and the processed data are input; a generation unit that inputs the analysis results acquired by the analysis unit into a second machine learning model that has been trained using learning data including the analysis results and information indicating a countermeasure policy, and generates advice information based on countermeasure policy information that indicates the countermeasure policy output from the second machine learning model; A network analysis system comprising:

2. the input unit inputs part designation data that designates a target network device among a plurality of network devices; the processing unit processes the performance data based on the performance data, the part designation data, and a threshold value to generate processed data. The network analysis system of claim 1 .

3. the first machine learning model includes a third machine learning model and a fourth machine learning model; the third machine learning model is trained to output trend information indicating a past trend when the network configuration data and the processed data are input; the fourth machine learning model is trained to output transition information indicating a future transition when the network configuration data and the processed data are input; 2. The network analysis system of claim 1, wherein the second machine learning model is trained to output the response policy information when the trend information output from the third machine learning model and the transition information output from the fourth machine learning model are input.

4. the second machine learning model is trained to output response policy information including a plurality of response policies; 4. The network analysis system according to claim 3, wherein the generator determines priorities of the plurality of handling policies based on the handling policy information output from the second machine learning model, a difference between the performance data and the threshold, and generates the advice information including a handling policy with a high priority.

5. a learning device configured to learn parameters of the first machine learning model so that an analysis result is output from the first machine learning model when the network configuration data and the processed data are input to the first machine learning model; The network analysis system of claim 1 .

6. The computer inputting performance data of the target network device; generating processed data by processing the performance data based on the performance data and a threshold value; a step of inputting the network configuration data and the processed data into a first machine learning model that has been trained using a learning dataset including the network configuration data of the network device, processed data, and analysis results showing past trends and future trends, and acquiring the analysis results output from the first machine learning model, the first machine learning model being trained to output the analysis results when the network configuration data and the processed data are input; inputting the acquired analysis results into a second machine learning model trained using learning data including the analysis results and information indicating a countermeasure policy, and generating advice information based on countermeasure policy information indicating the countermeasure policy output from the second machine learning model; A network analysis method, including:

7. On the computer, inputting performance data of the target network device; generating processed data by processing the performance data based on the performance data and a threshold value; a step of inputting the network configuration data and the processed data into a first machine learning model that has been trained using a learning dataset including the network configuration data of the network device, processed data, and analysis results showing past trends and future trends, and acquiring the analysis results output from the first machine learning model, the first machine learning model being trained to output the analysis results when the network configuration data and the processed data are input; inputting the acquired analysis results into a second machine learning model trained using learning data including the analysis results and information indicating a countermeasure policy, and generating advice information based on countermeasure policy information indicating the countermeasure policy output from the second machine learning model; A network analysis program that runs

Images