Network monitoring system, network monitoring method, and network monitoring program

Abstract

To perform detailed network monitoring that is closer to a user's experience in an audio conference system including an audio conference server that relays audio conferences.SOLUTION: A network monitoring system 100 monitors the quality of an audio conference system that includes an audio conference server. A server status measurement unit 110 measures a server status 511 of the audio conference server. A communication quality measurement unit 120 measures communication quality 512 between each of a plurality of terminals and the audio conference server. An audio quality measurement unit 130 measures the quality of audio at each of the plurality of terminals as audio quality 513. A quality model creation unit 140 creates a quality model 52 of the audio conference system using the server status 511, the communication quality 512, and the audio quality 513. A quality estimation unit 150 uses the quality model 52 to estimate the quality of an estimation target system, which is the audio conference system whose quality is to be estimated.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a network monitoring system, a network monitoring method, and a network monitoring program, and more particularly to a network monitoring system, a network monitoring method, and a network monitoring program that monitor the quality of experience of network users and take measures to improve it. [Background technology]

[0002] Indicators of network communication quality include packet loss rate, jitter, which indicates fluctuations in the time required for packet communication, and delay time between routes. Network monitoring systems are in operation to measure these data. In recent years, communication requirements involving voice and video transmission, such as web conferencing, have been increasing. Indicators that indicate the quality of such communication requirements include the "R-value" (short for "Rating factor" or "total voice transmission quality") defined by the ITU-T. This is said to be more likely to represent the user experience than indicators such as packet loss mentioned above.

[0003] There are products for measuring the R value that are placed near each terminal that is performing voice communication, acquire the communication content, and calculate the R value. With such products, if there are many terminals participating in voice communication and there are many communication opportunities and continuous measurement is required, many products are required, which makes them more expensive than measurement systems for packet loss, etc.

[0004] As a technique for measuring voice quality inexpensively, for example, there is a technique disclosed in Patent Document 1. In the technology of Patent Document 1, a voice quality measuring device is placed at both ends of the path along which voice communication is carried out. Then, a network simulator is used to add packet loss or delay during communication while the voice communication is carried out. A basic data acquisition device acquires the results and creates a table of R values ​​for each combination of path, packet loss, and delay. This table is called basic data. The above configuration is applied to each communication path used, and basic data is acquired for each path. Thereafter, during normal voice communication, it is possible to estimate voice quality close to what the user experiences by acquiring packet loss, delay, etc. using a general network monitoring system that measures packet loss, etc., and comparing the data with the basic data, without using a voice quality measuring device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 4536415 Summary of the Invention [Problem to be solved by the invention]

[0006] Meanwhile, the form of audio communication used in recent web conferences is a service system that relays audio conferences. This system also generally includes an audio conference server that acts as a relay server for audio conferences, and each terminal exchanges audio data via this audio conference server. While such services are often built on the cloud, they can also be technically realized in environments other than the cloud. In this service that relays audio conferences, in addition to the state of the communication path, conditions such as the load on the audio conference server, the load on the terminal, or the state of the device connected to the terminal also affect the audio quality.

[0007] Therefore, there is a problem that it is difficult to approximate the voice quality experienced by the user when estimating voice quality using only data such as packet loss or delay on the communication path as in Patent Document 1.

[0008] The present disclosure aims to realize detailed network monitoring at low cost that is closer to the user's experience in an audio conference system that includes an audio conference server that relays audio conferences. [Means for solving the problem]

[0009] The network monitoring system according to the present disclosure comprises: A network monitoring system for monitoring the quality of an audio conference system including an audio conference server that relays audio conferences held by multiple terminals over a network, comprising: a server status measurement unit that measures the status of the audio conference server as a server status; a communication quality measurement unit that measures communication quality between each of the plurality of terminals and the audio conference server; a voice quality measurement unit that measures the quality of voice at each of the plurality of terminals as voice quality; a quality model creation unit that creates a model of the quality of the audio conference system as a quality model using the server status, the communication quality, and the audio quality; a quality estimation unit that estimates quality in an estimation target system, which is an audio conference system whose quality is to be estimated, using the quality model; Equipped with. [Effects of the Invention]

[0010] In the network monitoring system according to the present disclosure, a quality model of the audio conference system is created using the server status, communication quality, and audio quality of the audio conference server. Then, the quality model is used to estimate the quality of the target system, which is the audio conference system whose quality is to be estimated. Therefore, the network monitoring system according to the present disclosure has the effect of realizing detailed network monitoring that is closer to the user's experience at low cost for an audio conference system equipped with an audio conference server. [Brief explanation of the drawings]

[0011] [Figure 1]1 shows an example of the overall configuration of an audio conference system 800 according to the first embodiment. [Figure 2] 1 is a diagram illustrating an example of a configuration of a network monitoring system 100 according to a first embodiment. [Figure 3] FIG. 4 is a flowchart showing a model creation process in the network monitoring process according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of the configuration of a measurement result database 51 according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of a function definition of a quality model 52 according to the first embodiment. [Figure 6] FIG. 4 is a flowchart showing a quality estimation process in the network monitoring process according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing an example of visualization of an estimation result 53 according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a configuration of a network monitoring system 100 according to a modified example of the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a configuration of a network monitoring system 100 according to a second embodiment. [Figure 10] FIG. 10 is a diagram showing an example of predicted and measured values ​​of the R value according to the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a configuration of a network monitoring system 100 according to a third embodiment. [Figure 12] FIG. 11 is a diagram showing an example of the configuration of a cause and action correspondence table 54 according to the third embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a configuration of a network monitoring system 100 according to a fourth embodiment. [Figure 14] FIG. 13 is a schematic diagram showing an example of a report explanation process according to the fourth embodiment. [Figure 15] FIG. 13 is a schematic diagram illustrating an example of a pre-adjustment process according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present embodiment will be described below with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of the embodiment, the description of identical or corresponding parts will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, the sized relationships between components in the following drawings may differ from the actual relationships. Furthermore, in the description of the embodiment, directions or positions such as up, down, left, right, front, rear, front and back may be indicated. These notations are used for convenience of explanation and do not limit the placement, direction or orientation of devices, instruments, parts, etc.

[0013] Embodiment 1 ***Configuration Description*** FIG. 1 shows an example of the overall configuration of an audio conference system 800 according to this embodiment. The audio conference system 800 is a system that provides an audio conference service. The audio conference system 800 includes an audio conference server 300 that relays an audio conference held by multiple terminals 200 via a network 400. In Fig. 1, users 10 of the terminals 200 are holding an audio conference. The audio conference server 300 is realized, for example, by a cloud service. The network monitoring system 100 is a system that monitors the quality of the audio conference system 800. A user 10 of the network monitoring system 100 is also called an administrator.

[0014] A voice quality measuring device 201, which is a device for measuring voice quality, may be connected to the terminal 200 that conducts the voice conference. Alternatively, the terminal 200 that conducts the voice conference may be equipped with software for measuring voice quality.

[0015] FIG. 2 is a diagram showing an example of the configuration of a network monitoring system 100 according to this embodiment. The network monitoring system 100 is a computer. The network monitoring system 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0016] The network monitoring system 100 includes a server status measurement unit 110, a communication quality measurement unit 120, a voice quality measurement unit 130, a quality model creation unit 140, a quality estimation unit 150, a disclosure unit 160, and a storage unit 190 as functional elements. The storage unit 190 stores a measurement result database 51, a quality model 52, and an estimation result 53. The measurement result database 51 includes a server state 511, a communication quality 512, and a voice quality 513.

[0017] The functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 are realized by software. The storage unit 190 is provided in the memory 921. The storage unit 190 may be provided in the auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0018] The processor 910 is a device that executes a network monitoring program. The network monitoring program is a program that realizes the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0019] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that stores data. A specific example of the auxiliary storage device 922 is a HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0020] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network. Although one input interface 930 is shown in FIG. 2, multiple input interfaces 930 may also be present.

[0021] The output interface 940 is a port to which a cable of an output device such as a display is connected. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. The output interface 940 is also called a display interface. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display. Although one output interface 940 is shown in FIG. 2, multiple output interfaces 940 may be present.

[0022] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0023] The network monitoring program is executed in the network monitoring system 100. The network monitoring program is loaded into the processor 910 and executed by the processor 910. In addition to the network monitoring program, the memory 921 also stores an OS. OS is an abbreviation for Operating System. The processor 910 executes the network monitoring program while running the OS. The network monitoring program and the OS may be stored in an auxiliary storage device 922. The network monitoring program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that part or all of the network monitoring program may be incorporated into the OS.

[0024] The network monitoring system 100 may include multiple processors that replace the processor 910. These multiple processors share the task of executing the network monitoring program. Each processor is a device that executes the network monitoring program, just like the processor 910. Furthermore, the network monitoring system 100 may not be a single device, but may be a system made up of multiple devices.

[0025] Data, information, signal values ​​and variable values ​​used, processed or output by the network monitoring program are stored in memory 921 , secondary storage device 922 , or registers or cache memory within processor 910 .

[0026] The "unit" in each of the server status measurement unit 110, communication quality measurement unit 120, voice quality measurement unit 130, quality model creation unit 140, quality estimation unit 150, and disclosure unit 160 may be interpreted as a "circuit," "step," "procedure," "process," or "circuitry." The network monitoring program causes a computer to execute a server status measurement process, a communication quality measurement process, a voice quality measurement process, a quality model creation process, a quality estimation process, and a disclosure process. The "processes" in the server status measurement process, the communication quality measurement process, the voice quality measurement process, the quality model creation process, the quality estimation process, and the disclosure process may be interpreted as a "program," "program product," "a computer-readable storage medium storing a program," or "a computer-readable recording medium recording a program." The network monitoring method is a method performed by the network monitoring system 100 executing the network monitoring program. The network monitoring program may be provided by being stored in a computer-readable recording medium, or may be provided as a program product.

[0027] ***Explanation of Operation*** Next, the operation of network monitoring system 100 according to this embodiment will be described. The operating procedure of network monitoring system 100 corresponds to a network monitoring method. Furthermore, a program that realizes the network monitoring process, which is the operation of network monitoring system 100, corresponds to a network monitoring program.

[0028] The network monitoring process includes a model creation process for creating a quality model 52, and a quality estimation disclosure process for estimating and disclosing the quality of the audio conference system 800 using the quality model 52. First, the model creation process will be described.

[0029] FIG. 3 is a flow diagram showing the model creation process in the network monitoring process according to this embodiment.

[0030] <Model creation process> <Server status measurement process: Step S101> The server status measurement unit 110 measures the status of the audio conference server 300 as the server status 511 . Specifically, the server status measurement unit 110 measures the server status 511 including the CPU usage, memory usage, communication volume, number of conferences, and number of participants in the audio conference server 300. The server status measurement unit 110 measures information such as the CPU usage, memory usage, communication volume, number of conferences, number of participants in audio conferences, and connection method as the communication status from the audio conference server 300. The server status measurement unit 110 registers the server status 511 obtained by the measurement in the measurement result database 51.

[0031] <Communication quality measurement process: Step S102> The communication quality measurement unit 120 measures the communication quality 512 between each of the plurality of terminals 200 and the audio conference server 300 . Specifically, the communication quality measurement unit 120 measures information such as packet loss rate, jitter, and delay as the communication quality 512 of the path between the audio conference server 300 and each of the plurality of terminals 200. The communication quality measurement unit 120 registers the communication quality 512 obtained by the measurement in the measurement result database 51.

[0032] <Voice quality measurement process: Step S103> The voice quality measurement unit 130 measures the quality of the voice at each of the plurality of terminals 200 as voice quality 513. The voice quality measurement unit 130 measures the voice quality 513 including an R value that is an index indicating the quality of the voice at each of the plurality of terminals 200. The voice quality measurement unit 130 measures an index such as an R value during communication as voice quality 513 using a voice quality measurement device 201 connected to the terminal 200 performing voice communication or software for measuring voice quality. The voice quality measurement unit 130 registers the voice quality 513 obtained by the measurement in the measurement result database 51.

[0033] The server status measurement unit 110 and communication quality measurement unit 120 use a variety of measurement methods, including API, SNMP, and telemetry. Server status measurement processing and communication quality measurement processing are performed using devices or software that implement each measurement function. API is an abbreviation for Application Programming Interface. SNMP is an abbreviation for Simple Network Management Protocol.

[0034] By the above processing of steps S101 to S103, data on the quality of the audio conference system 800 is accumulated as the measurement result database 51. The purpose here is to create a quality model 52 using the data accumulated in the measurement result database 51 as input. Therefore, collecting a large amount of measurement data is effective in improving the accuracy of the quality model 52.

[0035] FIG. 4 is a diagram showing an example of the configuration of the measurement result database 51 according to this embodiment. The measurement result database 51 stores a server status 511, communication quality 512, and voice quality 513 obtained by measurement. Specifically, the measurement result database 51 stores the server status 511, communication quality 512, and voice quality 513 in association with the time at which they were measured. Additionally, the measurement result database 51 stores time supplements, which are supplementary information about the measurement location and the measurement time. The measurement location includes the IP address of the audio conference server 300 and information specifying the communication path. The time supplement includes information about the day of the week and holidays on which the measurement was taken. The measurement result database 51 according to this embodiment is a list showing at which server or route each measurement result was obtained at a certain time.

[0036] <Quality Model Creation Process: Step S104> The quality model creating unit 140 creates a model of the quality of the audio conference system 800 as a quality model 52 using the server state 511 , communication quality 512 and voice quality 513 . Specifically, the quality model creating unit 140 creates a quality model 52 using data stored in the measurement result database 51 as an input. In step S105, the quality model creating unit 140 stores the created quality model 52 in the storage unit 190.

[0037] FIG. 5 is a diagram showing an example of a function definition of the quality model 52 according to this embodiment. An example of the quality model 52 is the function definition shown in Fig. 5. The function definition shown in Fig. 5 is a function in which various measurement values ​​are input variables and a voice quality value such as an R value is an output variable. The implementation method will utilize machine learning (deep learning) or statistical methods such as the SARIMA method.

[0038] FIG. 6 is a flow diagram showing the quality estimation disclosure process in the network monitoring process according to this embodiment. Here, the audio conference system 800 whose quality is to be estimated is assumed to be an estimation target system 801 .

[0039] <Quality Estimation Disclosure Processing> <Server status measurement process: Step S201> The server status measurement unit 110 measures the server status 511 in the estimation target system 801 . The method of measuring the server state 511 in the estimation target system 801 is the same as step S101 described with reference to FIG.

[0040] <Communication quality measurement process: Step S202> The communication quality measurement unit 120 measures the communication quality 512 in the estimation target system 801 . The method of measuring the communication quality 512 in the estimation target system 801 is the same as step S102 described with reference to FIG.

[0041] <Quality Estimation Process: Step S203> The quality estimation unit 150 uses the quality model 52 stored in the storage unit 190 to estimate the quality of an estimation target system 801, which is the audio conference system 800 whose quality is to be estimated. The quality estimation unit 150 compares the server state 511 and communication quality 512 measured for the estimation target system 801 with the quality model 52 to estimate the voice quality of the estimation target system 801 . The quality estimation unit 150 stores the estimation result 53 obtained by the estimation in the storage unit 190.

[0042] <Disclosure process: Step S204> The disclosure unit 160 visualizes the estimation result 53 obtained by estimating the quality of the estimation target system 801 and discloses it to an output device. The disclosure unit 160 discloses the visualized estimation result 53 to an output device that can be confirmed by the user 10 via the output interface 940 or the communication device 950.

[0043] FIG. 7 is a diagram showing an example of visualization of the estimation result 53 according to this embodiment. In Figure 7, the estimated R value 53 and the normal range taking into account time fluctuations are visualized in a graph. The disclosure unit 160 visualizes the estimation results 53 using a technique such as a graph or a table, outputs the visualization to an output device via the output interface 940 or the communication device 950, and discloses the visualization to the user 10. The disclosure unit 160 may also calculate a predetermined threshold value or a normal range indicating a standard state for the measurement data based on the contents of the measurement result database 51, and disclose the calculated normal range to the user 10 together with the visualized estimation results 53. Furthermore, the disclosure unit 160 may notify the user 10 of the issuance of an alert based on a comparison between the estimation result 53 and the normal range. Specifically, the disclosure unit 160 notifies the user 10 of the issuance of an alert when, based on a comparison between the estimation result 53 and the normal range, a deviation from the normal range is currently occurring or is likely to occur in the future.

[0044] ***Other Configurations*** <Variation 1> The network monitoring system 100 of FIG. 2 has been described as having a model generation function and a quality estimation disclosure function in one device. On the other hand, the network monitoring system may be composed of a plurality of devices, including a model creation device having a model generation function and a quality estimation disclosure device having a quality estimation disclosure function. Although a single model creation device is used, a quality estimation disclosure device can be constructed and operated for each customer network. Alternatively, the model creation device and the quality estimation disclosure device may be integrated and operated as a single unit.

[0045] If the model creation device and the quality estimation disclosure device are separate devices, the model creation device that executes the model creation process needs to be connected to a voice quality measurement device or software that measures voice quality. On the other hand, the quality estimation disclosure device that executes the quality estimation disclosure process does not need to have a voice quality measurement device or software that measures voice quality. This reduces the cost of the quality estimation disclosure process.

[0046] <Variation 2> In this embodiment, the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 are realized by software. As a variation, the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 may be realized by hardware. Specifically, the network monitoring system 100 includes an electronic circuit 909 instead of a processor 910 .

[0047] FIG. 8 is a diagram showing an example of the configuration of a network monitoring system 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0048] The functions of the server status measurement unit 110, communication quality measurement unit 120, voice quality measurement unit 130, quality model creation unit 140, quality estimation unit 150, and disclosure unit 160 may be realized by a single electronic circuit, or may be distributed across multiple electronic circuits.

[0049] As another modification, some of the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 may be realized by electronic circuits, and the remaining functions may be realized by software. Also, some or all of the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 may be realized by firmware.

[0050] Each of the processor and the electronic circuit is also called a processing circuitry. That is, the functions of the server status measurement unit 110, the communication quality measurement unit 120, the voice quality measurement unit 130, the quality model creation unit 140, the quality estimation unit 150, and the disclosure unit 160 are realized by the processing circuitry.

[0051] ***Explanation of the effect of this embodiment*** In an audio conference system having an audio conference server that relays audio conferences, it is difficult to approximate the audio quality experienced by users by estimating audio quality using only data such as packet loss or delay on the communication path. The network monitoring system according to this embodiment uses functions such as a server status measurement function and a communication quality measurement function, which allows estimation of voice quality indicators that are closer to the user's experience. Therefore, the network monitoring system according to this embodiment can perform detailed network monitoring that is closer to the user's experience in an audio conference system equipped with an audio conference server, without the need for a regular use of an expensive voice quality measurement device.

[0052] Embodiment 2 In this embodiment, differences from and additions to the first embodiment will be mainly described. In this embodiment, components having the same functions as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

[0053] ***Configuration Description*** FIG. 9 is a diagram showing an example of the configuration of a network monitoring system 100 according to this embodiment. The network monitoring system 100 according to this embodiment includes a model update unit 171 in addition to the components described in the first embodiment.

[0054] ***Explanation of Operation*** The model update unit 171 updates the quality model 52 . For example, for a certain audio conference system 800, when the difference between the predicted value of measurement data obtained by estimation using the quality model 52 and the actual measured value of the measurement data in the audio conference system 800 becomes larger than a threshold, the model update unit 171 updates the quality model 52. The measurement data is, for example, an R value. The model update unit 171 continuously updates the quality model 52 . The quality estimation unit 150 estimates the quality using the updated quality model 52 .

[0055] FIG. 10 is a diagram showing an example of predicted and measured values ​​of the R value according to the present embodiment. The model update unit 171 determines whether the difference between the predicted value and the actually measured value of the R value for a certain audio conference system 800 exceeds a threshold, and updates the quality model 52 if the difference exceeds the threshold. The determination of the difference between the predicted R value and the actually measured R value may be carried out continuously, or may be carried out periodically or irregularly at a predetermined time, for example. Furthermore, the quality index used to determine the difference between the predicted value and the actual measurement value may be other than the R value. For example, it may be an index of communication quality such as packet loss, delay, or jitter. Furthermore, a combination of multiple quality indexes may be used to determine the difference between the predicted value and the actual measurement value.

[0056] Here, "continuously updating the quality model" can be done, for example, as follows: First, the quality model may be updated periodically, such as monthly or weekly. Alternatively, the R value estimated by the quality model may be updated when the deviation from the actual measurement value becomes large. This will be explained using Figure 10. In Figure 10, if the quality value obtained at a certain time from the quality model 52 is the "predicted value" and the value actually measured by the voice quality measurement unit 130 is the "actual measurement value," the difference between the predicted value and the actual measurement value at that time can be found as the "residual." Statistical methods include a method for evaluating the difference between predictions and actual measurements using an index called the "coefficient of determination." The formula is: Coefficient of determination = 1 - (sum of squares of residuals / sum of squares of actual measurements). The coefficient of determination ranges from 0 to 1, and the closer it is to 1, the smaller the difference between the actual measured value and the predicted value. Therefore, the coefficient of determination is calculated every month, week, etc., and when it reaches a standard value, such as less than 0.5, the quality model is updated. The standard value can be set arbitrarily. This ensures that diagnosis is always performed using a quality model that is up-to-date or that has a good fit with the predicted values.

[0057] ***Explanation of the effect of this embodiment*** In an audio conference system, the correlation between the server status, communication quality, and audio quality can change over time due to changes in usage volume or usage methods. Therefore, if the quality model in the audio conference system is not updated as needed, deviations may occur. According to the network monitoring system of this embodiment, diagnosis can be performed based on an updated quality model, making it possible to track fluctuations over time in the correlation between voice quality and measurements such as communication quality and server quality.

[0058] Embodiment 3 In this embodiment, differences from and additions to the second embodiment will be mainly described. In this embodiment, components having the same functions as those in the second embodiment are given the same reference numerals, and the description thereof will be omitted.

[0059] ***Configuration Description*** FIG. 11 is a diagram showing an example of the configuration of a network monitoring system 100 according to this embodiment. The network monitoring system 100 according to this embodiment includes a cause response unit 172 in addition to the components described in the first and second embodiments. Also, a cause and action correspondence table 54 is stored in a storage unit 190. The cause and solution correspondence table 54 contains a list of causes of quality degradation and solutions corresponding to those causes.

[0060] ***Explanation of Operation*** FIG. 12 is a diagram showing an example of the configuration of the cause and action correspondence table 54 according to this embodiment. The cause response unit 172 analyzes the estimation result 53 obtained by estimating the quality of the estimation target system 801, and identifies the cause of the quality degradation. The cause response unit 172 uses the cause / action correspondence table 54 to select an action corresponding to the identified cause, and executes the action. Specifically, the cause response unit 172 extracts data that are the main causes of fluctuations from various measurement data accumulated in the measurement result database 51 by using statistical analysis such as multiple regression analysis or AI technology such as machine learning. AI stands for artificial intelligence. Based on this result, the cause handling unit 172 selects a process corresponding to the identified cause from the cause and process correspondence table 54 and executes the process.

[0061] As shown in Figure 12, the cause and action correspondence table 54 is composed of a correspondence table of causes of status changes and the actions to be taken when they occur. By selecting and executing an action, the worsened status can be improved. Each action is prepared as a program for changing the settings of the network device or audio conference system to which it is applied.

[0062] ***Effects of this embodiment*** After estimating the voice quality, if quality degradation is occurring or can be predicted, it is necessary to apply or propose measures to improve or prevent the quality degradation. In the network monitoring system according to this embodiment, when a deterioration in quality is detected or predicted, action for improvement can be automatically executed, thereby shortening the time it takes for quality to deteriorate.

[0063] Embodiment 4 In this embodiment, differences from and additions to the third embodiment will be mainly described. In this embodiment, components having the same functions as those in the third embodiment are given the same reference numerals, and the description thereof will be omitted.

[0064] ***Configuration Description*** FIG. 13 is a diagram showing an example of the configuration of a network monitoring system 100 according to this embodiment. The network monitoring system 100 according to this embodiment includes a report commentary unit 173 and a pre-adjustment unit 174 in addition to the components described in the first to third embodiments. Note that the network monitoring system 100 according to this embodiment may include either the report explanation unit 173 or the pre-adjustment unit 174. Alternatively, the network monitoring system 100 according to this embodiment may include both the report explanation unit 173 and the pre-adjustment unit 174.

[0065] The report explanation unit 173 is implemented, for example, using a generation AI technology. The report explanation unit 173 has a "language processing function" and a "data processing function" inside. The pre-adjustment unit 174 is implemented using technology as a generation AI, similar to the report explanation unit 173. Also, similar to the report explanation unit 173, the pre-adjustment unit 174 internally has a "language processing function" and a "data processing function."

[0066] ***Explanation of Operation*** Here, the report explanation process by the report explanation unit 173 and the pre-adjustment process by the pre-adjustment unit 174 will be described.

[0067] The report explanation unit 173 receives a question 30 from the user 10 and interprets the content of the question 30 using a language processing function. The report explanation unit 173 refers to the measurement result database 51 and the cause-action correspondence table 54 based on the content of the question 30, and creates an answer 31 to the question 30. The report explanation unit 173 presents the answer 31 to the user 10.

[0068] The pre-adjustment unit 174 receives a question 30 including attributes of the audio conference from the user 10 and interprets the content of the question 30 using a language processing function. The pre-adjustment unit 174 compares the attributes of the audio conference with the quality model 52 and selects a time slot suitable for the attributes of the audio conference. The pre-adjustment unit 174 creates an answer 31 including the selected time slot and presents it to the user 10.

[0069] <Report Explanation Processing> FIG. 14 is a schematic diagram showing an example of the report explanation process according to the present embodiment. The user 10 gives a question 30 such as "time period, cause, and solution of the quality deterioration" to the report explanation section 173 via the input interface 930 or the communication device 950. The report explanation unit 173 uses the "language processing function" to interpret the content of the question 30. The report explanation unit 173 uses the "data processing function" to refer to the measurement result database 51, the quality model 52, or the cause-action correspondence table 54, and creates an answer 31 to the question 30. The report explanation unit 173 presents the answer 31 to the user 10 via the output interface 940 or the communication device 950.

[0070] Furthermore, the user 10 may ask a question such as "When is the best time to have no quality issues?" to arrange a meeting or the like in advance for a time when there are no issues.

[0071] <Pre-adjustment processing> FIG. 15 is a schematic diagram showing an example of the pre-adjustment process according to this embodiment. The user 10 makes a conference arrangement inquiry 30 to the pre-arrangement unit 174 via the input interface 930 or the communication device 950. Specifically, the user 10 includes in the inquiry 30 attributes of the audio conference, such as conference partners, duration, and video or camera specifications. The pre-adjustment unit 174 uses its "language processing function" to interpret the content of the question 30. Furthermore, the pre-adjustment unit 174 uses its "data processing function" to select a time period in which good quality can be expected from the attributes of the audio conference included in the question 30, by referring to the quality model 52. The report explanation unit 173 presents the answer 31 including the selected time period to the user 10 via the output interface 940 or the communication device 950.

[0072] ***Effects of this embodiment*** The report explanation unit of the network monitoring system according to this embodiment can provide the user with an easy-to-understand explanation of the report contents such as the estimation results, or can suggest actions for improvement in an easy-to-understand manner. Furthermore, the advance adjustment unit of the network monitoring system according to this embodiment can propose conference adjustments that can be expected to produce good audio quality.

[0073] In the above first to fourth embodiments, each part of the network monitoring system has been described as an independent functional block. However, the configuration of the network monitoring system does not have to be as in the above-described embodiments. The functional blocks of the network monitoring system may have any configuration as long as they can realize the functions described in the above-described embodiments. Furthermore, the network monitoring system may be a system composed of multiple devices, rather than a single device. Furthermore, it is possible to combine two or more parts of the first to fourth embodiments. Alternatively, it is possible to implement only one part of these embodiments. In addition, it is possible to implement any combination of these embodiments, either as a whole or in part. That is, in the first to fourth embodiments, the embodiments can be freely combined, or any of the components in each embodiment can be modified, or any of the components in each embodiment can be omitted.

[0074] The above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as needed. For example, the procedures described using flow charts or sequence diagrams may be modified as appropriate.

[0075] Various aspects of the present disclosure are summarized below as appendices.

[0076] (Appendix 1) A network monitoring system for monitoring the quality of an audio conference system including an audio conference server that relays audio conferences held by multiple terminals over a network, comprising: a server status measurement unit that measures the status of the audio conference server as a server status; a communication quality measurement unit that measures communication quality between each of the plurality of terminals and the audio conference server; a voice quality measurement unit that measures the quality of voice at each of the plurality of terminals as voice quality; a quality model creation unit that creates a model of the quality of the audio conference system as a quality model using the server status, the communication quality, and the audio quality; a quality estimation unit that estimates quality in an estimation target system, which is an audio conference system whose quality is to be estimated, using the quality model; A network monitoring system comprising: (Appendix 2) The network monitoring system includes: a measurement result database that stores the server status, the communication quality, and the voice quality in association with supplementary information on measurement locations and measurement times; The quality model creation unit 2. The network monitoring system according to claim 1, wherein the quality model is created using data accumulated in the measurement result database as input. (Appendix 3) The quality estimation unit A network monitoring system according to claim 2, which estimates the quality of the estimation target system by comparing the server state and the voice quality measured for the estimation target system with the quality model. (Appendix 4) The server status measurement unit 4. The network monitoring system according to claim 2 or 3, wherein the server status including the usage of a central processing unit, memory usage, communication volume, number of conferences, and number of participants in the audio conference server is measured. (Appendix 5) The network monitoring system includes: 5. The network monitoring system according to claim 2, further comprising a display unit that visualizes the estimation result obtained by estimating the quality of the audio conference system and outputs the visualization result to an output device. (Appendix 6) The network monitoring system includes: 6. The network monitoring system according to claim 2, further comprising a model update unit that updates the quality model. (Appendix 7) The voice quality measurement unit measuring the voice quality including an R value which is an index indicating the voice quality at each terminal of the plurality of terminals; The model update unit A network monitoring system as described in Appendix 6, which updates the quality model when the difference between the predicted R value obtained by estimation using the quality model and the actual measured R value in the audio conference system becomes larger than a threshold value. (Appendix 8) The network monitoring system includes: 8. The network monitoring system according to claim 2, further comprising a cause response unit that analyzes an estimation result obtained by estimating the quality of the audio conference system, identifies a cause of quality degradation, selects a remedial action corresponding to the identified cause using a cause and remedial action correspondence table in which the cause of quality degradation and the remedial action corresponding to the cause are set, and executes the remedial action. (Appendix 9) The network monitoring system includes: 9. A network monitoring system as described in Appendix 8, comprising a report commentary unit that receives a question from a user, interprets the content of the question using a language processing function, refers to the measurement result database and the cause and action correspondence table based on the content of the question, creates an answer to the question, and presents it to the user. (Appendix 10) The network monitoring system includes: 9. The network monitoring system according to claim 8, further comprising a pre-adjustment unit that receives a question from a user, the question including attributes of the audio conference, interprets the content of the question using a language processing function, compares the attributes of the audio conference with the quality model, selects a time period suitable for the attributes of the audio conference, creates an answer including the time period, and presents the answer to the user. (Appendix 11) A network monitoring method for monitoring the quality of an audio conference system including an audio conference server that relays an audio conference held by a plurality of terminals via a network, comprising: a computer measures the status of the audio conference server as a server status; a computer measuring communication quality between each of the plurality of terminals and the audio conference server; a computer measures the quality of the voice at each of the plurality of terminals as voice quality; a computer creates a quality model of the quality of the audio conference system using the server status, the communication quality, and the audio quality; A network monitoring method in which a computer uses the quality model to estimate quality in an estimation target system, which is an audio conference system whose quality is to be estimated. (Appendix 12) A network monitoring program for monitoring the quality of an audio conference system including an audio conference server that relays audio conferences held by a plurality of terminals over a network, comprising: a server status measurement process for measuring the status of the audio conference server as a server status; a communication quality measurement process for measuring communication quality between each of the plurality of terminals and the audio conference server; a voice quality measurement process for measuring the quality of voice at each of the plurality of terminals as voice quality; a quality model creation process for creating a quality model of the audio conference system using the server status, the communication quality, and the audio quality; a quality estimation process for estimating quality in an estimation target system, which is a voice conference system whose quality is to be estimated, using the quality model; A network monitoring program that causes a computer to run [Explanation of symbols]

[0077] 10 User, 30 Question, 31 Answer, 51 Measurement result database, 511 Server status, 512 Communication quality, 513 Voice quality, 52 Quality model, 53 Estimation result, 54 Cause and treatment correspondence table, 100 Network monitoring system, 110 Server status measurement unit, 120 Communication quality measurement unit, 130 Voice quality measurement unit, 140 Quality model creation unit, 15 Quality estimation unit, 160 Disclosure unit, 171 Model update unit, 172 Cause response unit, 173 Report explanation unit, 174 Pre-adjustment unit, 190 Memory unit, 200 Terminal, 201 Voice quality measurement device, 300 Audio conference server, 400 Network, 800 Audio conference system, 801 Estimation target system, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 Input interface, 940 Output interface, 950 Communication equipment.

Claims

1. A network monitoring system for monitoring the quality of an audio conference system including an audio conference server that relays audio conferences held by multiple terminals over a network, comprising: a server status measurement unit that measures the status of the audio conference server as a server status; a communication quality measurement unit that measures communication quality between each of the plurality of terminals and the audio conference server; a voice quality measurement unit that measures the quality of voice at each of the plurality of terminals as voice quality; a quality model creation unit that creates a model of the quality of the audio conference system as a quality model using the server status, the communication quality, and the audio quality; a quality estimation unit that estimates quality in an estimation target system, which is an audio conference system whose quality is to be estimated, using the quality model; A network monitoring system comprising:

2. The network monitoring system includes: a measurement result database that stores the server status, the communication quality, and the voice quality in association with supplementary information on measurement locations and measurement times; The quality model creation unit 2. The network monitoring system according to claim 1, wherein the quality model is created using data stored in the measurement result database as an input.

3. The quality estimation unit 3. The network monitoring system according to claim 2, wherein the quality of the estimation target system is estimated by comparing the server status and the communication quality measured for the estimation target system with the quality model.

4. The server status measurement unit 4. A network monitoring system according to claim 2, wherein the server status including the usage of a central processing unit, memory usage, communication volume, number of conferences, and number of participants in the audio conference server is measured.

5. The network monitoring system includes:

4. The network monitoring system according to claim 2, further comprising a disclosure unit that visualizes the estimation result obtained by estimating the quality of the audio conference system and outputs the visualized result to an output device.

6. The network monitoring system includes:

4. The network monitoring system according to claim 2, further comprising a model update unit that updates the quality model.

7. The voice quality measurement unit measuring the voice quality including an R value which is an index indicating the quality of the voice at each terminal of the plurality of terminals; The model update unit 7. The network monitoring system according to claim 6, wherein the quality model is updated when a difference between a predicted R value obtained by estimation using the quality model and an actual R value measured in the audio conference system becomes greater than a threshold value.

8. The network monitoring system includes:

4. The network monitoring system according to claim 2, further comprising a cause response unit that analyzes an estimation result obtained by estimating the quality of the audio conference system, identifies a cause of quality degradation, selects a remedial action corresponding to the identified cause using a cause and remedial action correspondence table in which the cause of quality degradation and the remedial action corresponding to the cause are set, and executes the remedial action.

9. The network monitoring system includes:

9. The network monitoring system of claim 8, further comprising a report commentary unit that receives a question from a user, interprets the content of the question using a language processing function, references the measurement result database and the cause and action correspondence table based on the content of the question, creates an answer to the question, and presents the answer to the user.

10. The network monitoring system includes:

9. The network monitoring system according to claim 8, further comprising a pre-adjustment unit that receives a question from a user including attributes of an audio conference, interprets the content of the question using a language processing function, compares the attributes of the audio conference with the quality model, selects a time period suitable for the attributes of the audio conference, creates an answer including the time period, and presents the answer to the user.

11. A network monitoring method for monitoring the quality of an audio conference system including an audio conference server that relays an audio conference held by a plurality of terminals via a network, comprising: a computer measures the status of the audio conference server as a server status; a computer measuring communication quality between each of the plurality of terminals and the audio conference server; a computer measures the quality of the voice at each of the plurality of terminals as voice quality; a computer creates a quality model of the quality of the audio conference system using the server status, the communication quality, and the audio quality; A network monitoring method in which a computer uses the quality model to estimate quality in an estimation target system, which is an audio conference system whose quality is to be estimated.

12. A network monitoring program for monitoring the quality of an audio conference system including an audio conference server that relays audio conferences held by a plurality of terminals over a network, comprising: a server status measurement process for measuring the status of the audio conference server as a server status; a communication quality measurement process for measuring communication quality between each of the plurality of terminals and the audio conference server; a voice quality measurement process for measuring the quality of voice at each of the plurality of terminals as voice quality; a quality model creation process for creating a quality model of the audio conference system using the server status, the communication quality, and the audio quality; a quality estimation process for estimating quality in an estimation target system, which is a voice conference system whose quality is to be estimated, using the quality model; A network monitoring program that causes a computer to run

Images