Diagnostic device, diagnostic method, diagnostic program, and diagnostic system

Abstract

To provide a diagnostic device, a diagnostic method, a diagnostic program, and a diagnostic system that are capable of efficiently diagnosing faults in a communication network.SOLUTION: A diagnostic device includes an identification unit for identifying a predetermined communication device group that accommodate a predetermined communication service from among a plurality of communication devices included in a communication network, a command generation unit for generating a command sequence which a predetermined communication device included in the communication device group is caused to execute, on the basis of the identified predetermined communication device group, a collection unit for causing the predetermined communication device to execute the generated command sequence and collecting a predetermined command response indicating an operating state of the predetermined communication devices, and a discrimination unit for discriminating a fault type in the predetermined communication service on the basis of the predetermined command response. The command generation unit obtains the command sequence by providing the identified predetermined communication device group to a trained model trained by deep reinforcement learning.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a diagnostic device, a diagnostic method, a diagnostic program, and a diagnostic system. [Background technology]

[0002] For example, Patent Document 1 describes a network used in an IP relay network managed by a telecommunications carrier (carrier). A technique for monitoring network devices and their communication paths is disclosed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-184638 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a communication carrier receives a report from a user that communication is not possible, the communication carrier performs a fault diagnosis by issuing various commands to each communication device constituting the communication network and checking the response results.

[0005] However, in recent years, the number and types of communication devices constituting a communication network have become more diverse in accordance with the expansion of the scale of the communication network and the services. Furthermore, since the diagnostic methods differ for each communication device, the fault diagnosis work performed by communication carriers has become very complicated.

[0006] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide a diagnostic device, a diagnostic method, a diagnostic program, and a diagnostic system that can efficiently perform fault diagnosis in a communication network. [Means for solving the problem]

[0007] A diagnostic device according to one aspect of the present invention includes an identification unit that identifies a group of specific communication devices that accommodate a specific communication service from among a plurality of communication devices included in a communication network; a command generation unit that generates a command sequence to be executed by a specific communication device included in the group of communication devices based on the identified group of specific communication devices; a collection unit that causes the specific communication devices to execute the generated command sequence and collects specific command responses indicating the operating status of the specific communication devices; and a discrimination unit that discriminates a fault type in the specific communication service based on the specific command responses, wherein the command generation unit obtains the command sequence by providing the identified group of specific communication devices to a learned model learned by deep reinforcement learning.

[0008] A diagnostic method according to one aspect of the present invention includes the steps of identifying a group of predetermined communication devices that accommodate a predetermined communication service from among a plurality of communication devices included in a communication network; generating a command sequence to be executed by a predetermined communication device included in the predetermined communication device group based on the identified group of predetermined communication devices; causing the predetermined communication devices to execute the generated command sequence and collecting predetermined command responses indicating the operating status of the predetermined communication devices; and determining a fault type in the predetermined communication service based on the predetermined command responses, wherein the step of generating the command sequence includes providing the identified group of predetermined communication devices to a learned model learned by deep reinforcement learning to obtain the command sequence.

[0009] A diagnostic program according to one aspect of the present invention is a diagnostic program to be executed by a computer, and includes the steps of: identifying a group of predetermined communication devices that accommodate a predetermined communication service from among a plurality of communication devices included in a communication network; generating a command sequence to be executed by a predetermined communication device included in the group of predetermined communication devices based on the identified group of predetermined communication devices; having the predetermined communication devices execute the generated command sequence and collect a predetermined command response indicating the operating status of the predetermined communication device; and determining a fault type in the predetermined communication service based on the predetermined command response, wherein the step of generating the command sequence includes providing the identified group of predetermined communication devices to a learned model learned by deep reinforcement learning to obtain the command sequence.

[0010] A diagnostic system according to one aspect of the present invention is a diagnostic system comprising a plurality of communication devices and a diagnostic device included in a communication network, wherein the communication devices comprise a management unit that manages the operating status of the communication devices, and a transmission unit that transmits a command response indicating the operating status of the communication devices to the diagnostic device, and the diagnostic device comprises: an identification unit that identifies a predetermined communication device group accommodating a predetermined communication service from among the plurality of communication devices; a command generation unit that generates a command sequence to be executed by a predetermined communication device included in the predetermined communication device group based on the identified predetermined communication device group; a collection unit that causes the predetermined communication device to execute the generated command sequence and collects predetermined command responses indicating the operating status of the predetermined communication device; and a discrimination unit that discriminates a fault type in the predetermined communication service based on the predetermined command response, and the command generation unit obtains the command sequence by providing the identified predetermined communication device group to a learned model learned by deep reinforcement learning. Effect of the Invention

[0011] According to the present invention, fault diagnosis in a communication network can be carried out efficiently. [Brief description of the drawings]

[0012] [Figure 1]FIG. 1 is a diagram showing an example of a schematic configuration of a diagnostic system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a diagnostic device and a communication device according to an embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional block configuration of a communication device according to an embodiment. [Figure 4] FIG. 4 is a configuration diagram showing an example of a functional block configuration of a diagnostic device according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a NW configuration. [Figure 6] FIG. 6 is a diagram illustrating an example of commands and confirmation rules executed by the communication device. [Figure 7] FIG. 7 is a diagram for explaining the state vector. [Figure 8] FIG. 8 is a diagram showing an example of labels assigned to state vectors. [Figure 9] FIG. 9 is a flowchart illustrating an example of a trained model generation process performed by a diagnosis device in one embodiment. [Figure 10] FIG. 10 is a diagram showing an example of visualizing feature vectors. [Figure 11] FIG. 11 is a diagram illustrating the flow of repetitions in the trained model generation process. [Figure 12] FIG. 12 is a diagram for explaining state space vectors. [Figure 13] FIG. 13 is a diagram for explaining the experience replay process. [Figure 14] FIG. 14 is a flowchart illustrating an example of the command determination process. [Figure 15] FIG. 15 is a flowchart illustrating an example of the emulator execution process. [Figure 16] FIG. 16 is a diagram illustrating an example of a Q value in deep reinforcement learning when generating a trained model. [Figure 17] FIG. 17 is a graph showing an example of a distribution of search scores for command sequences. [Figure 18] FIG. 18 is a flowchart for explaining an example of a failure type determination process performed by the diagnostic device in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An embodiment of the present invention will be described below. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, the drawings are schematic. Therefore, specific dimensions, etc. should be judged in light of the following description. In addition, it goes without saying that the drawings include parts with different dimensional relationships and ratios. Furthermore, the technical scope of the present invention should not be interpreted as being limited to the embodiment.

[0014] <Diagnosis system configuration> First, a schematic configuration of a diagnostic system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a configuration diagram showing an example of a schematic configuration of a diagnostic system 1 in an embodiment.

[0015] As shown in FIG. 1, the diagnostic system 1 includes a diagnostic device 10, a plurality of communication networks N1 to N4, and a user network U.

[0016] Each of the communication networks N1 to N4 is managed by a telecommunications carrier. The communication networks N1 to N4 are configured by a plurality of communication devices 20-1 to 20-17 arranged in facilities managed by the telecommunications carrier. The communication devices 20 are network devices such as a router, a switch, a hub, a firewall, and an ONU (Optical Network Unit).

[0017] In the following description, when the communication devices 20-1 to 20-17 are not distinguished from each other, they are referred to as communication devices 20. The number of communication devices 20 included in the communication networks N1 to N4 is not limited to the example shown in FIG. 1. The number of communication devices 20 constituting the communication networks N1 to N4 is not limited, and may be, for example, one, or may be two or more. In the example shown in FIG. 1, four communication networks N1 to N4 are illustrated as communication networks managed by the telecommunications carrier, but the number of communication networks is not limited to this example. The number of communication networks managed by the telecommunications carrier is not limited, and may be, for example, one, two, three, or five or more. In the following description, when the communication networks N1 to N4 are not distinguished from each other, they are referred to as communication networks N. Although not illustrated, at least one of the communication networks N, for example, the communication network N4, may be connected to the Internet.

[0018] The user network U is a communication network managed by a user of a service provided by a telecommunications carrier. The communication network managed by the user includes, for example, a wireless LAN communication environment in a personal user's home, a network of a corporate user, etc. In the example of FIG. 1, only the user network U is illustrated, but there is no limit to the number of user networks.

[0019] The communication network N1 is sometimes called an access network or an access network because it is connected to the user network U. On the other hand, the communication networks N2 to N4 are sometimes called a core network or a core network.

[0020] The diagnostic device 10 is a device that monitors the operating states of multiple communication devices 20 that constitute communication networks N1 to N4 managed by a telecommunications carrier. More specifically, the diagnostic device 10 monitors the state of each interface of the communication device 20 and the processing load of the communication device 20 itself based on a command response received from each communication device 20. The interface of the communication device 20 means a physical interface that connects a cable and / or a logical interface created within the physical interface. A logical interface is sometimes called a virtual interface. In addition, a command response received from the communication device 20 is obtained by executing a command generated by the diagnostic device 10 in the communication device 20, and is a response result of the communication device 20 to the command.

[0021] Here, an outline of a communication service failure response task in a network operation task that is currently widely performed will be described. First, a user who detects a communication service failure reports the failure to a communication carrier's office, including a service ID assigned to each user. An operator of the communication carrier searches a database or the like using the service ID as a key, and confirms the communication network that accommodates the communication service used by the user, such as an access network or relay network, and the communication device 20 included in the communication network. Next, the operator uses an information processing device or the like to remotely log in to the communication device 20 and execute a command for performing a failure diagnosis. Then, the operator receives a response result to the executed command from each of the multiple communication devices 20, and identifies the suspected failure location.

[0022] Furthermore, the operator uses the failure response system to arrange for workers to respond to failure recovery, to give instructions for recovery work on-site, and to exchange information such as the results of normality checks between the terminals used by the workers.

[0023] Generally, there are a wide variety of commands for performing fault diagnosis, and each command has a confirmation rule for performing fault diagnosis based on the response result. The confirmation rules have different confirmation contents depending on the communication network, and the commands to be executed by each communication network and each communication device 20 are different accordingly. For this reason, in order to perform fault diagnosis such as identifying the fault location and / or fault cause for a certain fault, it is necessary to confirm the response results of the commands for all communication devices 20 that accommodate the user's communication service and make a judgment based on the results. In addition, there are usually not just one but a large number of commands to be executed for each communication device 20, and the response results are complex text messages that are assumed to be judged visually. Therefore, the current fault diagnosis work is mainly manual work that depends on the operator's specialized knowledge, skilled skills, long-term experience, etc.

[0024] (Diagnosis system overview) When the diagnostic device 10 receives a report from a user who uses a service provided by a telecommunications carrier that communication is not possible, for example, via telephone or e-mail, the diagnostic device 10 identifies a plurality of communication devices 20 that accommodate the communication service of the user from among a plurality of communication devices 20 included in the communication networks N1 to N3, using the reported service ID as a key. Specifically, in the example shown in FIG. 1, the route through which the user's communication service passes is represented by a thick line. Thus, the plurality of communication devices 20 that accommodate the user's communication service are communication devices 20-3, 20-4, 20-8, 20-9, and 20-13. Next, the diagnostic device 10 generates a command sequence including a command to be executed by the plurality of communication devices 20 that accommodate the identified communication service of the user to check the operation state of the communication device 20. Then, the diagnostic device 10 remotely logs in to each of the plurality of communication devices 20, and executes a command to check the operation state of the communication device 20, thereby collecting a command response indicating the operation state of each communication device 20 from each communication device 20. Next, the diagnostic device 10 determines the type of failure based on the collected command response results.

[0025] In this embodiment, the "fault event" constitutes a state vector described later, and the "fault type" is assigned to each fault event and is estimated. In other words, the "fault event" means the content of a fault that has "actually occurred," and the "fault type" means the content of a fault that is "estimated."

[0026] <Hardware configuration> Next, the hardware configuration of the diagnostic device and the communication device according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a configuration diagram showing an example of the hardware configuration of the diagnostic device 10 and the communication device 20 in an embodiment.

[0027] As shown in FIG. 2, the diagnostic device 10 and the communication device 20 each include, for example, a processor 31, a memory 32, a storage device 33, a communication device , an input device 35, and an output device .

[0028] The processor 31 is configured to control the operation of each part of the diagnostic device 10 or the communication device 20. The processor 31 is configured to include an integrated circuit such as a central processing unit (CPU), a digital signal processor (DSP), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), or a system-on-a-chip (SoC).

[0029] The memory 32 and the storage device 33 are configured to store programs, data, etc. The memory 32 is configured, for example, of a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), and / or a Random Access Memory (RAM), etc. The storage device 33 is configured, for example, of a storage such as a Hard Disk Drive (HDD), a Solid State Drive (SSD), and / or an embedded Multi Media Card (eMMC).

[0030] The communication device 34 is configured to communicate via a wired and / or wireless network. The communication device 24 includes, for example, a network card, a communication module, etc. In addition, when at least one of the diagnostic device 10 and the communication device 20 includes an antenna (not shown), the communication device 34 may include an amplifier, an RF (Radio Frequency) device that performs processing related to wireless signals, and a BB (BaseBand) device that performs baseband signal processing.

[0031] The RF device performs, for example, D / A (Digital to Analog) conversion, modulation, frequency conversion, power amplification, etc. on the digital baseband signal received from the BB device to generate a radio signal to be transmitted from the antenna. The RF device also performs frequency conversion, demodulation, A / D (Analog to Digital) conversion, etc. on the radio signal received from the antenna to generate a digital baseband signal and transmit it to the BB device. The BB device performs a process of converting the digital baseband signal into an IP packet, and a process of converting the IP packet into a digital baseband signal.

[0032] The input device 35 is configured to allow a user to input information through an operation of the user. The input device 35 includes, for example, a keyboard, a touch panel, a mouse, and / or a microphone.

[0033] The output device 36 is configured to output information. The output device 36 includes, for example, a display device such as a liquid crystal display, an EL (Electro Luminescence) display, or an LED (Light Emitting Diode) display, and / or a speaker.

[0034] In addition, although not shown, each of the diagnostic device 10 and the communication device 20 may further include at least one of various sensors such as a GPS (Global Positioning System) receiver, a direction sensor, a gravity sensor, a temperature sensor, an acceleration sensor, etc., various biometric authentication functions such as fingerprints, retina, iris, face, voiceprint, etc., various devices such as a camera, a microphone, a speaker, a light, etc., and an input / output interface including a connection terminal.

[0035] <Function block configuration> (Communication Equipment) Next, a functional block configuration of a communication device according to an embodiment will be described with reference to Fig. 3. Fig. 3 is a configuration diagram showing an example of a functional block configuration of a communication device 20 in an embodiment. Note that Fig. 3 is intended to show the functional blocks described in this embodiment, and does not exclude the communication device 20 from having functional blocks other than those shown in the figure.

[0036] As shown in FIG. 3, the communication device 20 includes, as functional blocks, a storage unit 200, a communication processing unit 201, a management unit 202, an input unit 203, and an output unit 204.

[0037] The storage unit 200 stores various information necessary for the operation of the communication device 20, such as a routing table and various setting information. The communication processing unit 201 routes received data based on the routing table and the like. The management unit 202 manages the operation state of the communication device 20, specifically, the CPU usage rate, memory usage rate, the state of each interface, the data transmission / reception state, and the like. The input unit 203 accepts input of commands and the like provided in the communication device 20. The output unit 204 transmits a command response indicating the operation state of the communication device 20 to the diagnostic device 10.

[0038] The storage unit 200 may be realized using at least one of the memory 32 and the storage device 33 included in the communication device 20. The communication processing unit 201 may be realized using the communication device 34 included in the communication device 20. The management unit 202, the input unit 203, and the output unit 204 may be realized by the processor 31 of the communication device 20 executing a program stored in the storage device 33. The program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a Universal Serial Bus (USB) memory or a Compact Disc ROM (CD-ROM).

[0039] (Diagnostic Equipment) Next, a functional block configuration of a diagnostic device according to an embodiment will be described with reference to Figs. 4 to 8. Fig. 4 is a configuration diagram showing an example of a functional block configuration of the diagnostic device 10 in an embodiment. Fig. 5 is a diagram showing an example of a NW configuration DB 100a. Fig. 6 is a diagram showing an example of commands and confirmation rules executed by the communication device 20. Fig. 7 is a diagram for explaining a state vector. Fig. 8 is a diagram showing an example of a label assigned to a state vector. Note that Fig. 4 is for illustrating the functional blocks described in this embodiment, and does not exclude the diagnostic device 10 from having functional blocks other than those shown in the figure.

[0040] As shown in FIG. 4, the diagnostic device 10 includes, as functional blocks, a memory unit 100, an identification unit 101, a command generation unit 102, a collection unit 103, a preprocessing unit 104, a discrimination unit 105, an output unit 106, and a learning unit 107.

[0041] The memory unit 100 is configured to store a NW (Network) configuration DB (Data Base) 100a, a trained model 100b, a command response DB 100c, a trained model for discrimination 100d, and a teacher data DB 100e.

[0042] The NW configuration DB 100a is a database for storing information about each communication device 20 constituting the communication network N. The NW configuration DB 100a stores, for example, information indicating the connection relationship of each communication device 20, the model of the communication device 20, the communication device 20 accommodating the communication service of each user, and the identifiers of the interface and port in each communication device 20 accommodating the communication service of each user.

[0043] In the example shown in FIG. 5, the NW configuration DB 100a stores information indicating which communication device 20 the traffic of the communication service specified by each service ID passes through in which order in each communication network N1 to N4. For example, it is shown that the traffic of the communication service of service ID 1 passes through multiple communication devices 20 associated with "A1", "A2", "B1", "B2", "C1", "C2", "D1", and "D2" in the communication networks N1 to N4. Note that FIG. 5 is only an example, and the communication service of each user may pass through more communication networks N and communication devices 20. In addition, the communication devices 20 that construct each communication network N may include communication devices 20 of different models. In addition, the NW configuration DB 100a may store a physical port number or virtual port number that accommodates the communication service specified by the service ID, an interface number, etc. in each communication device 20 in association with the service ID.

[0044] The command response DB 100b stores command responses collected by sending commands to multiple communication devices 20 that accommodate a user's communication service when an operator of a communication carrier or the like identifies a fault event. The command response DB 100b stores a large amount of command responses collected when fault diagnoses were performed in the past. The command response DB 100b is intended to be used as a fault event in deep reinforcement learning, which will be described later.

[0045] The trained model 100c is a model trained by deep reinforcement learning to output a command sequence. The command sequence is a series of commands required for fault diagnosis and includes one or more commands. Details of the trained model 100c will be described later.

[0046] The trained model 100c may be generated by the learning unit 107 described later, or may be generated by another information processing device (not shown) and then received and stored in the storage unit 100. In the following description, unless otherwise specified, the trained model 100c is assumed to be generated by the learning unit 107.

[0047] The trained model for discrimination 100d is a model trained to output a label indicating an estimated fault type when a state vector generated based on command responses collected from a plurality of communication devices 20 is input. The trained model for discrimination 100d is generated by learning using training data that associates a state vector generated based on command responses collected from a plurality of communication devices 20 accommodating communication services in the communication network N with a label indicating a fault type corresponding to the feature vector.

[0048] The trained model for discrimination 100d may be any model, such as a decision tree (e.g., CART, XG Boost, etc.), SVM (support vector machine), neural network, multi-layer neural network, random forest, logistic regression analysis, etc.

[0049] In addition, the trained model for discrimination 100d may be generated by the learning unit 107 described later, similar to the trained model 100c, or may be generated by another information processing device (not shown) and stored in the storage unit 100 by receiving it, etc. In the following description, unless otherwise specified, it is assumed that the trained model for discrimination 100d was generated by the learning unit 107.

[0050] The teacher data DB100e stores teacher data in which state vectors generated based on command responses obtained from a plurality of communication devices 20 accommodating the user's communication service are associated with labels indicating the correct answers of the fault types. The above-mentioned trained model for discrimination 100d may be trained using the teacher data DB100e. The teacher data stored in the teacher data DB100e may be generated in advance through past fault diagnosis work, or may be generated using labeled state vectors generated by classifying state vectors into a plurality of classes by clustering feature vectors generated based on a large number of command responses from each communication device 20 collected in the past using unsupervised learning, and assigning labels indicating fault types to each class.

[0051] The identifying unit 101 is configured to identify a group of predetermined communication devices that accommodate a predetermined communication service from among a plurality of communication devices 20 included in the communication network N. The predetermined communication service is a communication service that is a target for which the diagnostic device 10 investigates a fault type, and may be, for example, a communication service of a user when a fault report is received from the user.

[0052] The specified communication service may include any communication service, such as a communication service from a user network U through a communication network N to the Internet, or vice versa, or a communication service from one user network U, such as a user company's base A, through a communication network N to another user network U, such as a base B of the same user company, or vice versa.

[0053] The command generation unit 102 is configured to generate, based on the identified predetermined communication device group, a command sequence to be executed by a predetermined communication device 20 included in the communication device group. The predetermined communication device group refers to one or more communication devices 20 through which data flowing through the communication service to be investigated passes, among a large number of communication devices 20 present in the communication network N. The command sequence includes one or more commands to be executed by one or more communication devices 20 included in the predetermined communication device group in order to perform fault diagnosis.

[0054] More specifically, the command generating unit 102 is configured to generate a command sequence using the trained model 100c. The trained model 100c is generated by learning a data set including response results of each command for all command types (hereinafter also referred to as "command types") used in fault diagnosis. Details of the data set will be described later.

[0055] The collection unit 103 is configured to cause a communication device 20 included in a specific communication device group identified by the command generation unit 102 to execute the command sequence generated by the command generation unit 102, and to collect a specific command response indicating the operating status of the communication device 20 included in the specific communication device group.

[0056] The preprocessing unit 104 is configured to generate a state vector based on the predetermined command responses collected by the collection unit 103. The state vector is a vector whose elements are the command responses for all commands that can be executed in the communication device 20 that accommodates the user's communication service for fault diagnosis. When the predetermined command responses collected by the collection unit 103 are multiple command responses, the preprocessing unit 104 generates a state vector for each command response. Details of the state vector will be described later.

[0057] The determining unit 105 is configured to determine the type of fault in a predetermined communication service based on the predetermined command responses collected by the collecting unit 103 .

[0058] More specifically, the discrimination unit 105 is configured to input the state vector generated by the preprocessing unit 104 to the trained model for discrimination 100d and acquire a label indicating the fault type output from the trained model for discrimination 100d. When the predetermined command response collected by the collection unit 103 is a plurality of command responses and the preprocessing unit 104 generates a state vector for each command response, the discrimination unit 105 inputs the generated plurality of state vectors to the trained model for discrimination 100d. This allows the fault type in the predetermined communication service to be discriminated.

[0059] In this way, by inputting a specific feature vector generated based on a specific command response into the trained model for discrimination 100d and obtaining a label indicating the fault type output from the trained model for discrimination 100d, the fault type can be easily determined.

[0060] The output unit 106 is configured to output information related to the fault type determined by the determination unit 105. For example, the output unit 106 may display a fault type ID as an identifier of the fault type, a message indicating the content of the fault type, or the like on a display provided in the diagnostic device 10 or a device connected to the diagnostic device 10.

[0061] In this manner, information on the fault type discriminated by the discriminating unit 105 is output. This makes it possible to easily notify the operator of the diagnostic device 10 of the information on the discriminated type.

[0062] The learning unit 107 is configured to generate the trained model 100c by deep reinforcement learning given a predetermined group of communication devices that accommodate communication services in a communication network as an environment. Details of deep reinforcement learning will be described later. This makes it possible to easily generate a trained model that generates a command sequence to be executed by a predetermined communication device 20 included in the identified predetermined group of communication devices.

[0063] Moreover, the learning unit 107 is configured to generate a trained model for discrimination 100d by training a model using the teacher data DB 100e.

[0064] As described above, the command generation unit 102 generates a command sequence required for fault diagnosis to be executed by the identified communication devices 20 accommodating the user's communication service. In the example shown in Fig. 6, a command sequence including a plurality of commands corresponding to a confirmation rule determined in advance for each communication device 20 is generated.

[0065] Here, the type of command executed for the communication device 20 accommodating the communication service of each user differs depending on the communication device 20. Furthermore, the communication device 20 accommodating the communication service of each user differs for each user.

[0066] Therefore, in this embodiment, in order to make the dimension of the state vector the same for all users, the number of all commands of all communication devices 20 accommodating the communication services of each user managed by the diagnostic device 10 is set as the number of dimensions of the state vector. For example, if there are 121 commands in total for all communication devices 20 accommodating the communication services of each user, the state vector will be a 121-dimensional vector.

[0067] Moreover, the command response from the communication device 20 is composed of a complex text message that is assumed to be visually judged by a human. On the other hand, in order to train the model and generate the trained model 100c, it is necessary to be able to treat the command response as a data set for training. Therefore, the preprocessing unit 104 of the diagnostic device 10 compares the response result of the command of each communication device 20 with a predefined confirmation rule, and converts the confirmation result for the response content of each command into a real number to generate a state vector. The real number may be, for example, two numbers, "0" representing normality and "1" representing abnormality, or may be further classified into multiple values.

[0068] As shown in FIG. 7, the "communication network" refers to all communication networks N that accommodate the communication services of each user, and the "communication device" refers to all communication devices 20 that accommodate the communication services of each user, which are installed in each communication device 20. Also, the "command ID" is an identifier that uniquely identifies all commands of all communication devices 20 that accommodate the communication services of each user. For example, if the total number of commands of all communication devices 20 that accommodate the communication services of each user is 121 commands, then Cn=C121. Note that, in the example shown in FIG. 7, communication networks N1 to N4 are illustrated, but this is illustrated for the convenience of explaining the state vector, and the number of communication networks N is not limited to this example.

[0069] One state vector is generated by converting the command responses of commands C1 to Cn into "0" or "1" according to a predefined confirmation rule. For example, if the command response of command C1 is converted into "0", the command response of command C2 is converted into "0", the command response of command C3 is converted into "1", ..., the command response of command C121 is converted into "0", respectively, the state vector will have the format of (C1, C2, C3, ..., C121) = (0, 0, 1, ..., 0).

[0070] As described above, the teacher data stored in the teacher data DB 100d is a labeled state vector in which a label is added to a state vector. Fig. 8 shows the labels added to the state vector in a table format including, for example, nine types of label IDs from "1" to "9" and the fault types represented by the label IDs. In this case, the labeled state vector is, for example, a state vector in the format of (label ID, C1, C2, C3,..., C121) = (5, 0, 0, 1,..., 0).

[0071] Here, in order to shorten the processing time of the fault diagnosis work, the model may be trained so that a command sequence to be used in the fault diagnosis is obtained from the commands used in the past fault diagnosis and the command responses to those commands. In this case, the model may be trained so that the obtained command sequence is a command sequence containing fewer or a smaller number of commands. As a result, the fault location and the cause of the fault can be identified more quickly, and the processing time can be shortened.

[0072] One possible learning method is reinforcement learning, a type of machine learning in which an agent (also called a "learner") is given data and made to learn. Reinforcement learning is characterized by having the agent learn from given data with an emphasis on maximizing value in the long term. In reinforcement learning, the agent takes "actions" in the "environment" (also called "preconditions") and repeats a cycle of receiving "rewards" based on the results, learning optimal actions. Therefore, after sufficient reinforcement learning has been performed, the agent will take various actions, memorize the results through trial and error, and will be able to determine which action is most valuable when certain preconditions are given.

[0073] Generally, Q-learning is used as a representative algorithm in reinforcement learning. Q-learning is a learning method in which the value of an action (also called the "Q value") is entered into a Q-table every time an action is taken, and the value is updated every time a new action is taken.

[0074] However, in the case of a command sequence in a large-scale and complicated communication service, when the number of past states, i.e., the number of commands included in the command sequence, increases, the state space becomes enormous. For example, if the total number of all commands of all communication devices 20 that accommodate the communication service is 121 commands, i.e., the action space is 121, and if the response content of each command is expressed in two states, normal and abnormal, the state space becomes 2 to the power of 121 (=2 121 ) Therefore, in Q-learning, the computational resources required for learning increase and expand to an amount of data that is difficult to handle.

[0075] In contrast, in this embodiment, the command generation unit 102 is configured to obtain a command sequence by providing a specified group of communication devices to the trained model 100c trained by deep reinforcement learning. That is, the trained model 100c is generated by deep reinforcement learning, which is a combination of reinforcement learning and deep learning (also called "deep learning"). Then, the command generation unit 102 obtains a command sequence as an action by providing a specified group of communication devices as an environment to the trained model 100c generated by deep reinforcement learning.

[0076] In this way, by providing a specified group of communication devices to the trained model 100c generated by deep reinforcement learning to obtain a command sequence, it is possible to obtain the most valuable action, that is, a command sequence containing a reduced number of commands. Therefore, the location and cause of the failure can be identified in a short time, and the failure diagnosis in the communication network can be performed efficiently.

[0077] The trained model 100c is trained using a value network for determining commands included in a command sequence, and a target value network for estimating a maximum delay reward for a plurality of state space vectors, each of which is a deep neural network, using state space vectors based on command responses collected from a plurality of communication devices accommodating communication services in a communication network as input. Details of the state space vectors will be described later. In this way, by using two neural networks, it is possible to approximate a complex and diverse state space, and reinforcement learning of a huge amount of past history is possible.

[0078] In addition, the trained model is trained using multiple state space vectors so that the target value network is updated by the value network in a cycle different from the cycle in which the weights in the value network are updated. This makes it possible to suppress divergence and oscillation of the delayed reward, which is the predicted value by the target value network.

[0079] Furthermore, the weights in the value network are updated in each of the multiple state space vectors, and the target value network is updated in the value network for each of the multiple state space vectors. This makes it possible to perform deep reinforcement learning in which the objective function is common to each failure event.

[0080] The storage unit 100 may be realized using at least one of the memory 32 and the storage device 33 included in the diagnostic device 10. The identification unit 101, the command generation unit 102, the collection unit 103, the preprocessing unit 104, the discrimination unit 105, the output unit 106, and the learning unit 107 may be realized by the processor 31 of the diagnostic device 10 executing a program stored in the storage device 33. The program can be stored in a storage medium. The storage medium storing the program may be a computer-readable non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM.

[0081] <Processing procedure for generating trained models> Next, with reference to FIG. 9 to FIG. 15, a processing procedure of trained model generation performed by a diagnostic device according to an embodiment will be described. FIG. 9 is a flowchart for explaining an example of trained model generation processing S110 performed by a diagnostic device 10 in an embodiment. FIG. 10 is a diagram showing an example of visualizing a feature vector. FIG. 11 is a diagram for explaining a flow of repetition in trained model generation processing S110. FIG. 12 is a diagram for explaining a state space vector. FIG. 13 is a diagram for explaining an experience replay processing S116. FIG. 14 is a flowchart for explaining an example of a command determination processing S130. FIG. 15 is a flowchart for explaining an example of an emulator execution processing S140. In FIG. 10, the vertical axis corresponds to the command type, and the horizontal axis corresponds to each failure event.

[0082] In the following description, unless otherwise specified, it is assumed that the number of failure events J is preset to "300" and the maximum number of learning M is preset to "40".

[0083] (Trained model generation process) As shown in Fig. 9, the learning unit 107 first performs processing (S111). The initial processing includes setting initial values ​​to variables and initializing the data area to be used and two neural networks to be used in deep reinforcement learning. Specifically, the variables are a variable j that is an index indicating the selected failure event, a variable n that is an index indicating the number of states, a variable m that is an index indicating the number of learnings, and the like. The learning unit 107 sets, for example, "0" as the initial values ​​of the variable j, the variable n, and the variable m.

[0084] As described above, the data set used in deep reinforcement learning is a state vector. As shown in FIG. 10, the feature vector has the total number of dimensions of all command types, for example, 121 dimensions. The location where a point exists indicates that the command response of the command type is "abnormal." As is clear from FIG. 10, it can be seen that the frequency of command types whose command responses are "abnormal" is low compared to all command types in all past failure events.

[0085] In this embodiment, the characteristics of this state vector are utilized, and a learning approximation is performed by preferentially executing commands that result in an "abnormal" command response in each failure event.

[0086] Here, the flow of repetition in the trained model generation process S110 is as shown in FIG. 11. That is, for each failure event, a state in which a certain command is executed and a command response is obtained is regarded as one step (also written as "step"), and each step is repeated until the end judgment condition is met. In the example shown in FIG. 11, for one failure event, n steps from "step1" to "stepn" are performed by executing n commands (n is an integer equal to or greater than 2) respectively, and a command is searched for. Note that, for convenience of illustration and explanation, FIG. 11 shows an example in which the same n steps are performed for each failure event, but this is not limiting. The number of steps performed may be different for each failure event.

[0087] Then, when the search for all failure events is completed, this learning is counted as one time and is repeated a number of times, for example up to a learning count m (m is an integer equal to or greater than 2).

[0088] As described above, each step is a state in which a command is executed and a command response is obtained, and therefore can be represented by a state vector. In addition, generally, a certain step, for example, the nth step n, has n-1 steps, from the 1st step to the (n-1)th step, performed before it. Therefore, as shown in FIG. 12, the nth step n can be represented by a state space vector including a state vector when the step is performed and (n-1) past state vectors. Hereinafter, the number of past state vectors included in the state space vector is also referred to as the history number.

[0089] 9, after the initial process S111, the learning unit 107 updates the variable m (S112). For example, the learning unit 107 updates the variable m by adding "1".

[0090] Next, the learning unit 107 updates the variable j (S113). For example, the learning unit 107 updates the variable m by adding "1". Then, the learning unit 107 reads out and selects a failure event corresponding to the variable j from the command response DB 100b using the updated variable j as a key (S114). The learning unit 107 converts the selected failure event into a state vector. Furthermore, the learning unit 107 prepares an array whose number corresponds to the elements of the converted state vector. In this step S114, the learning unit 107 sets "0" representing unexecuted to all the arrays, thereby performing so-called zero padding.

[0091] Next, the learning unit 107 performs a command determination process (S130). In the command determination process S130, the ε-greedy strategy, which is one of the reinforcement learning methods, is adopted to determine the next command to be executed. The command determination process S130 will be described in detail later.

[0092] Next, the learning unit 107 performs an emulator execution process (S130). In the emulator execution process S140, a reward (also called an "instant reward") is calculated for the command determined in the command determination process S130, and a termination determination is performed. Details of the emulator execution process S140 will be described later.

[0093] Next, the learning unit 107 reflects the command response of the command in the array that corresponds to the command for which a reward was given by the emulator execution process S140 in the array filled with zeros in step S114 (S115). Since the command response is, for example, "1" or "0", it is possible to determine whether a certain command has been executed or not by referring to the corresponding array.

[0094] Next, the learning unit 107 performs an experience replay process (S116). As shown in the upper part of FIG. 13, in the experience replay process S116, a plurality of state space vectors are stored in the data area initialized in the initial process S111. Each state space vector is a concatenation of a plurality of state vectors. For example, the state space vector in the first row includes "state n", which is a state vector in which the nth command in the command sequence has been executed, and "state 1" to "state n-1", which are state vectors in which (n-1) past commands in the command sequence have been executed. In addition, the state space vector further includes "command n", which indicates the command type of the command determined in the command determination process S130, and "reward n", which indicates the reward given to the command in the emulator execution process S140.

[0095] In the example shown in Fig. 13, m such state space vectors (m is an integer equal to or greater than 2) are queued in the data area. Note that the number of queued state space vectors depends on the size of the allocated data area.

[0096] In the experience replay process S116, some of the m state space vectors are extracted, as shown in the lower part of Fig. 13. In the example shown in Fig. 13, l (l is an integer between 2 and m) state space vectors that are queued are extracted. The number of state space vectors that are extracted depends on the size of the memory block.

[0097] Then, among the multiple state space vectors extracted, the state vectors corresponding to the past history are concatenated to form a new state space vector, which is then input to the target value network. The output estimated by the target value network is the maximum delayed reward (also called the "maximum delayed reward"). Meanwhile, the immediate reward among the multiple state space vectors extracted is added to this maximum delayed reward, and the result is input to the value network.

[0098] Returning to the explanation of FIG. 9, next, the learning unit 107 updates the value network (S117). Specifically, when the immediate reward and the maximum delayed reward of the extracted multiple state space vectors shown in the lower part of FIG. 13 are added and input to the value network, the value network is fitted to the extracted multiple state space vectors. As a result, the weights in the value network are updated, and the value network is updated. In this way, the weights of the value network are updated for each state, that is, each time an execution command is determined. Note that the update of the value network in step S117 may be performed in the experience replay process S116.

[0099] Next, the learning unit 107 updates the state space vector (S118). For example, when updating the state space vector in the first row shown in the upper part of Fig. 13 to the state space vector in the second row, the (n-1)th past state vector from the second state vector in the state space vector in the first row and the nth current state vector are concatenated to generate the state space vector in the second row. Note that, at this point in time, the (n+1)th command to be executed, the (n+1)th state vector, and its reward are undecided for the updated state space vector.

[0100] Next, the learning unit 107 judges whether or not the end determination flag FL is "1" (S119). The end determination flag FL is set in the emulator execution process S140.

[0101] If it is determined in step S119 that the end determination flag FL is not "1", the command determination process S130 through step S119 are repeated until the end determination flag FL becomes "1".

[0102] On the other hand, if the result of the judgment in step S119 is that the end judgment flag FL is "1", it is considered that the updating of the weights of the value network for the failure event selected in step S114 has been completed. Therefore, the learning unit 107 updates the target value network with the value network (S120). In this way, the target value network is updated in a cycle different from the cycle in which the weights of the value network are updated, and the weights of the value network for which the update has been completed are reflected in the target value network.

[0103] Next, the learning unit 107 judges whether or not the variable j is equal to or greater than the number of fault events J (S121). If the judgment result in step S121 indicates that the variable j is not equal to or greater than the number of fault events J, that is, is less than the number of fault events J, steps S113 to S121 are repeated until the variable j becomes equal to or greater than the number of fault events J.

[0104] On the other hand, if the result of the judgment in step S121 is that the variable j is equal to or greater than the number of fault events J, it is considered that deep reinforcement learning has been performed once for all the prepared fault events. Next, the learning unit 107 judges whether the variable m is equal to or greater than the maximum learning number M (S122). If the result of the judgment in step S122 is that the variable m is not equal to or greater than the maximum learning number M, that is, if the variable m is equal to the maximum learning number M, steps S112 to S122 are repeated until the variable m is equal to or greater than the maximum learning number M.

[0105] On the other hand, if the result of the determination in step S122 is that the variable m is equal to or greater than the maximum learning number M, it is considered that deep reinforcement learning has been performed the maximum learning number M times for all prepared failure events. Therefore, the learning unit 107 ends the trained model generation process S110.

[0106] (Command execution process) As shown in Fig. 14, the learning unit 107 first judges whether the order of the command to be determined in the state space vector is smaller than a predetermined number of past states (S131). Specifically, for example, in the state space vector shown in Fig. 12, when the number of linked past states is 5, that is, (n-1) in "state (n-1)" is 5, it is judged whether the order of the command to be determined is 6 or later, that is, whether n in "command n" is 6 or more.

[0107] If the result of the determination in step S131 is that the order of the command to be determined is smaller than the number of past states, it is considered that there are already a sufficient number of past state vectors, and these multiple state vectors can be linked and converted into a state space vector. In this case, the learning unit 107 calculates a decay value ε that decays as the number of states increases (S132). Such a decay value ε can be calculated by various calculation methods. For example, the decay value ε is calculated by the following formula (1) using an initial value init, a lower limit value min, the total number of commands epi_st in the selected failure event, the current command order st, and a constant ε0. ε=ε0-{(init-min) / epi_st}×st …(1)

[0108] After step S132, the learning unit 107 determines whether the calculated attenuation value ε is equal to or greater than a random value (S133). The random value is a random number between 0 and 1, and a new random value is generated every time step S132 is performed.

[0109] If the result of the determination in step S133 is that the attenuation value ε is equal to or greater than the random value, the learning unit 107 concatenates a plurality of past state vectors to convert them into a state space vector (S134). The concatenated plurality of past state vectors correspond to, for example, the state vectors from "State 1" to "State n-1" in the example shown in FIG.

[0110] After step S134, the learning unit 107 inputs the converted state space vector into the value network, and determines the command output from the value network as the command to be executed (S135). The determined command corresponds to the command "command n" in the example shown in FIG.

[0111] After step S135, the learning unit 107 ends the command determination process S130.

[0112] On the other hand, if the result of the determination in step S131 is that the order of the command to be determined is not greater than the number of past states, i.e., is equal to or less than the number of past states, or if the result of the determination in step S133 is that the attenuation value ε is not greater than or equal to the random value, i.e., is less than the random value, the learning unit 107 determines a command to be executed that is randomly selected from the unexecuted commands (S136). The determined command corresponds to, for example, the command "command n" in the example shown in FIG.

[0113] After step S136, the learning unit 107 ends the command determination process S130.

[0114] (Emulator execution process) 15, the learning unit 107 first sets an initial value to the end determination flag FL (S141). The initial value to be set is, for example, "0."

[0115] Next, the learning unit 107 generates a new state vector based on the failure event determined in step S114 and the command determined by the command determination process S130 (S142). The generated state vector corresponds to the state vector of "state n" in the example shown in FIG.

[0116] Next, the learning unit 107 assigns a reward to the command determined by the command determination process S130 based on the command response in the fault event determined in step S114 (S143). For example, if the command response is "1" indicating an abnormality, a reward of "100" is assigned, and if the command response is "0" indicating normality, a reward of "-1" is assigned. In this way, by setting a positive high reward for a command whose command response indicates an abnormality, it becomes possible to contribute to the search for a command that can identify the fault type.

[0117] Next, the learning unit 107 performs an end judgment process to judge whether or not to end the search for commands to be executed (S144). In the end judgment process, based on the array reflecting the command responses in step S115, it is judged whether or not all of the commands with abnormal command responses among the failure events determined in step S114 have been executed. If all of the commands with abnormal command responses have been executed, the end judgment flag is set to "1".

[0118] Then, after the end determination process S144, the learning unit 107 ends the emulator execution process S140.

[0119] In this way, a trained model 100c that generates a command sequence is generated by learning through deep reinforcement learning using a value network and a target value network, each of which is a deep neural network.

[0120] <Evaluation of search performance> Next, the search performance of a command sequence generated by a diagnostic device according to an embodiment will be described with reference to Figs. 16 and 17. Fig. 16 is a diagram showing an example of a Q value in deep reinforcement learning when generating a trained model 100c. Fig. 17 is a graph showing an example of a distribution of search scores of a command sequence. In Fig. 16, the vertical axis corresponds to a failure event, the horizontal axis corresponds to the number of steps in the search learning, and each plot corresponds to a Q value represented by a color on the color scale shown on the right side of the graph. Figs. 16 and 17 are for the case where the maximum number of learnings M is set to 10 and the number of past state vectors included in the state space vector, that is, the number of histories, is set to 3 in the trained model generation process S110 described above.

[0121] As shown in Fig. 16, deep reinforcement learning progresses from the upper left to the lower right in the figure. According to ε decay in the command determination process S130, a command is acquired by the value network, and the command is executed in the emulator execution process S140. As the variable m, which is the number of iterations of deep reinforcement learning, increases, the color of the plot gradually becomes darker and the Q value increases. From Fig. 16, it can be seen that a command with a high Q value is executed while searching for a command for each failure event.

[0122] Fig. 17 shows the distribution of search scores when searching for a command sequence that can identify the fault type for 1000 fault events. This search is also called a search test. The search score is a value obtained by estimating the fault type for an array of values ​​indicating normal (or OK) or abnormal (or NG) obtained for each command execution in the search test, and normalizing the number of commands included in the command sequence that converged to the correct fault type assigned as a label by the total number of commands, for example, 121. In other words, a smaller search score means that the correct fault type could be identified with a smaller number of commands or a shorter command sequence.

[0123] The graph on the right side of Fig. 17 shows the distribution of search scores when using a command sequence randomly generated from all commands. In this case, the search scores for the 1000 failure events are distributed over a relatively wide range, with the average search score being 0.41.

[0124] In contrast, the graph shown on the left side of Fig. 17 is the distribution of search scores when using a command sequence obtained by the trained model 100c trained by deep reinforcement learning. In this case, the search scores for 1000 fault events were distributed in a relatively narrow range, specifically, mainly in the range greater than 0 and less than or equal to 0.2, and the average search score was 0.12. As a result, the fault diagnosis device 20 equipped with the trained model 100c trained by deep reinforcement learning was able to estimate the fault content with a command sequence including about 15 commands for a scale of 121 total commands and 9 fault types (9 classes), and the improvement effect of the search performance was confirmed.

[0125] <Fault type determination procedure> Next, a procedure of a fault type determination process performed by the diagnostic device according to an embodiment will be described with reference to Fig. 18. Fig. 185 is a flowchart for explaining an example of a fault type determination process S150 performed by the diagnostic device 10 in an embodiment.

[0126] In the following description, it is assumed that the trained model 100c is stored in advance in the memory unit 100.

[0127] When a fault report is received from a user, as shown in FIG. 18, the identifying unit 101 refers to the NW configuration DB 100a and identifies a plurality of communication devices 20 that accommodate the communication service of the user who has received the report (S151).

[0128] Next, the command generating unit 102 generates a command sequence required for fault diagnosis based on the identified multiple communication devices 20 accommodating the user's communication service (S152). More specifically, the command generating unit 102 obtains a command sequence by providing the multiple communication devices 20 identified in step S151 as an environment to the trained model 100c. The command sequence thus obtained becomes a series of commands to be executed by the multiple communication devices 20 accommodating the user's communication service.

[0129] Next, the collection unit 103 logs into each of the communication devices 20 identified in step S151, executes the command sequence generated in step S152, and collects command responses (S153).

[0130] Next, the preprocessing unit 104 converts the command responses collected in step S154 into a state vector (S154). More specifically, the preprocessing unit 104 generates a state vector by converting the received command responses into "0" indicating normality or "1" indicating abnormality according to a predefined confirmation rule. Here, the preprocessing unit 104 sets "0" to a dimension corresponding to a command ID not included in the received command response in the state vector having elements C1 to Cn. For example, when the state vector has 121 dimensions from C1 to C121, it is assumed that the command response includes a total of 80 command responses with command IDs from 1 to 40 and 61 to 100. In this case, the preprocessing unit 104 sets "0" to a dimension corresponding to command IDs from 41 to 60 and 101 to 121 among C1 to C121 of the state vector.

[0131] Next, the discrimination unit 105 inputs the state vector generated in step S154 to the trained model for discrimination 100d, acquires a label indicating the fault type output from the trained model for discrimination 100d, and discriminates the fault type (S155). This allows the fault type of the fault reported by the user to be discriminated.

[0132] Next, the output unit 106 outputs the information on the fault type acquired in step S155 to a display or the like (S156). The information on the fault type is, for example, a statement describing the contents of the fault type.

[0133] After step S156, the diagnostic device 10 ends the failure type determination process S150.

[0134] The sequence and flow charts described in this embodiment may be rearranged as long as no inconsistency occurs in the processing.

[0135] Furthermore, the processes described in the present embodiment may be realized by hardware in the device, or may be realized by a processor executing a program stored in a storage device. When a program is executed, the program may be stored in a storage medium. The storage medium storing the program may be a computer-readable non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a USB memory or a CD-ROM.

[0136] An exemplary embodiment of the present invention has been described above. According to the diagnostic device 10, diagnostic method, diagnostic program, and diagnostic system 1 of the present embodiment, a specified group of communication devices is given to the trained model 100c generated by deep reinforcement learning to obtain a command sequence. This makes it possible to obtain the most valuable action, that is, a command sequence containing a reduced number of commands. Therefore, the fault location and the fault cause can be identified in a short time, and fault diagnosis in the communication network can be performed efficiently. Therefore, the technology according to the present embodiment can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure and promote industrialization and innovation."

[0137] The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The present invention may be modified / improved without departing from the spirit thereof, and equivalents are also included in the present invention. That is, those in which a person skilled in the art appropriately changes the design of each embodiment are also included in the scope of the present invention as long as they have the characteristics of the present invention. For example, the elements and their arrangements, materials, conditions, shapes, sizes, etc. of the embodiments are not limited to those exemplified, and can be appropriately changed. In addition, each embodiment is an example, and it goes without saying that partial replacement or combination of the configurations shown in different embodiments is possible, and these are also included in the scope of the present invention as long as they include the characteristics of the present invention. [Explanation of symbols]

[0138] 1...diagnosis system, 10...diagnosis device, 20...communication device, 31...processor, 32...memory, 33...storage device, 34...communication device, 34...output device, 35...input device, 36...output device, 100...storage unit, 100...wireless communication system, 100a...network configuration DB, 100b...command response DB, 100c...trained model, 100d...trained model for discrimination, 100e...teaching data DB, 101...identification unit, 102...command generation unit, 103...collection unit, 104...preprocessing unit, 105...discrimination unit, 106...output unit, 107...learning unit, 107...learning unit, 200...memory unit, 201...communication processing unit, 202...management unit, 203...input unit, 204...output unit, S110...learned model generation processing, S116...experience replay processing, S130...command determination processing, S140...emulator execution processing, S150...fault type discrimination processing.

Claims

1. an identification unit that identifies a predetermined communication device group that accommodates a predetermined communication service from among a plurality of communication devices included in a communication network; a command generation unit that generates, based on the specified group of communication devices, a command sequence to be executed by a specified communication device included in the group of communication devices; a collection unit that causes the predetermined communication device to execute the generated command sequence and collects a predetermined command response indicating an operation state of the predetermined communication device; a determination unit that determines a type of fault in the predetermined communication service based on the predetermined command response, The command generation unit obtains the command sequence by providing the identified predetermined communication device group to a trained model trained by deep reinforcement learning. Diagnostic equipment.

2. The trained model is trained using a value network for determining commands to be included in the command sequence using state space vectors based on command responses collected from a plurality of communication devices accommodating communication services in the communication network, each of which is a deep neural network, and a target value network for estimating a maximum delay reward for the plurality of state space vectors. The diagnostic device of claim 1 .

3. The learned model is trained using a plurality of state space vectors so as to update the target value network with the value network in a period different from a period in which the weights in the value network are updated. The diagnostic device of claim 2.

4. Weights in the value network are updated in each of the plurality of state space vectors, and the target value network is updated with the value network for each of the plurality of state space vectors. The diagnostic device of claim 3.

5. A learning unit that generates the trained model by deep reinforcement learning given an environment of a predetermined group of communication devices that accommodate communication services in the communication network. The diagnostic device of claim 1 .

6. the discrimination unit inputs a predetermined feature vector generated based on the predetermined command response to a trained model for discrimination trained using training data in which a feature vector generated based on command responses collected from a plurality of communication devices accommodating communication services in the communication network and a label indicating a fault type corresponding to the feature vector are associated with each other, and obtains a label indicating a fault type output from the trained model for discrimination; The diagnostic device of claim 1 .

7. An output unit that outputs information related to the determined fault type is further provided. The diagnostic device of claim 1 .

8. Identifying a group of predetermined communication devices that accommodate a predetermined communication service from among a plurality of communication devices included in a communication network; generating a command sequence to be executed by a predetermined communication device included in the specified group of communication devices based on the specified group of communication devices; causing the predetermined communication device to execute the generated command sequence and collecting a predetermined command response indicating an operational state of the predetermined communication device; determining a fault type in the predetermined communication service based on the predetermined command response; The step of generating the command sequence includes providing the identified predetermined communication device group to a trained model trained by deep reinforcement learning to obtain the command sequence. Diagnostic methods.

9. A diagnostic program for causing a computer to execute the program, Identifying a group of predetermined communication devices that accommodate a predetermined communication service from among a plurality of communication devices included in a communication network; generating a command sequence to be executed by a predetermined communication device included in the specified group of communication devices based on the specified group of communication devices; causing the predetermined communication device to execute the generated command sequence and collecting a predetermined command response indicating an operational state of the predetermined communication device; determining a fault type in the predetermined communication service based on the predetermined command response; The step of generating the command sequence includes providing the identified predetermined communication device group to a trained model trained by deep reinforcement learning to obtain the command sequence. Diagnostic program.

10. A diagnostic system including a plurality of communication devices and a diagnostic device included in a communication network, The communication device includes: A management unit for managing an operation state of the communication device; a transmission unit that transmits a command response indicating an operation state of the communication device to the diagnostic device; Equipped with The diagnostic device comprises: an identification unit that identifies a predetermined communication device group that accommodates a predetermined communication service from among the plurality of communication devices; a command generation unit that generates, based on the specified group of communication devices, a command sequence to be executed by a specified communication device included in the specified group of communication devices; a collection unit that causes the predetermined communication device to execute the generated command sequence and collects a predetermined command response indicating an operation state of the predetermined communication device; a determination unit that determines a type of a fault in the predetermined communication service based on the response to the predetermined command; Equipped with The command generation unit obtains the command sequence by providing the identified predetermined communication device group to a trained model trained by deep reinforcement learning. Diagnostic system.

Images