Trained model generation device, fault detection device, trained model generation method, fault detection method, trained model generation program, and fault detection program

Abstract

A trained model generation device (100) comprises: an observation unit or units (10, 11, 12, ..., 19) for collecting and observing feature quantities of one or a plurality of base station devices (300, 301, 302, ..., 309); a preprocessing unit (40) for scaling the collected feature quantities as training data for creating a trained model, said scaling being performed per base station device (300, 301, 302, ..., 309), per date and time at which the feature quantities were collected, and per feature quantity; and a training apparatus (50) for using the scaled training data to perform machine learning for detecting a fault in the base station device or devices (300, 301, 302, ..., 309) and generate a trained model.

Description

Learning model generation device, fault detection device, learning model generation method, fault detection method, learning model generation program, and fault detection program

[0001] The present invention relates to a learning model generation device, a fault detection device, a learning model generation method, a fault detection method, a learning model generation program, and a fault detection program.

[0002] For example, a base station device is known as a device that relays wireless communications between mobile phones, smartphones, etc. (see Non-Patent Document 1). Also, LTE (Long Term Evolution) has been disclosed as a wireless access method between mobile phones, smartphones, etc. and a base station device (see Non-Patent Document 2).

[0003] To provide users with a comfortable mobile communication service, it is necessary to maintain network quality. Specifically, network quality maintenance work consists of three phases: (1) monitoring, (2) analysis, and (3) action. Systematized procedures are established based on alarm information mainly from faulty devices. However, there are also anomalies that are not defined as alarms (hereinafter referred to as silent anomalies), and discovering and responding to these silent anomalies is an important task for network maintainers (see Non-Patent Document 3).

[0004] Furthermore, as an approach to detecting silent anomalies, Non-Patent Document 3 considers detecting silent anomalies by multiplying the communication volume of a wireless base station by an AutoEncoder.

[0005] Incidentally, AI (Artificial Intelligence) is currently experiencing its third boom, and the functions that AI performs can be broadly divided into three types: "identification," "prediction," and "execution." Therefore, with regard to maintaining network quality, studies are being conducted to improve the efficiency of maintenance work by applying the AI ​​functions of "identification" and "prediction" to (1) monitoring and (2) analysis, and by applying the AI ​​function of "execution" to (3) measures (see Non-Patent Document 4).

[0006] "Saito Shigeki, Ebine Yoshio, Tamori Toshihiro, Kitagawa Masakiyo, Shintaku Masayoshi," "Radio Base Station Equipment," NTT DoCoMo Technical Journal, Vol. 1 No. 1, pp. 33-38, July 1993. [online], [Retrieved June 12, 2023], Internet <https: / / www.docomo.ne.jp / binary / pdf / corporate / technology / rd / technical_journal / bn / vol1_1 / vol1_1_033jp.pdf> "Okubo Naoto, Umesh Anil, Iwamura Mikio, Arata Hiroyuki," "Overview of LTE Wireless System Achieving High Speed, Large Capacity, and Low Latency," NTT DoCoMo Technical Journal, Vol. 19 No. 1, pp. 11-19, Apr. 2011. [online] [Retrieved June 12, 2023], Internet <https: / / www.docomo.ne.jp / binary / pdf / corporate / technology / rd / technical_journal / bn / vol19_1 / vol19_1_011jp.pdf> “Detecting ‘unusual’ behavior in base stations using AutoEncoder”, NTT DOCOMO Developer Blog, [online] [Retrieved June 12, 2023], Internet <https: / / nttdocomo-developers.jp / entry / 202212231200> “Akiaki Shiotsu, Tatsuya Kimura, Masahiro Yokota, Koji Yamamoto, Kazuhisa Yamagishi”, “Realizing advanced maintenance operations with AI”, NTT DOCOMO Technical Journal, Vol. 27 No. 1, pp. 28-35, Apr. 2019, [online], [Retrieved June 12, 2023], Internet: <https: / / www.docomo.ne.jp / binary / pdf / corporate / technology / rd / technical_journal / bn / vol27_1 / vol27_1_006jp.pdf>

[0007] For example, a fault detection device is generally known that determines whether a base station device (communication device) is faulty based on the traffic volume of data collected by a traffic volume monitoring device. The mechanism by which the fault detection device detects a fault can be constructed by machine learning, for example, using AutoEncoder described in Non-Patent Document 3.

[0008] However, when a fault detection device detects a fault using machine learning, the fault detection device may not be able to properly detect a fault because there are large differences in traffic characteristics between base stations and time periods, and even if it does detect a fault, there may be many false positives.

[0009] The present invention has been made in view of the above points, and an object of the present invention is to improve the accuracy of detecting a failure in a communication device and to reduce false detections.

[0010] The learning model generation device of the present invention is characterized by comprising an observation unit that collects and observes features of one or more communication devices; a preprocessing unit that scales the collected features to serve as learning data for creating a learning model, for example, the communication device, the date and time the features were collected, and each of the features; and a learning device that uses the scaled learning data to perform machine learning to detect failures in the communication devices and generate a learning model.

[0011] According to the present invention, it is possible to improve the accuracy of detecting a failure in a communication device and reduce false detections.

[0012] FIG. 1 is a block diagram showing the overall configuration of a fault detection system according to an embodiment of the present invention. FIG. 2 is a database showing learning data stored in a learning database. FIG. 3 is an explanatory diagram showing a schematic hardware configuration of a learning model generation device according to an embodiment of the present invention. FIG. 4 is an explanatory diagram showing a schematic hardware configuration of a fault detection device according to an embodiment of the present invention. FIG. 5 is a flowchart showing a process in which a learning model generation device according to an embodiment of the present invention performs machine learning and generates a learning model. FIG. 6 is a flowchart showing a process in which a fault detection device according to an embodiment of the present invention collects feature quantities of each base station device and estimates whether the base station device is normal or not based on those feature quantities.

[0013] Next, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described. Hereinafter, the same members and configurations will be denoted by the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0014] <Configuration of Failure Detection System> FIG. 1 is a block diagram showing the overall configuration of a failure detection system 250 according to this embodiment.

[0015] 1, a fault detection system 250 according to this embodiment is configured to include a learning model generation device 100 and a fault detection device 200. The fault detection system 250 acquires features from base station devices 301, 302, ..., 309. By acquiring the features of the base station devices 301, 302, ..., 309, the fault detection system 250 detects whether or not there is a fault in each of the base station devices 301, 302, ..., 309.

[0016] In this embodiment, the learning model generation device 100 and the fault detection device 200 will be described separately, but the fault detection system 250 can realize the learning model generation device 100 and / or the fault detection device 200 in a single unit by including the learning model generation device 100 and the fault detection device 200. Furthermore, the fault detection system 250 may share the components of the learning model generation device 100 and the fault detection device 200.

[0017] Each of the base station devices 301, 302, ..., 309 shown in Figure 1 is a device that relays communication between a mobile phone or smartphone and a switching center. Each of the base station devices 301, 302, ..., 309 relays communication between the mobile phone or smartphone and the switching center, thereby enabling communication between terminals such as mobile phones and smartphones. Note that each of the base station devices 301, 302, ..., 309 has the same configuration, and unless otherwise specified, will be simply referred to as base station device 300. Furthermore, base station device 300 is an example of a communication device, and is not particularly limited to a wireless base station.

[0018] The learning model generation device 100 includes observation units 11, 12, ..., 19, a learning database 20, a label assignment unit 30, a preprocessing unit 40, a learning device 50, and a model 60 (learning model).

[0019] Each of the observation units 11, 12, ..., 19 collects and observes the features of one or more base station devices 301, 302, ..., 309. That is, each of the observation units 11, 12, ..., 19 collects and observes the features of each of the corresponding base station devices 300. The features of the base station devices 300 include any information, such as the communication volume, fluctuation, and CPU (Central Processing Unit) usage rate of the base station devices 300. This allows each of the observation units 11, 12, ..., 19 to collect the communication status and state of the corresponding base station devices 300. Note that each of the observation units 11, 12, ..., 19 has the same configuration and will be simply referred to as the observation unit 10 unless otherwise specified. Furthermore, while the observation unit 10 is illustrated as a pair with the base station device 300, this is not limited thereto, and one observation unit 10 may collect the features of multiple base station devices 300.

[0020] The learning database 20 stores the features collected by each observation unit 10. For example, the learning database 20 stores information about which base station device 300 the communication traffic was collected from and when the traffic was collected. The learning database 20 may also associate each collected feature with a label used in machine learning and store the associated feature. The label stores the status of the base station device 30 as either normal or faulty.

[0021] 2 is a database showing the training data stored in the training database 20. When the observation unit 10 collects communication volumes as features, the training database 20 stores the communication volumes and labels for each ID (IDentification) of the base station device 300 and the date and time of collection.

[0022] 2, the learning database 20 has columns for the ID of the base station device 300, date, time, communication volumes A to C, and label. For example, the ID of the base station device 300 "0001" indicates the ID of the base station device 301, and communication volumes A to C and labels are entered for 1:00 on March 27, 2023, 1:00 PM on March 27, 2023, and 1:00 AM on March 28, 2023, respectively. Note that in FIG. 2, all of the labels for the base station device 300 with the ID "0001" are normal.

[0023] Furthermore, the ID of base station device 300 "0002" indicates the ID of base station device 302, and similarly, communication volumes A to C and labels are input for 1:00 on March 27, 2023, 1:00 PM on March 27, 2023, and 1:00 AM on March 28, 2023. Note that in FIG. 2, for the label of base station device 300 with ID "0002", a fault label is assigned to 1:00 on March 27, 2023 and 1:00 PM on March 27, 2023, and a normal label is assigned only to 1:00 on March 28, 2023.

[0024] Note that labels are optional components, and for example, when unsupervised learning is performed in the learning device 50 described below, the learning database 20 does not need to be assigned labels.

[0025] The labeling unit 30 assigns a label indicating whether the feature is normal or not (i.e., a fault) to the feature held in the training database 20. The labeling unit 30 may assign labels by a user, and is not particularly limited.

[0026] The preprocessing unit 40 scales the collected feature quantities as learning data for creating a learning model for each communication device, the date and time when the feature quantities were collected, and each feature quantity. The preprocessing unit 40 performs normalization or standardization as scaling of the collected communication volume, for example. In this embodiment, a specific example will be described in which the preprocessing unit 40 performs standardization.

[0027] First, the preprocessing unit 40 calculates the average value for each i, j, k for all sampled traffic volumes using the following equation.

[0028]

[0029] Next, the preprocessing unit 40 calculates the standard deviation for each of i, j, and k using the following formula.

[0030]

[0031] Then, the pre-processing unit 40 performs standardization using the following formula:

[0032]

[0033] In this way, the preprocessing unit 40 scales the collected features as learning data for creating a learning model.

[0034] The learning device 50 performs machine learning for detecting a fault in the base station device 300 using the scaled training data, and generates a model 60 (training model). The learning device 50 performs machine learning on, for example, the standardized communication volume. The learning device 50 may also perform machine learning using a label indicating whether the state of the base station device 300 is normal (or whether it is faulty) at the time the training data is observed. The learning device 50 performs machine learning using the label, thereby generating a model 60 that estimates whether the state is normal (or whether it is faulty).

[0035] When performing machine learning without using labels, the learning device 50 can perform grouping to classify each base station device 300 into groups based on the feature amount of each base station device 300.

[0036] The model 60 is the result (learning model) of machine learning of the feature quantities in the learning device 50. By applying this model 60 to the fault detection unit 51 of the fault detection device 200 and inputting the feature quantities of the base station device 300 that is the estimation target, the fault detection device 200 can determine whether the base station device 300 that is the estimation target is normal or not with the performance of the learning model of the model 60 improved.

[0037] The fault detection device 200 is configured to include observation units 11, 12, . . . , 19, an estimation database 21, a preprocessing unit 41 (estimation preprocessing unit), a fault detection unit 51, and a model 60.

[0038] The estimation database 21 has the same configuration as the training database 20 .

[0039] The preprocessing unit 41 has the same configuration as the preprocessing unit 40. The preprocessing unit 41 performs scaling on the feature quantities of the base station device 300 to be estimated, for each base station device 300, the date and time when the feature quantities were collected, and for each feature quantity.

[0040] The failure detection unit 51 inputs the feature amounts of the base station device 300 to be estimated, which have been scaled by the preprocessing unit 41, into the model 60, and estimates whether the base station device 300 to be estimated is normal. For example, the preprocessing unit 41 standardizes the communication volume of the base station device 300 to be estimated, and inputs the standardization as preprocessing to the machine-learned model 60, thereby enabling the failure detection unit 51 to highly accurately estimate whether each base station device 300 is normal or not for each time and output the estimation result 70.

[0041] 3 is an explanatory diagram showing a schematic hardware configuration of the learning model generation device 100 according to this embodiment. The learning model generation device 100 is configured as, for example, a server.

[0042] The learning model generation device 100 is configured to include a CPU (Central Processing Unit) 101, a memory unit 102, a ROM (Read Only Memory) 103, a RAM (Random Access Memory) 104, an operation unit 105, a display unit 106, a communication unit 107, a learning database 20, a learning device 50, and a model 60.

[0043] The CPU 101 reads and executes a program (e.g., a learning model generation program) from the storage unit 102 or the ROM 103, thereby embodying the observation unit 10, the label assignment unit 30, and the preprocessing unit 40 shown in FIG. 1 and controlling the overall operation of the learning model generation device 100. At this time, various data stored in the storage unit 102 is referenced. The storage unit 102 is also configured by a large-capacity storage medium such as a semiconductor memory or a hard disk drive, and embodies the learning database 20.

[0044] The CPU 101 transmits and receives various data to and from external devices connected to a communication network such as a local area network (LAN) or a wide area network (WAN) via the communication unit 107. The CPU 101, for example, collects communication traffic from each base station device 300 using the observation unit 10 and stores the communication traffic in the learning database 20 (see FIG. 1 ).

[0045] The communication unit 107 is configured by a communication control card such as a network interface card (NIC).

[0046] The operation unit 105 includes various operation keys such as a numeric keypad, a start key, etc. The operation unit 105 accepts various input operations by the user and outputs operation signals to the CPU 101.

[0047] The display unit 106 is, for example, a liquid crystal display, and displays characters, figures, images, and the like.

[0048] 4 is an explanatory diagram showing a schematic hardware configuration of the failure detection device 200 according to this embodiment. The failure detection device 200 is configured by, for example, an edge terminal.

[0049] The fault detection device 200 includes a CPU 201 , a storage unit 202 , a ROM 203 , a RAM 204 , and a model 60 .

[0050] The CPU 201 reads and executes a program (e.g., a fault detection program) from the storage unit 202 or the ROM 203, thereby embodying the observation unit 10, the preprocessing unit 41, and the fault detection unit 51, and controlling the overall operation of the fault detection device 200. The model 60 of the learning model generation device 100 shown in FIGS. 1 and 3 is applied to the model 60 of the fault detection device 200, and is stored, for example, in the storage unit 202. The storage unit 202 is configured, for example, by a large-capacity storage medium such as a semiconductor memory or a hard disk drive, and embodies the estimation database 21.

[0051] <Operation of each device in the failure detection system> [During machine learning by the learning model generation device] The processing of the learning model generation device 100 according to this embodiment will be described. Fig. 5 is a flowchart showing the processing by which the learning model generation device 100 according to this embodiment performs machine learning to generate a model 60, which is a learning model.

[0052] First, the learning model generation device 100 of the fault detection system 250 collects features from each of the base station devices 301, 302, ..., 309 (step S10). Each of the observation units 11, 12, ..., 19 of the learning model generation device 100 collects and observes the features of each of the corresponding base station devices 300. Then, each of the observation units 11, 12, ..., 19 stores the collected features in the learning database 20 (step S11).

[0053] Next, the labeling unit 30 assigns a label indicating whether the feature held in the learning database 20 is normal or not (step S12). Note that if the learning model generation device 100 does not include the labeling unit 30 and the learning device 50 performs unsupervised learning, this labeling is not performed.

[0054] The preprocessing unit 40 scales the collected feature quantities as learning data for creating a learning model for each communication device, the date and time when the feature quantities were collected, and each feature quantity (step S13). The preprocessing unit 40 performs normalization or standardization as scaling for the collected feature quantities. In addition, any information such as the communication volume, fluctuation, and CPU usage rate of the base station device 300 can be applied to the feature quantities.

[0055] The learning device 50 uses the scaled learning data to perform machine learning for detecting a fault in the base station device 300, and generates a model 60 (learning model) (step S14). The learning device 50 performs machine learning on, for example, the standardized communication volume. As a result, the learning device 50 can generate the model 60, which is the result of the machine learning.

[0056] [During Estimation of Fault Detection Device] The processing of the fault detection device 200 according to this embodiment will be described below. Fig. 6 is a flowchart showing the processing of the fault detection device 200 according to this embodiment, which collects feature quantities of each base station device 300 and estimates whether the base station device 300 is normal or not based on the feature quantities.

[0057] The fault detection device 200 of the fault detection system 250 collects features from each of the base station devices 301, 302, ..., 309 (step S20). Each observation unit 11, 12, ..., 19 of the fault detection device 200 collects and observes the feature of each corresponding base station device 300. Then, each observation unit 11, 12, ..., 19 stores the collected feature in the estimation database 21 (step S21).

[0058] In this embodiment, the fault detection device 200 is configured to estimate whether the base station device 300 is operating normally, and the base station device 300 that is the subject of this estimation is not limited to the base station device 300 whose features are collected by the learning model generation device 100. In other words, the fault detection device 200 can estimate whether the base station device 300 is operating normally even using features of the base station device 300 that are not collected by the learning model generation device 100.

[0059] The preprocessing unit 41 performs scaling on the feature quantities of the base station device 300 to be estimated, taking into account the base station device 300, the date and time when the feature quantities were collected, and each feature quantity (step S22). As with the preprocessing unit 40, the preprocessing unit 41 performs normalization or standardization as scaling on the collected feature quantities. In addition, any information such as the communication volume, fluctuation, and CPU usage rate of the base station device 300 can be applied to the feature quantities.

[0060] The fault detection unit 51 inputs the feature quantities of the base station device 300 to be estimated, which have been scaled by the preprocessing unit 41, into the model 60, and estimates whether the base station device 300 to be estimated is normal or not (step S23). For example, by inputting the feature quantities of the base station device 300 to be estimated, which have been scaled, into the model 60 that has been machine-learned by standardizing it as preprocessing, the fault detection unit 51 can estimate with high accuracy whether each base station device 300 is normal or not for each time and output the estimation result 70.

[0061] <Operations and Effects> The effects of the learning model generation device 100 and the fault detection device 200 of the fault detection system 250 according to the present invention will be described below.

[0062] The learning model generation device 100 according to the present invention is characterized by comprising an observation unit 10 that collects and observes features of one or more base station devices 300, a pre-processing unit 40 that scales the collected features as learning data for creating a learning model for the base station device 300, the date and time the features were collected, and each feature, and a learning device 50 that uses the scaled learning data to perform machine learning to detect faults in the base station device 300 and generate a learning model.

[0063] The learning model generation device 100 according to the present invention scales the collected features as learning data using the preprocessing unit 40, based on the base station device 300, the date and time when the features were collected, and for each feature. The learning model generation device 100 also uses the scaled learning data using the learning unit 50 to perform machine learning for detecting faults in the base station device 300, thereby generating a model 60.

[0064] Therefore, the learning model generation device 100 according to the present invention can reduce the differences in the characteristics of features for each base station device 300 and each time period by performing machine learning using scaled learning data.

[0065] As a result, the learning model generation device 100 according to the present invention can improve the accuracy of the model 60, thereby improving the accuracy of detecting faults in the base station device 300 and reducing false positives.

[0066] Furthermore, the learning model generation device 100 according to the present invention is characterized in that the collected and observed feature quantities of the base station device 300 include the communication volume of the base station device 300.

[0067] According to the learning model generation device 100 of the present invention, by including communication volume in the feature quantities of the base station device 300, the accuracy of detecting a failure in the base station device 300 can be further improved.

[0068] The pre-processing unit 40 of the learning model generating device 100 according to the present invention is characterized in that it performs normalization or standardization as scaling of the collected feature quantities.

[0069] The learning model generation device 100 of the present invention can reduce differences in the characteristics of features for each base station device 300 and each time period by normalizing or standardizing the features for each base station device 300 and each time period during which they are collected.

[0070] As a result, the learning model generation device 100 according to the present invention can improve the accuracy of detecting failures in the base station device 300 by reducing differences in the characteristics of the features through normalization or standardization.

[0071] The fault detection device 200 according to the present invention is characterized by comprising: a model 60 that is generated to detect a fault in the base station device 300 by performing machine learning on the feature quantities of the base station device 300 using the base station device 300, the date and time the feature quantities were collected, and learning data scaled for each feature quantity; a pre-processing unit 41 that performs scaling on the feature quantities of the base station device 300 that is the subject of estimation, using the base station device 300, the date and time the feature quantities were collected, and each feature quantity; and a fault detection unit 51 that inputs the feature quantities of the base station device 300 that is the subject of estimation that have been scaled by the pre-processing unit 41 into the model 60 and estimates whether the base station device 300 that is the subject of estimation is normal or not.

[0072] The fault detection device 200 according to the present invention performs scaling of the feature quantities of the base station device 300 to be estimated, for each base station device 300, the date and time when the feature quantities were collected, and for each feature quantity, by the pre-processing unit 41. Furthermore, the fault detection device 200 inputs the feature quantities of the base station device 300 to be estimated that have been scaled by the pre-processing unit 41 into the model 60, and estimates whether the base station device 300 to be estimated is normal.

[0073] As a result, the failure detection device 200 according to the present invention can improve the accuracy of detecting a failure in the base station device 300 that is the estimation target, and can reduce false detections.

[0074] The present invention is not limited to the above-described embodiments, and many modifications can be made by a person skilled in the art within the technical concept of the present invention.

[0075] 10, 11, 12, 19 Observation unit 20 Learning database 21 Estimation database 30 Label assignment unit 40 Preprocessing unit 41 Preprocessing unit (estimation preprocessing unit) 50 Learning device 51 Fault detection unit 60 Model (learning model) 70 Estimation result 100 Learning model generation device 101, 201 CPU 102, 202 Storage unit 103, 203 ROM 104, 204 RAM 105 Operation unit 106 Display unit 107 Communication unit 200 Fault detection device 300, 301, 302, 309 Base station device (communication device)

Claims

1. A learning model generation device comprising: an observation unit that collects and observes features of one or more communication devices; a pre-processing unit that scales the collected features as learning data for creating a learning model, for the communication device, the date and time when the features were collected, and each of the features; and a learning device that uses the scaled learning data to perform machine learning for detecting failures in the communication devices and generate a learning model.

2. The learning model generating device according to claim 1, characterized in that the feature amount includes communication volume of a base station device.

3. The learning model generating device according to claim 1, characterized in that the preprocessing unit performs normalization or standardization as scaling of the collected features.

4. A fault detection device comprising: a learning model generated for detecting a fault in a communication device by performing machine learning on features of the communication device using learning data scaled for the communication device, the date and time when the feature was collected, and each of the feature; an estimation pre-processing unit that performs scaling on the features of a communication device to be estimated, for the communication device, the date and time when the feature was collected, and each of the feature; and a fault detection unit that inputs the features of the communication device to be estimated scaled by the estimation pre-processing unit into the learning model and estimates whether the communication device to be estimated is normal or not.

5. A learning model generation method of a learning model generation device, wherein the learning model generation device executes the steps of: collecting features of one or more communication devices; scaling the collected features as learning data for creating a learning model for the communication device, the date and time when the features were collected, and each of the features; and performing machine learning using the scaled learning data to detect failures in the communication devices and generate a learning model.

6. A fault detection method for a fault detection device, wherein the fault detection device executes the steps of: scaling features of a communication device to be estimated for the communication device, the date and time when the feature was collected, and each of the feature; and performing machine learning on the features of the communication device using learning data obtained by scaling the communication device, the date and time when the feature was collected, and each of the feature, inputting the scaled features of the communication device to be estimated into a learning model generated for detecting a fault in the communication device, and estimating whether the communication device to be estimated is normal or not.

7. A learning model generation program for causing a computer to execute the steps of: collecting features of one or more communication devices; scaling the collected features as learning data for creating a learning model, for the communication device, the date and time when the features were collected, and each of the features; and performing machine learning using the scaled learning data to detect failures in the communication devices, thereby generating a learning model.

8. A failure detection program for causing a computer to execute the steps of: scaling features of a communication device to be estimated, for the communication device, the date and time when the features were collected, and for each of the features; performing machine learning on the features of the communication device using learning data obtained by scaling the communication device, the date and time when the features were collected, and for each of the features, inputting the scaled features of the communication device to be estimated into a learning model generated for detecting a failure of the communication device, and estimating whether the communication device to be estimated is normal or not.