Data analysis processing system, terminal device, server device, data analysis processing method, learned model acquisition method, and program

Abstract

To perform abnormality detection and identification of abnormality causes through real-time processing without mounting a large-scale AI model on a terminal device in a system including the terminal device that acquires data and a server device that collects the data acquired by the terminal device via a communication network.SOLUTION: In a data analysis processing system, a terminal device transmits data and / or input data acquired by a state determination processing unit that determines whether input data is normal or abnormal, and a feature analysis processing unit that acquires feature amount data of the input data to a server device, and the server device performs state analysis processing. In other words, the data analysis processing system allows the terminal device to only perform processing by the state determination processing unit and the feature analysis processing unit, determination result data indicating normal / abnormal and feature amount data of the input data (data to be processed) can be acquired through real-time processing without mounting a large-scale AI model on the terminal device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for detecting an anomaly in data and identifying the cause of the anomaly. [Background technology]

[0002] In recent years, technology has been developed that uses an image generation AI model to detect anomalies in data (see, for example, Patent Document 1). For example, Patent Document 1 discloses an image processing device that uses images to determine the state of an object. The image processing device is mounted on a mobile object and includes a camera that captures captured images of the object, an image generation unit that uses an image generation model that has learned normal states to generate a normal state image that shows a normal state based on the captured image captured by the camera, and a determination unit that determines the state of the object. The determination unit calculates the similarity between the captured image and the normal state image, and determines that the object is abnormal if the calculated similarity is small. This makes it possible to detect anomalies in data (objects). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-133306 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology disclosed in Patent Document 1, the size of the image generation AI model is large, making it difficult to install it on a terminal device (e.g., an edge terminal). Furthermore, the technology disclosed in Patent Document 1 requires processing using the image generation AI model, which makes it necessary to use a GPU, and even if a GPU is used, it is difficult to perform real-time processing. Furthermore, while the technology disclosed in Patent Document 1 can determine whether something is normal or abnormal, it cannot identify the cause of the abnormality when it occurs.

[0005] In view of the above, the present invention aims to provide a data analysis processing system, a terminal device, a server device, a data analysis processing method, and a program that can detect anomalies and identify the causes of anomalies through real-time processing in a system including a terminal device that acquires data and a server device that collects the data acquired by the terminal device via a communication network, without installing a large-scale AI model in the terminal device. The present invention also aims to provide a method (trained model acquisition method) and a program for installing trained models separately in the terminal device and the server device to ensure the security of the data analysis processing system. [Means for solving the problem]

[0006] In order to solve the above problem, a representative example (one aspect) of the invention disclosed in this application is a data analysis processing system that performs analytical processing on data that can distinguish between normal and abnormal states, and includes a terminal device and a server device.

[0007] The terminal device includes a state determination processing unit, a feature analysis processing unit, a data integration processing unit, and a first communication interface unit.

[0008] The state determination processing unit determines the state of the data to be processed, and obtains the result data of the determination as state determination data.

[0009] The feature analysis processing unit acquires feature amount data of the processing data from the processing target data.

[0010] The data integration processing unit acquires data including the state determination data and the feature amount data as integrated data. The first communication interface is a communication interface for transmitting communication data including the integrated data to the server device.

[0011] The server device includes a second communication interface unit, a cause identification data acquisition processing unit, and a status analysis processing unit.

[0012] The second communication interface unit is a communication interface for receiving communication data transmitted from a terminal device.

[0013] The factor identification data acquisition processing unit inputs feature data included in the communication data acquired by the second communication interface unit, and acquires factor identification data, which is data for identifying the cause of the occurrence of the abnormal state, from the feature data.

[0014] The status analysis processing unit performs status analysis processing based on the status determination data and the cause identification data included in the communication data acquired by the second communication interface unit. [Effects of the Invention]

[0015] According to the present invention, in a system including a terminal device that acquires data and a server device that collects the data acquired by the terminal device via a communication network, it is possible to realize a data analysis processing system, a terminal device, a server device, a data analysis processing method, and a program that can detect anomalies and identify the causes of anomalies through real-time processing without installing a large-scale AI model in the terminal device.Furthermore, according to the present invention, it is possible to realize a method (trained model acquisition method) and a program for installing trained models separately in the terminal device and the server device to ensure the security of the data analysis processing system. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic configuration diagram of a data analysis processing system 1000 according to a first embodiment. [Figure 2] FIG. 2 is a schematic configuration diagram of a cause identification data acquisition processing unit 22 of the server device 200 of the data analysis processing system 1000 according to the first embodiment. [Figure 3] 1 is a flowchart of a process (process for acquiring a trained model) executed by the data analysis processing system 1000. [Figure 4]1 is a flowchart of a process (process for acquiring a trained model) executed by the data analysis processing system 1000. [Figure 5] 1 is a flowchart of a process (process for acquiring a trained model) executed by the data analysis processing system 1000. [Figure 6] 1 is a flowchart of a process (process for acquiring a trained model) executed by the data analysis processing system 1000. [Figure 7] FIG. 1 is a diagram for explaining the learning process of a trainable model (a model for a two-class classification task) installed in the state determination processing unit 11, and the inference process (processing using a trained model). [Figure 8] FIG. 1 is a diagram for explaining the learning process of a multi-(N+1) class classification learning model. [Figure 9] A diagram for explaining a trained model of a multi-(N+1) class classification learning model and a trained model of feature analysis. [Figure 10] FIG. 10 is a diagram for explaining the learning process of a two-class classification learning model for factor identification (learning model for data acquisition process for i-th factor identification process). [Figure 11] FIG. 10 is a schematic configuration diagram of a data analysis processing system 2000 according to a second embodiment. [Figure 12] 10 is a flowchart of an additional learning process executed by the data analysis processing system 2000. [Figure 13] 10 is a flowchart of an additional learning process executed by the data analysis processing system 2000. [Figure 14] 10 is a flowchart of an additional learning process executed by the data analysis processing system 2000. [Figure 15] 10 is a flowchart of an additional learning process executed by the data analysis processing system 2000. [Figure 16] FIG. 10 is a schematic configuration diagram of a data analysis processing system 2000A according to a first modified example of the second embodiment. [Figure 17]A flowchart of a process in which a first divided trained model, which is a trained model obtained by training a first trainable model that is a trainable model having a feature extraction unit and a class classification unit, is installed in a first device (e.g., a terminal device), and a class classification trained model, which is a trained model trained using training data obtained by the first divided trained model, is installed in a second device (e.g., a server device). [Figure 18] A flowchart of a process in which a first divided trained model, which is a trained model obtained by training a first trainable model that is a trainable model having a feature extraction unit and a class classification unit, is installed on a first device (e.g., a terminal device), and a second divided trained model, which is a trained model obtained by extracting the remaining part of the first divided trained model, is installed on a second device (e.g., a server device). [Figure 19] A diagram showing the CPU bus configuration. DETAILED DESCRIPTION OF THE INVENTION

[0017] [First embodiment] The first embodiment will be described below with reference to the drawings.

[0018] <1.1: Data analysis processing system configuration> FIG. 1 is a schematic configuration diagram of a data analysis processing system 1000 according to the first embodiment.

[0019] FIG. 2 is a schematic configuration diagram of the cause identification data acquisition processing unit 22 of the server device 200 of the data analysis processing system 1000 according to the first embodiment.

[0020] 1, the data analysis processing system 1000 includes a terminal device 100, a server device 200, and a data storage unit DB1. The terminal device 100 and the server device 200 are connected to each other via a communication network so that they can communicate with each other. The data storage unit may also be located within the server device.

[0021] As shown in FIG. 1, the terminal device 100 includes a state determination processing unit 11, a feature analysis processing unit 12, a data integration processing unit 13, and a first communication interface .

[0022] The state determination processing unit 11 is a functional unit including a trained model (a trained model that performs a two-class classification task) acquired by machine learning, and receives data Din (for example, data that can identify a normal state / abnormal state). The state determination processing unit 11 inputs the data Din to the trained model (a trained model that performs a two-class classification task) and acquires data (state determination processing data) output from the trained model. The state determination processing unit 11 then outputs the data (state determination processing data) output from the trained model to the data integration processing unit 13 as data D11.

[0023] The feature analysis processing unit 12 is a functional unit that includes a trained model acquired by machine learning (a trained model that outputs feature data (e.g., a feature vector)), and receives data Din (e.g., data that can identify a normal state / abnormal state). The feature analysis processing unit 12 inputs the data Din to the trained model, and outputs the feature data (e.g., a feature vector) output from the trained model to the data integration processing unit 13 as data D12.

[0024] The data integration processing unit 13 receives data D11 output from the state determination processing unit 11 and data D12 output from the feature analysis processing unit 12. The data integration processing unit 13 performs data integration processing on the data D11 and the data D12, and outputs the processed data to the first communication interface 14 as data D2.

[0025] The first communication interface 14 is a communication interface for transmitting and receiving data to and from an external device (for example, the server device 200) via a wired and / or wireless communication network. The first communication interface 14 also receives data D2 output from the data integration processing unit 13, generates communication data Dtx(D2) including the data D2 (communication data (communication signal) in a format that can be transmitted to an external device via a wired and / or wireless communication network), and transmits the generated communication data Dtx(D2) to the server device 200.

[0026] As shown in FIG. 1, the server device 200 includes a second communication interface 21, a cause identification data acquisition processing unit 22, and a status analysis processing unit .

[0027] The second communication interface 21 is a communication interface for transmitting and receiving data to and from an external device (e.g., the terminal device 100) via a wired and / or wireless communication network. Furthermore, as shown in FIG. 1 , the second communication interface 21 is connected to a cause identification data acquisition processing unit 22 and a status analysis processing unit 23. The second communication interface 21 receives communication data transmitted from an external device (e.g., the terminal device 100). For example, the second communication interface 21 receives communication data Dtx(D2) transmitted from the terminal device 100, and acquires data D31 and / or D32 included in the communication data Dtx(D2). The second communication interface 21 then outputs the acquired data D31 to the status analysis processing unit 23. Furthermore, the second communication interface 21 outputs the acquired data D32 to the cause identification data acquisition processing unit 22.

[0028] As shown in FIG. 2, the factor identification data acquisition processing unit 22 includes N (N: natural number) trained models for factor identification data acquisition processing, ie, a first trained model for factor identification data acquisition processing to an Nth trained model for factor identification data acquisition processing. As shown in FIG. 2, the trained model for data acquisition processing for identifying the kth factor (k: natural number, 1≦k≦N) inputs data D32 (feature amount data) output from the second communication interface 21, processes the D32 (feature amount data) using the trained model for data acquisition processing for identifying the kth factor, and acquires the processed data as data D42_k. Then, the trained model for data acquisition processing for identifying the kth factor outputs the acquired data D42_k to the state analysis processing unit 23. Note that data D42_1 to D42_N acquired by the trained model for data acquisition processing for identifying the first factor to the trained model for data acquisition processing for identifying the Nth factor are collectively referred to as data D42.

[0029] The status analysis processing unit 23 receives data D31 output from the second communication interface 21 and data D42 output from the cause identification data acquisition processing unit 22. The status analysis processing unit 23 determines whether the status of the data Din acquired by the terminal device 100 is normal or abnormal based on the data D31, and if it determines that the status is abnormal (if the data D31 is data indicating an abnormal status), performs processing to identify the cause of the abnormality based on the data D42. The status analysis processing unit 23 also performs processing to identify the cause of the abnormality based on the data D42 output from the cause identification data acquisition processing unit 22, thereby performing status analysis processing, and acquires data indicating the results of the status analysis processing as data Dout (outputting the data Dout to the outside, for example).

[0030] The data storage unit DB1 is a functional unit capable of storing data, and reads predetermined data from a predetermined storage area in response to an external read command, and / or writes predetermined data to a predetermined storage area in response to an external write command. In response to a data write command from the status analysis processing unit 23, the data storage unit DB1 writes data specified by the data write command to a predetermined storage area. In response to a data read command from the status analysis processing unit 23, the data storage unit DB1 reads data from the storage area specified by the data read command, and outputs the read data to the status analysis processing unit 23.

[0031] <1.2: Operation of the data analysis processing system> The operation of the data analysis processing system 1000 configured as above will now be described.

[0032] 3 to 6 are flowcharts of the process (process for acquiring a trained model) executed by the data analysis processing system 1000.

[0033] FIG. 7 is a diagram for explaining the learning process of a trainable model (a model for a two-class classification task) installed in the state determination processing unit 11, and the inference process (processing using a trained model).

[0034] FIG. 8 is a diagram for explaining the learning process of the multi-(N+1) class classification learning model.

[0035] FIG. 9 is a diagram illustrating a trained model and a trained feature analysis model of a multi-(N+1) class classification training model.

[0036] FIG. 10 is a diagram for explaining the learning process of the factor identification two-class classification learning model (learning model for the process of acquiring data for the i-th factor identification process).

[0037] For ease of explanation, the operation of the data analysis processing system 1000 will be described below for the case where the data to be processed (data that can be considered as normal / abnormal data) in the data analysis processing system 1000 is image data of a manufactured product (as an example).

[0038] (1.2.1: Acquisition process of trained model (training process)) First, we will explain the process of acquiring a trained model executed by the data analysis processing system 1000. This process will be explained with reference to the flowcharts of FIGS.

[0039] (Step S1): In step S1, an acquisition process (learning process) is executed for a trainable model (a model for a two-class classification task) to be installed in the state determination processing unit 11 of the terminal device 100. Specifically, the following process is executed.

[0040] (Step S11): In step S11, a process of acquiring data in a normal state is executed. For example, an imaging device (not shown) is used to acquire data in a normal state (image data of a manufactured product in a normal state), and the acquired data is stored in the data storage unit DB1.

[0041] (Step S12): In step S12, a learning process for a two-class classification learning model (learnable model) is executed. Specifically, as shown in FIG. 7, normal state data (image data of a manufactured product in a normal state) stored in the data storage unit DB1 is input to the two-class classification learning model, and the output of the two-class classification learning model, i.e., classification result data, is obtained. The two-class classification learning model has two output nodes in the output layer, with the first output node being a node that outputs the probability that the input data is normal and the second output node being a node that outputs the probability that the input data is abnormal. Then, an error (e.g., cross-entropy error) between the data of the output node (two-dimensional vector) and the correct label (a two-dimensional vector, a vector (one-hot vector) in which the first-dimensional data (probability of being normal) is 1.0 and the second-dimensional data (probability of being abnormal) is 0.0) is obtained. Then, the parameters of the two-class classification learning model are updated using the obtained error, for example, by backpropagation. This process is repeated until the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) and the correct label becomes equal to or less than a predetermined value, or until the variation of the error falls within a predetermined range, and the learning is converged. When it is determined that the learning has converged, the parameters set in the two-class classification trainable model are acquired as the optimal parameters.

[0042] (Step S13): In step S13, a two-class classification trained model (a two-class classification model in which the optimal parameters acquired in step S12 are set) is installed in the state determination processing unit 11 of the terminal device 100. Specifically, the parameters of the two-class classification trained model of the state determination processing unit 11 of the terminal device 100 are set to the optimal parameters acquired in step S12. As a result, as shown in the lower diagram of FIG. 7, when image data (input data) of a manufactured product is input to the above-mentioned two-class classification trained model, normal / abnormal determination result data of the input image data of the manufactured product (indicating the probability that the value of the first node of the output node is normal) is output from the two-class classification trained model.

[0043] (Step S2): In step S2, a process for acquiring a trainable model (learning process) is executed to be installed in the feature analysis processing unit 12 of the terminal device 100. Specifically, the following process is executed.

[0044] (Step S21): In step S21, a process of acquiring data of a normal state and data of an abnormal state is executed. For example, an imaging device (not shown) is used to acquire data of a normal state (image data of a manufactured product in a normal state), and the acquired data is stored in the data storage unit DB1 together with a label indicating that the data is data of a normal state (this will be referred to as "state #0"). Also, an imaging device (not shown) is used to acquire data of an abnormal state (image data of a manufactured product in an abnormal state), and the acquired data is stored in the data storage unit DB1 together with a label indicating that the data is data of the i-th abnormal state (this will be referred to as "state #i" (i: natural number, N: natural number, 1≦i≦N)).

[0045] For example, state #0 and state #i are set as follows: #0: Normal #1: Color abnormality #2: Abnormal shape ... #N: Size Abnormal For ease of explanation, the following description will be given for the case where state #0 and state #i are set as described above (one example).

[0046] (Step S22): In step S22, loop 1 processing is started. Note that loop 1 processing is executed for each state from state #0 to state #N. Loop 1 processing is executed for state #k (each state from state #0 to #N) in order by incrementing variable k by +1.

[0047] (Step S23): In step S23, data of state #k (k: integer, 0≦k≦N) is acquired from the data storage unit DB1. It is preferable that the data of state #k acquired in step S23 is a plurality of pieces of data.

[0048] (Step S24): In step S24, a learning process for a multi-class (N+1 classes) classification learning model (feature extraction unit + class classification unit) is executed. As shown in Fig. 8, the multi-class (N+1 classes) classification learning model is composed of a feature extraction unit that extracts features from input data and a class classification unit that performs class classification based on the features acquired by the feature extraction unit.

[0049] The feature extraction unit of the multi-class (N+1 class) classification learning model is a trainable model having one or more convolutional layers, and receives input data (data corresponding to data Din) and inputs the input data to the convolutional layer to obtain feature data (e.g., a feature vector) that is the output of the convolutional layer.The feature extraction unit then outputs the obtained feature data (e.g., a feature vector) to the class classification unit.

[0050] The classification unit of the multi-class (N+1 class) classification learning model includes one or more fully connected layers and a softmax layer. The classification unit inputs feature data (e.g., feature vectors) output from the feature extraction unit of the multi-class (N+1 class) classification learning model. Specifically, the classification unit inputs the feature data to the one or more fully connected layers, and inputs the output from the final fully connected layer to the softmax layer. The classification unit then obtains the output (output distribution data) of the softmax layer. Note that since the classification unit performs N+1 class classification, the softmax layer has N+1 nodes (output nodes), and the N+1 i-th node (i: integer, 0≦i≦N) of the softmax layer is a node that outputs the probability of state #i.

[0051] Specifically, the learning process of the multi-class (N+1 class) classification learning model (feature extractor+classification unit) in step S24 is executed as follows.

[0052] The data of state #k (k: integer, 0≦k≦N) acquired in step S23 is input to a multi-class (N+1 class) classification learning model. Then, processing by the feature extraction unit of the multi-class (N+1 class) classification learning model (feature extraction processing) and processing by the classification unit (N+1 class classification processing) are executed, and output distribution data (N+1-dimensional vector) that is the output of the softmax layer of the classification unit is acquired.

[0053] Then, the error (e.g., cross-entropy error) between the data (N+1-dimensional vector) of the output node of the softmax layer of the classification unit and the correct label (an N+1-dimensional vector in which only the element (data) of the mth dimension (m: natural number, 1≦m≦N+1, m=k+1) (data corresponding to the probability of being in state #k (k: integer, 0≦k≦N)) is "1" and all elements other than the mth dimension element are "0") is obtained (a vector (one-hot vector). The obtained error is then used, for example, by backpropagation, to update the parameters of the multi-class (N+1 class) classification learning model (the parameters of the learnable model (convolutional layer, etc.) of the classification unit and the parameters of the learnable model (fully connected layer, etc.) of the feature extraction unit).

[0054] (Step S25): In step S25, a determination process is performed to determine whether the end condition of loop 1 processing is satisfied. Specifically, if it is determined that loop 1 processing has been executed for all states #0 to #N, the end condition of loop 1 processing is satisfied, so loop 1 processing is terminated and processing proceeds to step S26. On the other hand, if it is determined that loop 1 processing has not been executed for all states #0 to #N, processing returns to step S22, and loop 1 processing (steps S22 to S25) is executed.

[0055] (Step S26): In step S26, a determination process is performed to determine whether the learning process of the trainable model installed in the feature analysis processing unit 12 of the terminal device 100 has converged.If the result of the determination process is that the learning process has not converged, the process returns to step S22; on the other hand, if it is determined that the learning process has converged, the process proceeds to step S27.

[0056] The process of determining whether the learning process of the trainable model installed in the feature analysis processing unit 12 of the terminal device 100 has converged is, for example, determined to have converged in the following cases. (1) When the error (e.g., cross-entropy error) between the output node data (N+1-dimensional vector) of a multi-class (N+1 class) classification learning model and the correct label falls below a predetermined value. (2) When the variation in the error (e.g., cross-entropy error) between the output node data (N+1-dimensional vector) of a multi-class (N+1 class) classification learning model and the correct label falls within a specified range.

[0057] (Step S27): In step S27, a process is executed to load the feature analysis trained model into the feature analysis processing unit 12 of the terminal device 100. Specifically, the following process is executed.

[0058] In step S26, when it is determined that the learning process has ended (converged), the parameters set in the multi-class (N+1 class) classification learning model are acquired as optimal parameters.The multi-class (N+1 class) classification learning model in which the optimal parameters are set is acquired as a multi-class (N+1 class) classification trained model.The classifier unit is then deleted from the multi-class (N+1 class) classification trained model, and a model in which only the convolution processing unit is extracted is acquired as a feature analysis trained model (see FIG. 9).The acquired feature analysis trained model is then installed in the feature analysis processing unit 12 of the terminal device 100 (the convolution processing unit (the convolution processing unit in which the optimal parameters are set) of the multi-class (N+1 class) classification trained model is set as the feature analysis trained model, and the feature analysis trained model is installed in the feature analysis processing unit 12).

[0059] When input data is input to the multi-class (N+1 class) classification trained model acquired by the above learning process, it can appropriately determine that the input data is data for state #k (state #0: normal state, states #1 to #N: N different abnormal states) (it can determine which of the N+1 states the input data is in (it can classify into N+1 classes)). When input data is input to the feature analysis trained model extracted from only the convolution processing unit of the multi-class (N+1 class) classification trained model, it can output feature data (e.g., feature vector) for state #k if the input data is data for state #k (state #0: normal state, states #1 to #N: N different abnormal states).

[0060] (Step S3): In step S3, an acquisition process (learning process) of a trainable model is executed to be installed in the factor identification data acquisition processing unit 22 of the server device 200. Specifically, the following process is executed.

[0061] (Step S31): In step S31, input data (data of abnormal states #1 to #N) for state #i (1≦i≦N) is input to the feature analysis trained model acquired in step S2, and a process is executed to acquire the output feature data (e.g., feature vector) for state #i. Specifically, as shown in the upper diagram of FIG. 10, input data (data (e.g., image data) for state #i (1≦i≦N)) is input to the feature analysis trained model acquired in step S2, and the output feature data (e.g., feature vector) for state #i is acquired. The acquired feature data (e.g., feature vector) for state #i is stored, for example, in data storage unit DB1.

[0062] (Step S32): In step S32, loop 2 processing is started. Note that loop 2 processing is executed for each state (each abnormal state) from state #1 to state #N. Loop 2 processing is executed for state #i (each state from state #1 to state #N) in order by incrementing variable i (i: natural number, 1≦i≦N) by +1.

[0063] (Step S33): In step S33, data (data of an abnormal state) of state #i (i: natural number, 1≦i≦N) is acquired from the data storage unit DB1. Note that it is preferable that the data of state #i acquired in step S33 is a plurality of pieces of data.

[0064] (Step S34): In step S34, learning processing of the factor identification two-class classification learning model (learning model for data acquisition processing for i-th factor identification processing) is executed. Specifically, as shown in FIG. 10, data of state #i (i: natural number, 1≦i≦N) (abnormal state) stored in data storage unit DB1 (e.g., image data of a manufactured product in an abnormal state) is input to the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model), and the output of the learning model for data acquisition processing for i-th factor identification processing, i.e., classification result data, is obtained. Note that the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) has two output nodes in the output layer, with the first output node being a node that outputs the probability that the input data is normal and the second output node being a node that outputs the probability that the input data is abnormal. Then, the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) and the correct label (a two-dimensional vector in which the first-dimensional data (probability of being normal) is 0.0 and the second-dimensional data (probability of being abnormal) is 1.0 (vector (one-hot vector)) is obtained. Then, using the obtained error, for example, by error backpropagation, the parameters of the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) are updated. This process is repeated until the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) of the learning model for data acquisition processing for i-th factor identification processing and the correct label becomes equal to or less than a predetermined value, or until the amount of variation in the error falls within a predetermined range, thereby converging the learning. Then, when it is determined that the learning has converged, the parameters set in the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) are acquired as optimal parameters of the learning model for data acquisition processing for i-th factor identification processing.

[0065] (Step S35): In step S35, a determination process is performed to determine whether the termination condition for loop 2 processing is met. Specifically, if it is determined that loop 2 processing has been executed for all of states #1 to #N, the termination condition for loop 2 processing is met, so loop 2 processing is terminated and processing proceeds to step S36. On the other hand, if it is determined that loop 1 processing has not been executed for all of states #1 to #N, processing returns to step S32, and loop 2 processing (steps S32 to S35) is executed.

[0066] (Step S36): In step S36, a process is executed to install a factor identification two-class classification learned model (a learned model for the i-th factor identification data acquisition process) in the factor identification data acquisition processing unit 22 of the server device 200. Specifically, N factor identification two-class classification learned models, that is, the first factor identification data acquisition process learned model to the N-th factor identification data acquisition process learned model (a learned model for the i-th factor identification data acquisition process in which optimal parameters are set), are installed in the factor identification data acquisition processing unit 22 of the server device 200. As a result of the above, the process of acquiring the trainable model (training process) to be installed in the feature analysis processing unit 12 of the terminal device 100 is executed.

[0067] (1.2.3: Specific Operation of Data Analysis Processing System 1000) Next, the specific operation of the data analysis processing system 1000 will be described.

[0068] At time t (time step t), for example, image data of a manufactured product captured by an imaging device (not shown) is input to the terminal device 100 as data Din.

[0069] The state determination processing unit 11 includes (is equipped with) a trained model (a trained model that performs a two-class classification task) acquired by machine learning, inputs data Din to the trained model (a trained model that performs a two-class classification task), and acquires data (state determination processing data (normal / abnormal determination result data)) output from the trained model.The state determination processing unit 11 then outputs the data (state determination processing data) output from the trained model to the data integration processing unit 13 as data D11.

[0070] Specifically, when the data output from the trained model (state determination processing data) indicates that the input data Din is normal state data (when the value of the first output node (node that outputs the probability of a normal state) of the trained model is greater than the value of the second output node (node that outputs the probability of an abnormal state) of the trained model), the state determination processing unit 11 generates data D11 indicating a normal state.

[0071] On the other hand, if the data output from the trained model (state determination processing data) indicates that the input data Din is data for an abnormal state (if the value of the first output node (node that outputs the probability of a normal state) of the trained model is less than or equal to the value of the second output node (node that outputs the probability of an abnormal state) of the trained model), the state determination processing unit 11 generates data D11 indicating an abnormal state.

[0072] Then, the state determination processing unit 11 outputs the data D11 generated as described above to the data integration processing unit 13.

[0073] The feature analysis processing unit 12 includes (is equipped with) a trained model (a trained model that outputs feature data (e.g., a feature vector)) acquired by machine learning, and receives data Din as input. The feature analysis processing unit 12 inputs the data Din to the trained model, and outputs the feature data (e.g., a feature vector) output from the trained model to the data integration processing unit 13 as data D12.

[0074] The data integration processing unit 13 receives the data D11 (data indicating normal / abnormal state) output from the state determination processing unit 11 and the data D12 (feature amount data) output from the feature analysis processing unit 12.

[0075] The data integration processing unit 13 outputs data including the data D11 and the data D12 (feature amount data) to the first communication interface 14 as data D2.

[0076] When the first communication interface 14 receives data D2 from the data integration processing unit 13, it generates communication data Dtx(D2) including the data D2 (communication data (communication signal) in a format that can be transmitted to an external device via a wired and / or wireless communication network), and transmits the generated communication data Dtx(D2) to the server device 200.

[0077] When the second communication interface 21 of the server device 200 receives the communication data Dtx(D2) from the terminal device 100, it extracts the data D2 included in the communication data Dtx(D2).

[0078] When the data D2 extracted from the communication data Dtx(D2) includes data D11 (data indicating normal / abnormal state) and data D12 (feature data), the second communication interface 21 outputs the data D11 (data indicating normal / abnormal state) as data D31 to the state analysis processing unit 23, and outputs the data D12 (feature data) as data D32 to the cause identification data acquisition processing unit 22.

[0079] As shown in FIG. 2, the factor identification data acquisition processing unit 22 is equipped (mounted) with N (N: natural number) trained models for factor identification data acquisition processing acquired by learning processing, namely, a first factor identification data acquisition processing trained model to an Nth factor data acquisition processing trained model.

[0080] As shown in Figure 2, the trained model for data acquisition processing for identifying the kth factor (k: natural number, 1≦k≦N) inputs data D32 (feature data) output from the second communication interface 21, processes D32 (feature data) using the trained model for data acquisition processing for identifying the kth factor, and acquires the processed data (data indicating whether the kth factor is normal or abnormal (determination result data)) as data D42_k.

[0081] Specifically, the process is executed as follows.

[0082] The trained model for data acquisition processing for identifying the first factor inputs data D32 (feature data), processes the D32 (feature data) using the trained model for data acquisition processing for identifying the first factor, and acquires the processed data (data (judgment result data) indicating whether the first factor (color) is normal or abnormal) as data D42_1.

[0083] The trained model for data acquisition processing for identifying the second factor receives data D32 (feature data) as input, processes the D32 (feature data) using the trained model for data acquisition processing for identifying the second factor, and acquires the processed data (data (judgment result data) indicating whether the second factor (shape) is normal or abnormal) as data D42_2.

[0084] The same processing as described above is also performed for the trained model for data acquisition processing for identifying the third to N-1th factors, and data (judgment result data) indicating whether the third to N-1th factors are normal or abnormal is acquired as data D42_3 to D42_N-1, respectively.

[0085] The trained model for data acquisition processing for identifying the Nth factor inputs data D32 (feature data), processes the D32 (feature data) using the trained model for data acquisition processing for identifying the Nth factor, and acquires the processed data (data (determination result data) indicating whether the Nth factor (size) is normal or abnormal) as data D42_N.

[0086] The data D42_1 to D42_N acquired as described above are output from the cause identification data acquisition processing unit 22 to the state analysis processing unit 23 as data D42.

[0087] The state analysis processing unit 23 determines whether the state of the data Din acquired by the terminal device 100 is normal or abnormal based on the data D31, and if it determines that the state is abnormal (if the data D31 is data indicating an abnormal state), performs processing to identify the cause of the abnormality based on the data D42. Specifically, the state analysis processing unit 23 identifies data indicating an abnormality from among the data D42_1 to D42_N constituting the data D42, thereby identifying the cause of the abnormality. For example, if the data D42_1 is data indicating an abnormal state, the state analysis processing unit 23 identifies that the data Din acquired at the time step t is data of an abnormal state, and that the cause of the abnormal state is the first cause, i.e., an abnormality in "color." In other words, when data D42_k (k: natural number, 1≦k≦N) is data indicating an abnormal state, the state analysis processing unit 23 identifies that the data Din acquired at time step t is data of an abnormal state, and that the cause of the abnormal state is the kth cause (for example, if it is the second cause, it is an abnormality in "shape," and if it is the Nth cause, it is an abnormality in "size").

[0088] If it is determined based on the data D31 that the state of the data Din acquired by the terminal device 100 is normal, the state analysis processing unit 23 determines that the data Din acquired at the time step t is normal state data. Note that this determination result data may be held by the state analysis processing unit 23, or may be stored in the data storage unit DB1, for example.

[0089] Furthermore, when it is determined based on the data D31 that the state of the data Din acquired by the terminal device 100 is abnormal (when the data D31 is data indicating an abnormal state), the state analysis processing unit 23 determines that the data Din acquired at the time step t is data indicating an abnormal state. Then, this determination result data (data indicating an abnormal state) and abnormality factor data indicating the result of identifying the abnormality factor may be held by the state analysis processing unit 23, or may be stored in the data storage unit DB1, for example.

[0090] The state analysis processing unit 23 acquires the result data of the data analysis process acquired by the above process as data Dout (for example, outputs it to the outside).

[0091] As described above, the data analysis processing system 1000 can perform analysis processing on data acquired by the terminal device 100 at time t (time step t).

[0092] Data analysis processing can also be performed for time steps t+1 (the time (time step) when data Din is next acquired by terminal device 100) and thereafter (time steps t+2, t+3, ...) by performing the same processing as above in the data analysis processing system 1000.

[0093] <Summary> As described above, in the data analysis processing system 1000, the terminal device 100 is equipped with a trained model (two-class classification trained model) that determines whether input data (data that can be considered normal / abnormal) is normal / abnormal, and a trained model (feature analysis trained model) that acquires feature data of the input data (data that can be considered normal / abnormal), and the data acquired by the two trained models is transmitted to the server device 200, which performs state analysis processing (acquisition of data indicating normal / abnormal states, processing to identify the cause of the abnormality, etc.). In other words, in the data analysis processing system 1000, the terminal device 100 is simply equipped with the two trained models, and therefore normal / abnormal determination result data and feature data of the input data can be acquired by real-time processing (processing guaranteed to be completed within a predetermined time) without installing a large-scale AI model in the terminal device. In the data analysis processing system 1000, the server device 200 is equipped with a trained model for factor identification data acquisition processing (the kth trained model for factor identification data acquisition processing (k: natural number, 1≦k≦N)), which is an N number of two-class classification trained models that can identify the cause of an abnormal state, and the trained model for factor identification data acquisition processing identifies the cause of an abnormal state in the input data. In other words, in the data analysis processing system 1000, the server device 200 is equipped with a trained model for factor identification data acquisition processing, which is an N number of two-class classification trained models, and the N number of two-class classification trained models are processed in parallel to acquire data for identifying the cause of an abnormality, and the acquired data can be used to identify the cause of an abnormality, so that the cause of an abnormal state in the input data can be identified by real-time processing.

[0094] Furthermore, in the data analysis processing system 1000, the above processing is executed continuously in chronological order, so that the results of the data analysis processing can be checked at any timing, making it easy to identify the cause of the abnormality and take measures to deal with the abnormality.

[0095] Furthermore, in the data analysis processing system 1000, feature data acquired by the feature analysis trained model installed in the feature analysis processing unit 12 of the terminal device 100 is transmitted from the terminal device 100 to the server device 200, thereby ensuring data security. That is, the feature data acquired (output) by the feature analysis trained model installed in the feature analysis processing unit 12 of the terminal device 100 is input to a trained model for data acquisition processing for k-th factor identification (k: natural number, 1≦k≦N) installed in the factor identification data acquisition processing unit 22 of the server device 200, and processed by the trained model for data acquisition processing for k-th factor identification. Therefore, even if the feature data is acquired (e.g., intercepted) during communication, a third party without the trained model for data acquisition processing for k-th factor identification cannot obtain the data as meaningful data. This ensures data security. As described above, the data analysis processing system 1000 includes a terminal device 100 that acquires data and a server device 200 that collects the data acquired by the terminal device 100 via a communication network. This makes it possible to detect anomalies and identify the causes of the anomalies through real-time processing, without having to install a large-scale AI model in the terminal device 100.

[0096] Note that, although the above description has been given of a case where the input data Din is not transmitted from the terminal device 100 to the server device 200, if the state determination processing unit 11 determines that the terminal device 100 is in an abnormal state, the data Din determined to be in an abnormal state may be transmitted to the server device 200. In this case, the data Din is input to the data integration processing unit 13, which outputs data obtained by integrating the data D11, the data D12, and the data Din as data D2 to the first communication interface 14, and the first communication interface 14 transmits communication data Dtx(D2) including the data D2 to the server device 200. Then, the server device 200 receives the communication data Dtx(D2) via the second communication interface 21 and outputs the data Din included in the communication data Dtx(D2) to the state analysis processing unit 23. When the status analysis processing unit 23 receives data Din from the second communication interface 21, it recognizes that the data Din is data indicating an abnormal state, and stores the data Din and data identifying the cause of the abnormality (this data is acquired by the same processing as described above (processing by the data acquisition processing unit 22 for identifying the cause and the status analysis processing unit 23)) in the data storage unit DB1.

[0097] <First Modification> Next, a first modified example of the first embodiment will be described. Note that the same parts as those in the above embodiment are given the same reference numerals, and detailed description thereof will be omitted.

[0098] The data analysis processing system of the first modified example of the first embodiment has the same configuration as the data analysis processing system 1000 of the first embodiment.

[0099] In the data analysis processing system 1000 of the first embodiment, only when the data D31 indicates an abnormal state in the server device 200, the process of identifying the cause of the abnormality is executed by the cause identification data acquisition processing unit 22, and the state analysis processing is performed by the state analysis processing unit 23. However, in the data analysis processing system of the first variant of the first embodiment, even when the data D31 indicates a normal state in the server device 200, the process is executed by the cause identification data acquisition processing unit 22, the occurrence probability of the abnormality cause is acquired, and the state analysis processing is performed by the state analysis processing unit 23 based on the occurrence probability.

[0100] Specifically, in the data analysis processing system of the first modified example of the first embodiment, even when the data D31 indicates a normal state in the server device 200, the data D32 (feature data) is output from the second communication interface 21 to the factor identification data acquisition processing unit 22.

[0101] The first factor identification data acquisition processing trained model 22_1 to the Nth factor identification data acquisition processing trained model 22_N of the factor identification data acquisition processing unit 22 receive the data D32 (feature amount data) as input, and acquire output data D42_1 to D42_N (probability of being normal (output value of the first output node) and probability of being abnormal (output value of the second output node)) from the trained model. Then, the acquired output data D42_1 to D42_N are output to the state analysis processing unit 23.

[0102] The state analysis processing unit 23 receives output data D42_1 to D42_N from the cause identification data acquisition processing unit 22 and stores and holds the output data D42_1 to D42_N (the output data D42_1 to D42_N may be stored in the data storage unit DB1). The state analysis processing unit 23 continuously acquires the output data D42_1 to D42_N from the cause identification data acquisition processing unit 22 in chronological order and analyzes changes in the output data D42_1 to D42_N in chronological order. For example, if, at a certain time, output data D42_k (k: natural number, 1≦k≦N) is a normal value but is close to a value determined to be abnormal (for example, the probability of normality is 0.5 or less but is close to 0.5), the state analysis processing unit 23 can determine that there is a high probability that an abnormality will occur in the near future due to the k-th cause, and outputs the determination result as included in data Dout. This allows the user to, for example, check the data Dout and recognize that there is a high probability that an abnormality will occur in the near future due to the k-th factor, and to take measures.

[0103] Furthermore, the status analysis processing unit 23 analyzes time-series changes in the output data D42_k (k: natural number, 1≦k≦N) and detects that (1) the value indicating normality (probability of normality) is monotonically decreasing or has a monotonically decreasing trend (or the moving average value of the value indicating normality (probability of normality) is monotonically decreasing or has a monotonically decreasing trend), or (2) the value indicating abnormality (probability of abnormality) is monotonically increasing or has a monotonically increasing trend (or the moving average value of the value indicating abnormality (probability of abnormality) is monotonically increasing or has a monotonically increasing trend). In this case, the status analysis processing unit 23 can determine that there is a high probability that an abnormality will occur in the near future due to the kth factor, and outputs the determination result in the data Dout. As a result, for example, by checking the data Dout, a user can recognize that there is a high probability that an abnormality will occur in the near future due to the kth factor, and can take measures.

[0104] As described above, in the data analysis processing system of the first modified example of the first embodiment, even when the server device 200 indicates that the data D31 is in a normal state, the cause identification data acquisition processing unit 22 performs processing to acquire the probability of occurrence of the abnormality cause, and the state analysis processing unit 23 performs state analysis processing based on the occurrence probability, thereby enabling more accurate data analysis processing (for example, processing to determine that there is a high probability that an abnormality will occur in the near future due to the kth cause).

[0105] [Second embodiment] Next, a second embodiment will be described. Note that the same parts as those in the above embodiment (including the modified examples) are given the same reference numerals, and detailed description thereof will be omitted.

[0106] FIG. 11 is a schematic configuration diagram of a data analysis processing system 2000 according to the second embodiment.

[0107] In the data analysis processing system 2000 of the second embodiment, when a cause of a new abnormal state is discovered, additional learning is performed to deal with the cause of the new abnormal state.

[0108] <2.1: Data analysis processing system configuration> The data analysis processing system 2000 of the second embodiment has a configuration in which the terminal device 100 is replaced with a terminal device 100A and the server device 200 is replaced with a server device 200A in the data analysis processing system 1000 of the first embodiment. In the data analysis processing system 2000 of the second embodiment, communication data including input data Din is transmitted from the terminal device 100A to the server device 200A.

[0109] The terminal device 100A has a configuration in which the data integration processing unit 13 in the terminal device 100 of the first embodiment is replaced with a data integration processing unit 13A.

[0110] The data integration processing unit 13A has the same functions as the data integration processing unit 13, and further inputs input data Din, performs data integration processing on data D11, data D12, and / or Din, and outputs data including data D11, data D12, and / or Din to the first communication interface 14 as data D2.

[0111] The server device 200A has a configuration in which the second communication interface 21 in the server device 200 of the first embodiment is replaced with a second communication interface 21A, and the status analysis processing unit 23 is replaced with a status analysis processing unit 23A.

[0112] The second communication interface 21A has the same functions as the second communication interface 21 of the first embodiment, and further, when the communication data Dtx(D2) includes data Din, it acquires the data Din from the communication data Dtx(D2) and outputs the acquired data as data D30 to the status analysis processing unit 23A.

[0113] The status analysis processing unit 23A has the same functions as the status analysis processing unit 23 of the first embodiment, and further receives as input data D30 output from the second communication interface 21A. When the data D31 (data indicating normal / abnormal) indicates "abnormal" and the cause of the abnormal state cannot be identified based on the data D42 (when none of the values (values indicating the probability of normality) of the data D42_1 to D42_N are 0.5 or greater and the cause of the abnormal state cannot be identified), the status analysis processing unit 23A stores the data D30 (data corresponding to the data Din) and the data D32 (feature amount data) in the data storage unit DB1 together with a label indicating that the data is due to a new cause of the abnormal state.

[0114] On the other hand, when the data D31 (data indicating normal / abnormal) indicates "abnormal," and the cause of the abnormal state can be identified based on the data D42, the state analysis processing unit 23A stores the data D30 (data corresponding to the data Din) and the data D32 (feature amount data) in the data storage unit DB1 together with a label indicating that the data is due to a new cause of the abnormal state, and after retaining the data for a predetermined period, deletes the data, or does not store the data D30 (data corresponding to the data Din) and the data D32 (feature amount data) in the data storage unit DB1 and erases the data.

[0115] <2.2: Operation of the data analysis processing system> The operation of the data analysis processing system 2000 configured as above will now be described.

[0116] 12 to 15 are flowcharts of the additional learning process executed by the data analysis processing system 2000. FIG.

[0117] In the following, for convenience of explanation, as in the first embodiment, in the data analysis processing system 2000, the data to be processed (data that can be considered as normal / abnormal data) is assumed to be image data of manufactured products.

[0118] Then, when data D31 (data indicating normal / abnormal) indicates "abnormal" and the cause of the abnormal state cannot be identified based on data D42 (when none of the values (values indicating the probability of normality) of data D42_1 to D42_N are 0.5 or greater and the cause of the abnormal state cannot be identified), the status analysis processing unit 23A of the server device 200A stores data D30 (data corresponding to data Din) and data D32 (feature amount data) in the data storage unit DB1 together with a label indicating that the data is due to a new cause of the abnormal state. The data stored in the data storage unit DB1 is referred to as "cause-unspecified data."

[0119] Then, when the number of data with unspecified factors acquired by the above process exceeds a predetermined number, or at a predetermined timing (after at least a plurality of data with unspecified factors has been acquired), the data analysis processing system 2000 executes additional learning processing.

[0120] Below, with reference to the flowcharts of Figures 12 to 15, we will explain the additional learning processes (additional processing of the feature analysis learning model, additional processing of the learning model for data acquisition processing for factor identification processing, and additional learning of the existing learning model for data acquisition processing for factor identification processing) performed by the data analysis processing system 2000.

[0121] (Step SA1): In step SA1, additional learning of the feature analysis learning model is performed. Specifically, the following processing is performed.

[0122] (Step SA11): In step SA11, image data and feature amount data of data for which the cause of the abnormal state is unknown (could not be specified) (cause unspecified data) are obtained from the data storage unit DB1.

[0123] (Step SA12): In step SA12, a new abnormality factor is set. Specifically, data that can identify a common abnormality factor is collected from the factor-unspecified data acquired in step SA11, and a new label (a label that identifies the abnormality factor) is assigned to the collected data.

[0124] For example, (1) if one group of data collected as described above indicates an abnormality in "color unevenness," then state #N+1 is set to indicate an abnormal state in "color unevenness" (the data in that group is labeled as abnormal color unevenness), and (2) if one group of data collected as described above indicates an abnormality in "brightness," then state #N+2 is set to indicate an abnormal state in "brightness" (the data in that group is labeled as abnormal brightness).

[0125] In the following explanation, it is assumed that m pieces (m: natural number) of data to which new labels (labels that identify the cause of the abnormality) have been assigned have been acquired in this way. That is, it is assumed that states #N+1 to N+m (m: natural number) have been added as m new abnormal states.

[0126] (Step SA13): In step SA13, loop 3 processing is started. Note that loop 3 processing is executed for each of states #N+1 to #N+m (abnormal states to which new labels have been assigned). Loop 3 processing is executed for state #k (each of states #N+1 to #N+m) in order, with variable k incremented by +1.

[0127] (Step SA14): In step SA14, data of state #k (k: integer, N+1≦k≦N+m) (this data is the data obtained in steps SA11 and SA12, and is stored in data storage unit DB1, for example) is obtained.

[0128] (Step SA15): In step SA15, a learning process is executed for a multi-class (N+1+m classes) classification learning model (feature extraction unit + classification unit). Specifically, in the multi-class (N+1 classes) classification learning model (feature extraction unit + classification unit) (the multi-class (N+1 classes) classification learning model acquired in step S2 of the first embodiment) in which parameters (optimal parameters) are set when the feature analysis trained model installed in the feature analysis processing unit 12 was acquired, m nodes are added to the softmax layer (output layer) to form a multi-class (N+1+m classes) classification learning model (feature extraction unit + classification unit).

[0129] Then, additional learning (e.g., fine tuning, transfer learning, distillation, etc.) is performed on the multi-class (N+1+m classes) classification learning model (feature extraction unit + classification unit) using data from state #k (k: integer, N+1≦k≦N+m). Specifically, this is performed as follows:

[0130] The data of state #k (k: integer, N+1≦k≦N+m) acquired in step SA14 is input to the multi-class (N+1+m classes) classification learning model. Then, processing by the feature extraction unit of the multi-class (N+1+m classes) classification learning model (feature extraction processing) and processing by the classification unit (N+1+m class classification processing) are executed, and output distribution data (N+1+m-dimensional vector) that is the output of the softmax layer of the classification unit is acquired.

[0131] Then, the error (e.g., cross-entropy error) between the data (N+1+m-dimensional vector) of the output node of the softmax layer of the classification unit and the correct label (a vector (one-hot vector) in which only the element (data) of the i-th dimension (i: natural number, 1≦i≦N+1+m) (data corresponding to the probability of being in state #k (k: integer, N+1≦k≦N+m))) is "1" and all elements other than the i-th dimension element are "0") is obtained.The obtained error is then used, for example, by backpropagation to update the parameters of the multi-class (N+1+m classes) classification learning model (parameters of the learnable model (convolutional layer, etc.) of the classification unit and parameters of the learnable model (fully connected layer, etc.) of the feature extraction unit).

[0132] (Step SA16): In step SA16, a determination process is performed to determine whether the termination condition for loop 3 processing is satisfied. Specifically, if it is determined that loop 3 processing has been executed for all states #N+1 to #N+m, the termination condition for loop 3 processing is satisfied, so loop 3 processing is terminated and processing proceeds to step SA17. On the other hand, if it is determined that loop 3 processing has not been executed for all states #N+1 to #N+m, processing returns to step SA13, and loop 3 processing (steps SA13 to SA16) is executed.

[0133] (Step SA17): In step SA17, a determination process is performed to determine whether the learning process of the trainable model installed in the feature analysis processing unit 12 of the terminal device 100 has converged.If the result of the determination process is that the learning process has not converged, the process returns to step SA13; on the other hand, if it is determined that the learning process has converged, the process proceeds to step SA18.

[0134] The process of determining whether the learning process of the trainable model installed in the feature analysis processing unit 12 of the terminal device 100 has converged is, for example, determined to have converged in the following cases. (1) When the error (e.g., cross-entropy error) between the output node data (N+1+m-dimensional vector) of a multi-class (N+1+m classes) classification learning model and the correct label falls below a specified value. (2) When the variation in the error (e.g., cross-entropy error) between the output node data (N+1+m-dimensional vector) of a multi-class (N+1+m classes) classification learning model and the correct label falls within a specified range.

[0135] (Step SA18): In step SA18, a process is executed to load the feature analysis trained model into the feature analysis processing unit 12 of the terminal device 100. Specifically, the following process is executed.

[0136] In step SA18, when it is determined that the learning process has ended (converged), the parameters set in the multi-class (N+1+m classes) classification learning model are acquired as optimal parameters.The multi-class (N+1+m classes) classification learning model in which the optimal parameters are set is acquired as a multi-class (N+1+m classes) classification trained model.The classifier unit is then deleted from the multi-class (N+1+m classes) classification trained model, and a model in which only the convolution processing unit is extracted is acquired as a feature analysis trained model.The acquired feature analysis trained model is then installed in the feature analysis processing unit 12 of the terminal device 100 (the convolution processing unit (the convolution processing unit in which the optimal parameters are set) of the multi-class (N+1+m classes) classification trained model is set as the feature analysis trained model, and the feature analysis trained model is installed in the feature analysis processing unit 12).

[0137] When input data is input to the multi-class (N+1+m class) classification trained model acquired by the above learning process, the model can appropriately determine that the input data is data on state #k (state #0: normal state, states #1 to #N+m: each of N+m different abnormal states) (it can determine which of N+1+m states the input data is in (it can classify the data into N+1+m classes)). When input data is input to the feature analysis trained model extracted from only the convolution processing unit of the multi-class (N+1+m class) classification trained model, the model can output feature data (e.g., feature vector) for state #k if the input data is data on state #k (state #0: normal state, states #1 to #N+m: each of N+m different abnormal states).

[0138] (Step SA2): In step SA2, an additional process for the learning model for the factor identification process data acquisition process is executed. Specifically, the following process is executed.

[0139] (Step SA21): In step SA21, input data for state #i (N+1≦i≦N+m) (data for abnormal states #N+1 to #N+m) is input to the feature analysis trained model acquired in step SA1, and a process is executed to acquire the output feature data for state #i (e.g., feature vector). Specifically, input data for state #i (N+1≦i≦N+m) (data for abnormal states #N+1 to #N+m (e.g., image data)) is input to the feature analysis trained model acquired in step SA1, and the output feature data for state #i (e.g., feature vector) is acquired. The acquired feature data for state #i (e.g., feature vector) is stored in, for example, data storage unit DB1.

[0140] (Step SA22): In step SA22, loop 4 processing is started. Note that loop 4 processing is executed for each state (each abnormal state) from state #N+1 to state #N+m. Loop 4 processing is executed for state #i (each state from state #N+1 to state #N+m) in order by incrementing variable i (i: natural number, N+1≦i≦N+m) by 1.

[0141] (Step SA23): In step SA23, data (data of abnormal state) of state #i (i: natural number, N+1≦i≦N+m) is acquired from the data storage unit DB1.

[0142] (Step SA24): In step SA24, learning processing of the factor identification two-class classification learning model (learning model for data acquisition processing for i-th factor identification processing) is executed. Specifically, data (e.g., image data of a manufactured product in an abnormal state) for state #i (i: natural number, N+1≦i≦N+m) (abnormal state) stored in the data storage unit DB1 is input to the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model), and the output of the learning model for data acquisition processing for i-th factor identification processing, i.e., classification result data, is obtained. Note that the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) has two output nodes in the output layer, with the first output node being a node that outputs the probability that the input data is normal and the second output node being a node that outputs the probability that the input data is abnormal. Then, the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) and the correct label (a two-dimensional vector in which the first-dimensional data (probability of being normal) is 0.0 and the second-dimensional data (probability of being abnormal) is 1.0 (vector (one-hot vector)) is obtained. Then, using the obtained error, for example, by error backpropagation, the parameters of the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) are updated. This process is repeated until the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) of the learning model for data acquisition processing for i-th factor identification processing and the correct label becomes equal to or less than a predetermined value, or until the amount of variation in the error falls within a predetermined range, thereby converging the learning. Then, when it is determined that the learning has converged, the parameters set in the learning model for data acquisition processing for i-th factor identification processing (two-class classification learnable model) are acquired as optimal parameters of the learning model for data acquisition processing for i-th factor identification processing.

[0143] (Step SA25): In step SA25, a determination process is performed to determine whether the termination condition for loop 4 processing is satisfied. Specifically, if it is determined that loop 4 processing has been executed for all states #N+1 to #N+m, the termination condition for loop 4 processing is satisfied, so loop 4 processing is terminated and processing proceeds to step SA26. On the other hand, if it is determined that loop 4 processing has not been executed for all states #N+1 to #N+m, processing returns to step SA22, and loop 4 processing (steps SA22 to SA25) is executed.

[0144] (Step SA26): In step SA26, a process is executed to install a factor identification two-class classification trained model (a trained model for i-th factor identification processing data acquisition process) in the factor identification data acquisition processing unit 22 of the server device 200A. Specifically, the m factor identification two-class classification trained models acquired above, i.e., the (N+1)th to (N+m)th factor identification data acquisition processing trained models (the i-th factor identification data acquisition processing trained model for which optimal parameters are set (this will be denoted as "trained model AI#i" (i: integer, N+1≦i≦N+m))) are added (additionally installed) in the factor identification data acquisition processing unit 22 of the server device 200A. (Step SA3): In step SA3, additional learning of the existing learning model for the process of acquiring data for the process of factor identification is performed. Specifically, the following process is performed.

[0145] (Step SA31): In step SA31, input data for state #i (N+1≦i≦N+m) (data for abnormal states #N+1 to #N+m) is input to the feature analysis trained model acquired in step SA1, and a process is executed to acquire the output feature data for state #i (e.g., feature vector). Specifically, input data for state #i (N+1≦i≦N+m) (data for abnormal states #N+1 to #N+m (e.g., image data)) is input to the feature analysis trained model acquired in step SA1, and the output feature data for state #i (e.g., feature vector) is acquired. The acquired feature data for state #i (e.g., feature vector) is stored in, for example, data storage unit DB1.

[0146] (Step SA32): In step SA32, loop 5 processing is started. Note that loop 5 processing is executed for each state (each abnormal state) from state #1 to state #N. Loop 5 processing is executed for state #j (each state from state #1 to state #N) in order by incrementing variable j (j: natural number, 1≦j≦N) by +1.

[0147] (Step SA33): In step SA33, loop 6 processing is started. Note that loop 6 processing is executed for each state (each abnormal state) from state #N+1 to state #N+m. Loop 6 processing is executed for state #i (each state from state #N+1 to state #N+m) in order by incrementing variable i (i: natural number, N+1≦i≦N+m) by 1.

[0148] (Step SA34): In step SA34, data (data of abnormal state) of state #i (i: natural number, N+1≦i≦N+m) is acquired from the data storage unit DB1.

[0149] (Step SA35): In step SA35, learning processing is executed for the factor identification two-class classification learning model (jth factor identification processing data acquisition processing learning model) (N existing factor identification processing data acquisition processing learning models). Specifically, data (e.g., image data of a manufactured product in an abnormal state) for state #i (i: natural number, N+1≦i≦N+m) (abnormal state) stored in the data storage unit DB1 is input to the jth factor identification processing data acquisition processing learning model (two-class classification learnable model), and the output of the jth factor identification processing data acquisition processing learning model, i.e., classification result data, is acquired.

[0150] Then, in the learning model for data acquisition processing for j-th factor identification processing, an error (e.g., cross-entropy error) between the output node data (two-dimensional vector) and the correct label (a two-dimensional vector in which the first-dimensional data (probability of being normal) is 0.0 and the second-dimensional data (probability of being abnormal) is 1.0 (vector (one-hot vector)) is obtained. Then, using the obtained error, for example, by error backpropagation, the parameters of the learning model for data acquisition processing for j-th factor identification processing (two-class classification learnable model) are updated. This process is repeated until the error (e.g., cross-entropy error) between the output node data (two-dimensional vector) of the learning model for data acquisition processing for j-th factor identification processing and the correct label becomes equal to or less than a predetermined value, or until the amount of variation in the error falls within a predetermined range, thereby converging the learning. Then, when it is determined that the learning has converged, the parameters set in the learning model for data acquisition processing for j-th factor identification processing (two-class classification learnable model) are obtained as optimal parameters of the learning model for data acquisition processing for j-th factor identification processing.

[0151] (Step SA36): In step SA36, a determination is made as to whether the end condition for loop 6 processing is met. Specifically, if it is determined that loop 6 processing has been executed for all states #N+1 to #N+m, the end condition for loop 6 processing is met, so loop 6 processing is terminated and processing proceeds to step SA37. On the other hand, if it is determined that loop 6 processing has not been executed for all states #N+1 to #N+m, processing returns to step SA33, and loop 6 processing (steps SA33 to SA36) is executed.

[0152] (Step SA37): In step SA37, a determination is made as to whether the end condition for loop 5 processing is met. Specifically, if it is determined that loop 5 processing has been executed for all states #1 to #N, the end condition for loop 5 processing is met, so loop 5 processing is terminated and processing proceeds to step SA38. On the other hand, if it is determined that loop 5 processing has not been executed for all states #1 to #N, processing returns to step SA32, and loop 5 processing (steps SA32 to SA37) is executed.

[0153] (Step SA38): In step SA38, a process is executed to install a factor identification two-class classification trained model (a jth factor identification processing data acquisition processing trained model (j: natural number, 1≦j≦N)) in the factor identification data acquisition processing unit 22 of the server device 200A. Specifically, N factor identification two-class classification trained models that have been subjected to the additional learning (for example, fine tuning), that is, the first to Nth factor identification data acquisition processing trained models (the ith factor identification data acquisition processing trained model for which optimal parameters are set (this will be denoted as an "additional trained model AI#j" (j: integer, 1≦j≦N))) are installed in the factor identification data acquisition processing unit 22 of the server device 200A.

[0154] As a result of the above, additional learning processing (additional processing of the feature analysis learning model, additional processing of the learning model for factor identification data acquisition processing, and additional learning of the existing learning model for factor identification data acquisition processing) is performed in the data analysis processing system 2000.

[0155] Furthermore, the data analysis processing system 2000 equipped with the trained model after the additional learning process can detect N+m abnormal states, so by performing processing in the same manner as in the first embodiment, it becomes possible to detect (identify) additional abnormal states as well, thereby enabling more accurate data analysis processing.

[0156] <First Modification> Next, a first modified example of the second embodiment will be described. Note that the same parts as those in the above embodiment (including the modified example) are given the same reference numerals, and detailed description thereof will be omitted.

[0157] FIG. 16 is a schematic configuration diagram of a data analysis processing system 2000A according to a first modified example of the second embodiment.

[0158] In the data analysis processing system 2000A of the first modified example of the second embodiment, when a cause of a new abnormal state is discovered, additional learning is performed and the cause of the new abnormal state can be dealt with, similar to the data analysis processing system 2000 of the second embodiment. In the data analysis processing system 2000A of the first modified example of the second embodiment, the terminal device stores and holds data Din (image data) of the abnormal state, and transmits the data Din (image data) of the abnormal state to the server device in response to a request from the server device.

[0159] A data analysis processing system 2000A of a first modified example of the second embodiment has a configuration in which the terminal device 100 in the data analysis processing system 1000 of the first embodiment is replaced with a terminal device 100A and the server device 200 is replaced with a server device 200B.

[0160] The terminal device 100A has a configuration in which the data integration processing unit 13 in the terminal device 100 of the first embodiment is replaced with a data integration processing unit 13B, the first communication interface 14 is replaced with a first communication interface 14A, and a terminal data storage unit 15 is further added.

[0161] The server device 200B has a configuration in which the second communication interface 21 in the server device 200 of the first embodiment is replaced with a second communication interface 21B, and the status analysis processing unit 23 is replaced with a status analysis processing unit 23B.

[0162] The data integration processing unit 13B has the same functions as the data integration processing unit 13, and furthermore, when the data D11 output from the state determination processing unit 11 indicates an abnormal state, it outputs the data Din as data D20A to the terminal data storage unit 15. Furthermore, when the data integration processing unit 13B receives a request signal Req from the first communication interface 14A, it acquires the data indicated by the request signal Req as data D20B from the terminal data storage unit 15. Then, the data integration processing unit 13B outputs the data D20B to the first communication interface 14A as data D2.

[0163] If the data D11 indicates an abnormality, the data integration processing unit 13B generates data D2 including the data D11 and the data D12 and outputs the data D2 to the first communication interface 14A. That is, in this case, if the data D11 indicates an abnormality, the data integration processing unit 13B does not include the data Din in the data D2.

[0164] The first communication interface 14A has the same functions as the first communication interface 14, and further, when it receives communication data Dtx(Req) transmitted from the server device 200A, it extracts the request signal Req included in the communication data Dtx(Req) and outputs the request signal Req to the data integration processing unit 13B. Furthermore, when the first communication interface 14A receives data D2 including data D20B from the data integration processing unit 13B, it generates communication data Dtx(D2) for transmitting the data D2 and transmits the communication data Dtx(D2) to the server device 200B.

[0165] The terminal data storage unit 15 is a functional unit that stores and retains data. When it receives data D20A output from the data integration processing unit 13A, it stores and retains the data D20A. In addition, in accordance with a read command from the data integration processing unit 13A, the terminal data storage unit 15 reads predetermined data based on the read command and outputs the read data as data D20B to the data integration processing unit 13A. In order to ensure the necessary data storage capacity, the terminal data storage unit 15 may delete data older than the data specified in the read command (past data) when it receives a read command from the data integration processing unit 13B. In order to ensure the necessary data storage capacity, the terminal data storage unit 15 may delete data that is determined to be eligible for deletion when it receives a request signal from the server device 200B instructing it to delete the data stored in the terminal data storage unit 15. In addition, in order to ensure the necessary data storage capacity, the terminal data storage unit 15 may delete the data after a predetermined period of time has elapsed.

[0166] The server device 200B has a configuration in which the status analysis processing unit 23 in the server device 200 of the first embodiment is replaced with a status analysis processing unit 23B.

[0167] The second communication interface 21B has the same functions as the second communication interface 21, and further, when it receives a request signal Req output from the status analysis processing unit 23B, it generates communication data Dtx(Req) including the request signal Req and transmits the communication data Dtx(Req) to the terminal device 100A.

[0168] The status analysis processing unit 23B has the same functions as the status analysis processing unit 23, and further, when the data D31 (data indicating normal / abnormal) indicates "abnormal" and the cause of the abnormal state cannot be identified based on the data D42 (when none of the values (values indicating the probability of normality) of the data D42_1 to D42_N are 0.5 or greater and the cause of the abnormal state cannot be identified), the status analysis processing unit 23B generates a request signal Req requesting the terminal device 100A to transmit to the server device 200A data Din when the data D11 indicates an abnormality, and outputs the request signal Req to the second communication interface 21B. Then, in response to the request signal Req, the status analysis processing unit 23B inputs, as data D30, via the second communication interface 21B, data D2 acquired from the communication data Dtx(D2) transmitted from the terminal device 100A (data including data Din (data D20B) when the communication data was determined to be abnormal but the cause of the abnormality could not be identified). Then, the condition analysis processing unit 23B stores the input data D30 (data equivalent to data Din (data D20B) when an abnormality is determined but the cause of the abnormality cannot be identified) and data D32 (feature data) in the data storage unit DB1 together with a label indicating that the data is due to the cause of the new abnormal state.

[0169] The operation of the data analysis processing system 2000A of the first modified example of the second embodiment configured as above will be described below. Note that a description of the same parts as those in the above embodiment (including the modified examples) will be omitted.

[0170] If the data D11 indicates an abnormality, the data integration processing unit 13B generates data D2 including the data D11 and the data D12 and outputs the data D2 to the first communication interface 14A. That is, the data integration processing unit 13A does not include the data Din in the data D2. As a result, the data Din is not transmitted from the terminal device 100A to the server device 200B, thereby reducing the amount of communication traffic.

[0171] Furthermore, when the data D11 output from the state determination processing unit 11 indicates an abnormal state, the data integration processing unit 13B outputs the data Din as data D20A to the terminal data storage unit 15. Then, the terminal device 100A stores the data D20A.

[0172] When the data D31 (data indicating normal / abnormal) indicates "abnormal," and the status analysis processing unit 23B of the server device 200B is unable to identify the cause of the abnormal state based on the data D42 (when none of the values of the data D42_1 to D42_N (values indicating the probability of normality) are 0.5 or greater and the cause of the abnormal state cannot be identified), the status analysis processing unit 23B generates a request signal Req requesting the terminal device 100A to transmit data Din to the server device 200A when the data D11 indicates an abnormality, and outputs the request signal Req to the second communication interface 21A.

[0173] The second communication interface 21A generates communication data Dtx(Req) including the request signal Req input from the state analysis processing unit 23B, and transmits the communication data Dtx(Req) to the terminal device 100A.

[0174] The first communication interface 14A of the terminal device 100A receives the communication data Dtx(Req) transmitted from the server device 200B, extracts the request signal Req contained in the communication data Dtx(Req), and outputs the request signal Req to the data integration processing unit 13B.

[0175] When a request signal Req is input from the first communication interface 14A, the data integration processing unit 13B reads data D20B from the terminal data memory unit 15 in response to the request signal Req, which is data equivalent to data Din (data D20B) when an abnormality indicated by the request signal Req is determined to exist but the cause of the abnormality cannot be identified, and outputs data including the data D20B to the first communication interface 14A as data D2.

[0176] When the first communication interface 14A receives the data D2 from the data integration processing unit 13B, it generates communication data Dtx(D2) including the data D2 and transmits the communication data Dtx(D2) to the server device 200B.

[0177] The second communication interface 21B of the server device 200B receives the communication data Dtx(D2) transmitted from the terminal device 100A, extracts the data D2 included in the communication data Dtx(D2), and outputs it as data D30 to the state analysis processing unit 23B.

[0178] Then, the status analysis processing unit 23B stores the data D30 input from the second communication interface 21B (data equivalent to data Din (data D20B) when an abnormality is determined but the cause of the abnormality cannot be identified) and data D32 (feature data of the above data Din) in the data storage unit DB1, together with a label indicating that the data is due to the cause of the new abnormal state.

[0179] As described above, in the data analysis processing system 2000A, similar to the data analysis processing system 2000 of the second embodiment, data due to the cause of the occurrence of a new abnormal state (data when an abnormality is determined but the cause of the abnormality cannot be identified) can be stored in the data storage unit DB1. Then, in the data analysis processing system 2000A, similar to the data analysis processing system 2000 of the second embodiment, by executing additional learning processes (additional processing of the feature analysis learning model, additional processing of the learning model for data acquisition processing for factor identification processing, and additional learning of the existing learning model for data acquisition processing for factor identification processing), it is possible to perform additional processing of the feature analysis learning model, additional processing of the learning model for data acquisition processing for factor identification processing, and additional learning of the existing learning model for data acquisition processing for factor identification processing.

[0180] Then, in the data analysis processing system 2000A equipped with the trained model after the additional learning processing, data analysis processing can be performed by performing processing similar to that in the first embodiment. The data analysis processing system 2000A can detect N+m abnormal states, similar to the data analysis processing system 2000, and therefore can detect (identify) additional abnormal states as well, enabling more accurate data analysis processing.

[0181] [Other embodiments] In the above embodiment, in the data analysis processing system 1000, 2000, 2000A, the feature analysis trained model is installed in the terminal device 100, 100A, and the trained model for factor identification data acquisition processing is installed in the server device 200, 200A, 200B. However, this is not limited to this, and for example, the trained models may be installed in the first device and the second device as follows. This will be explained using the flowchart in FIG. 17.

[0182] Figure 17 is a flowchart of a process in which a first split trained model, which is a trained model obtained by training a first trainable model that is a trainable model having a feature extraction unit and a class classification unit, is installed on a first device (e.g., a terminal device), and a class classification trained model, which is a trained model trained using training data obtained by the first split trained model, is installed on a second device (e.g., a server device).

[0183] (Step SC1): In step SC1, a learning process is performed on a first trainable model that includes a feature extraction unit and a class classification unit, and a trained model of the first trainable model (first trained model) is obtained.

[0184] (Step SC2): In step SC2, a model obtained by deleting the class classification unit from the first trained model is acquired as a first divided trained model.

[0185] (Step SC3): In step SC3, the input data (e.g., image data) used in the learning process of the first trainable model is input to the first divided trained model, and the output from the first divided trained model is obtained as first learning data.

[0186] (Step SC4): In step SC4, a learning process is performed on one or more models capable of class classification learning using the first learning data, and a class classification trained model is obtained.

[0187] (Step SC5): In step SC5, the first divided trained model is loaded onto a first device (for example, a terminal device).

[0188] (Step SC6): In step SC6, the class classification trained model acquired in step SC4 is loaded onto a second device (for example, a server device).

[0189] By performing the above processing, a predetermined process (a process realized using the first-division trained model and the classification trained model) can be performed by installing the first-division trained model in a first device (e.g., a terminal device), transmitting output data (feature data) of the first-division trained model to a second device (e.g., a server device) via a network, and performing processing using the classification trained model installed in the second device. Since the output from the first-division trained model installed in the first device cannot be obtained as meaningful data unless processing is performed using the classification trained model installed in the second device, security of the transmitted data can be ensured when the output data (feature data) of the first-division trained model is transmitted from the first device to the second device.

[0190] Furthermore, a first trainable model, which is a trainable model having a feature extraction unit and a class classification unit, may be trained and a part of the first trainable model obtained may be installed in the first device, and the remaining part may be installed in the second device. This will be explained using the flowchart in FIG.

[0191] Figure 18 is a flowchart of a process in which a first split trained model, which is a trained model obtained by training a first trainable model that is a trainable model having a feature extraction unit and a class classification unit, is installed on a first device (e.g., a terminal device), and a second split trained model, which is a trained model obtained by extracting the remaining part of the first split trained model, is installed on a second device (e.g., a server device).

[0192] (Step SD1): In step SD1, a learning process is performed on a first trainable model that includes a feature extraction unit and a class classification unit, and a trained model of the first trainable model (first trained model) is obtained.

[0193] (Step SD2): In step SD2, a model in which the class classification unit is removed from the first trained model is obtained as a first divided trained model.

[0194] (Step SD3): In step SD3, a model in which the feature extraction unit is deleted from the first trained model is obtained as a second divided trained model.

[0195] (Step SD4): In step SD4, the first divided trained model is loaded onto a first device (for example, a terminal device).

[0196] (Step SD5): In step SD5, the second divided trained model is loaded onto a second device (for example, a server device).

[0197] By performing the above processing, a predetermined process (a process realized using the first divided trained model and the second divided trained model) can be performed by installing the first divided trained model in a first device (e.g., a terminal device), transmitting output data (feature data) of the first divided trained model to a second device (e.g., a server device) via a network, for example, and performing processing using the second divided trained model installed in the second device. Since the output from the first divided trained model installed in the first device cannot be obtained as meaningful data unless processing is performed using the second divided trained model installed in the second device, security of the transmitted data can be ensured when the output data (feature data) of the first divided trained model is transmitted from the first device to the second device.

[0198] In addition, in the above embodiment, the data analysis processing systems 1000, 2000, 2000A are described as being connected between the terminal devices 100, 100A and the server devices 200, 200A, 200B via a wired and / or wireless communication network (in the case of a 1:1 connection), but this is not limited to this, and the data analysis processing systems 1000, 2000, 2000A may be connected between n (n: natural number) terminal devices and m (m: natural number) server devices (n:m connection) via a wired and / or wireless communication network, or a mixed wired and wireless communication network.

[0199] In addition, in the above embodiments, the data analysis processing systems 1000, 2000, and 2000A are described as using a learnable model, but the learnable model may also be a machine learning model, a deep learning model, a neural network model, or other model.

[0200] Furthermore, each block (each functional unit) of the data analysis processing systems 2000, 2000A, the terminal devices 100, 100A, and the server devices 200, 200A, 200B described in the above embodiments may be individually implemented as a single chip using a semiconductor device such as an LSI, or may be integrated into a single chip to include some or all of the blocks. Furthermore, each block (each functional unit) of the data analysis processing systems 1000, 2000, 2000A, the terminal devices 100, 100A, and the server devices 200, 200A, 200B described in the above embodiments may be realized by multiple semiconductor devices such as LSIs.

[0201] Although we have referred to it as an LSI here, it may also be called an IC, system LSI, super LSI, or ultra LSI depending on the level of integration.

[0202] Furthermore, the method of integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. It is also possible to use FPGAs (Field Programmable Gate Arrays), which can be programmed after the LSI is manufactured, or reconfigurable processors, which allow the connections and settings of circuit cells inside the LSI to be reconfigured.

[0203] Furthermore, part or all of the processing of each functional block in each of the above embodiments may be realized by a program. And part or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. Furthermore, the programs for performing each processing are stored in a storage device such as a hard disk or ROM, and are executed in the ROM or by being read into the RAM.

[0204] Furthermore, each process in the above-described embodiments may be realized by hardware, or by software (including cases where it is realized together with an OS (operating system), middleware, or a predetermined library). Furthermore, it may be realized by a combination of software and hardware.

[0205] For example, when each functional unit of the above embodiment is realized by software, each functional unit may be realized by software processing using the hardware configuration shown in FIG. 19 (for example, a hardware configuration in which a CPU, GPU, processor, ROM, RAM, memory, input unit, output unit, etc. are connected via a bus).

[0206] Furthermore, when each functional unit of the above embodiment is realized by software, the software may be realized using a single computer having the hardware configuration shown in Figure 19, or may be realized by distributed processing using multiple computers.

[0207] Furthermore, the execution order of the processing method in the above embodiment is not necessarily limited to the description of the above embodiment, and the execution order can be changed within the scope of the gist of the invention. Furthermore, in the processing method in the above embodiment, some steps may be executed in parallel with other steps within the scope of the gist of the invention. Furthermore, in the processing method in the above embodiment, processes that are executed in parallel may be executed serially (sequentially).

[0208] The scope of the present invention includes a computer program for causing a computer to execute the above-described method and a computer-readable recording medium having the program recorded thereon, including, for example, a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a large-capacity DVD, a next-generation DVD, and a semiconductor memory.

[0209] The computer program is not limited to one recorded on the recording medium, but may be one transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, or the like.

[0210] Furthermore, in the description of this specification and the claims, "optimal" refers to the best state or the state that is brought to the best, and the "optimal parameters" of a system (model) refer to parameters that make the value of the objective function of the system (model) the optimal value. The "optimal value" is the maximum value when the system is in a better state as the value of the system's objective function increases, and is the minimum value when the system is in a better state as the value of the system's objective function decreases. The "optimal value" may also be an extreme value. The "optimal value" may also allow for a predetermined error (measurement error, quantization error, etc.), and may be a value within a predetermined range (a range that can be considered to have sufficiently converged).

[0211] The term "part" may also include the concept of "circuitry." A circuitry may be realized in whole or in part by hardware, software, or a combination of hardware and software.

[0212] The functions of the elements disclosed herein may be implemented using circuitry or processing circuitry, including general-purpose processors, special-purpose processors, integrated circuits, ASICs ("application-specific integrated circuits"), conventional circuitry, and / or combinations thereof, configured to perform the disclosed elements or programmed to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry when it includes transistors and other circuitry therein. In this disclosure, a circuitry, unit, or means is hardware that performs the recited function or hardware programmed to perform the function. The hardware may be any hardware disclosed herein or other known hardware that is programmed to perform or configured to perform the recited function. When the hardware is a processor, which may be considered as a type of circuitry, the circuitry, means, or unit is a combination of hardware and software, software used to configure the hardware, and / or processor.

[0213] The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the gist of the invention.

[0214] [Note] The present invention can also be realized as follows.

[0215] A first aspect of the present invention is a data analysis processing system that performs analysis processing on data that can distinguish between a normal state and an abnormal state, and includes a terminal device and a server device.

[0216] The terminal device includes a state determination processing unit, a feature analysis processing unit, a data integration processing unit, and a first communication interface unit.

[0217] The state determination processing unit determines the state of the data to be processed, and obtains the result data of the determination as state determination data.

[0218] The feature analysis processing unit acquires feature amount data of the processing data from the processing target data.

[0219] The data integration processing unit acquires data including the state determination data and the feature amount data as integrated data.

[0220] The first communication interface is a communication interface for transmitting communication data including the integrated data to the server device.

[0221] The server device includes a second communication interface unit, a cause identification data acquisition processing unit, and a status analysis processing unit.

[0222] The second communication interface unit is a communication interface for receiving communication data transmitted from a terminal device.

[0223] The factor identification data acquisition processing unit inputs feature data included in the communication data acquired by the second communication interface unit, and acquires factor identification data, which is data for identifying the cause of the occurrence of the abnormal state, from the feature data.

[0224] The status analysis processing unit performs status analysis processing based on the status determination data and the cause identification data included in the communication data acquired by the second communication interface unit.

[0225] In this data analysis processing system, a terminal device includes a state determination processing unit (e.g., a functional unit equipped with a trained model (a two-class classification trained model)) that determines whether data to be processed (input data) (data that can be considered normal / abnormal) is normal / abnormal, and a feature analysis processing unit (e.g., a functional unit equipped with a trained model (a feature analysis trained model)) that acquires feature data of the data to be processed (input data) (data that can be considered normal / abnormal). Data acquired by these units is then transmitted to a server device, where state analysis processing is performed. In other words, in this data analysis processing system, the terminal device simply performs processing using the above two functional units (the state analysis processing unit and the feature analysis processing unit), and normal / abnormal determination result data and feature data of the input data (data to be processed) can be acquired by real-time processing (processing that is guaranteed to be completed within a predetermined time) without installing a large-scale AI model in the terminal device.

[0226] The second invention is the first invention, wherein the feature analysis processing unit is an N+1 class classification learnable model that takes the data to be processed as input and performs N+1 class classification to classify the data into a normal state class and N (N: natural number) abnormal state classes, and has a feature analysis learned model that is a model obtained by removing the class classification unit of the N+1 class classification learned model, which is a learned model obtained by performing a learning process using normal state data and abnormal state data of the data to be processed, compared to the N+1 class classification learnable model that has a feature extraction unit and a class classification unit.

[0227] The data acquisition processing unit for factor identification has a two-class classification trained model for identifying the kth factor, which uses the kth abnormal data, a set of data classified into the kth class (k: natural number, 1≦k≦N), as learning data when abnormal state data of the data to be processed is input into an N+1 class classification trained model, and performs two-class classification based on the kth factor, which is the cause of the abnormality in the data classified into the kth class, to determine whether an abnormal state has occurred or not.

[0228] In this data analysis processing system, a terminal device includes a state determination processing unit (e.g., a functional unit equipped with a trained model (a two-class classification trained model)) that determines whether data to be processed (input data) (data that can be considered normal / abnormal) is normal / abnormal, and a feature analysis processing unit (e.g., a functional unit equipped with a trained model (a feature analysis trained model)) that acquires feature data of the data to be processed (input data) (data that can be considered normal / abnormal). Data acquired by these units is then transmitted to a server device, where state analysis processing is performed. In other words, in this data analysis processing system, the terminal device simply performs processing using the above two functional units (the state analysis processing unit and the feature analysis processing unit), and normal / abnormal determination result data and feature data of the input data (data to be processed) can be acquired by real-time processing (processing that is guaranteed to be completed within a predetermined time) without installing a large-scale AI model in the terminal device.

[0229] In this data analysis processing system, a server device is equipped with a trained model for factor identification data acquisition processing (a k-th trained model for factor identification data acquisition processing (k: natural number, 1≦k≦N)), which is an N number of two-class classification trained models that can identify the cause of an abnormal state, and the trained model for factor identification data acquisition processing identifies the cause of an abnormal state in the input data. In other words, in this data analysis processing system, a server device is equipped with a trained model for factor identification data acquisition processing, which is an N number of two-class classification trained models, and the N number of two-class classification trained models are processed in parallel to acquire data for identifying the cause of an abnormality, and the data is used to identify the cause of an abnormality, so that the cause of an abnormal state in the input data (data to be processed) can be identified by real-time processing.

[0230] A third invention is the second invention, further comprising a data storage unit that stores data.

[0231] When the state determination data indicates an abnormal state, the data integration processing unit acquires data including the state determination data, the feature amount data, and the processing target data as integrated data.

[0232] If the status determination data indicates an abnormality and the cause of the abnormality cannot be identified using the cause identification data acquired by the cause identification data acquisition processing unit, the status analysis processing unit stores the data to be processed contained in the communication data acquired by the second communication interface unit in the data storage unit as cause-unknown data.

[0233] As a result, in this data analysis processing system, when the status judgment data indicates an abnormality, data (data with unknown cause) can be stored in the data storage unit when the cause of the abnormality cannot be identified using the cause identification data acquired by the cause identification data acquisition processing unit.

[0234] The first communication interface unit and the second communication interface unit may be functional units (communication interfaces) that perform encryption / decryption and scrambling / descrambling and generate communication data.

[0235] A fourth invention is the second invention, in which the terminal device further includes a terminal data storage unit.

[0236] If the status determination data indicates an abnormality, the data integration processing unit stores the processing target data in the terminal data storage unit.

[0237] If the status determination data indicates an abnormality and the cause of the abnormality cannot be identified using the cause identification data acquired by the cause identification data acquisition processing unit, the status analysis processing unit generates a request signal to request the terminal device to send the data to be processed corresponding to the status determination data, which is data indicating an abnormality, to the server device.

[0238] The second communication interface transmits a request signal to the terminal device, and when the terminal device receives the request signal, it acquires the processing target data corresponding to the status determination data, which is data indicating an abnormality, stored in the terminal data storage unit, and transmits communication data including the processing target data to the server device via the first communication interface unit.

[0239] Then, the status analysis processing unit acquires processing target data corresponding to the status determination data, which is data indicating an abnormality contained in the communication data, and stores the acquired processing target data in the data storage unit as data with unknown causes.

[0240] As a result, in this data analysis processing system, when the status determination data indicates an abnormality, the data (data with unknown cause) when the cause of the abnormality cannot be identified using the data for determining the cause acquired by the data acquisition processing unit for determining the cause can be stored in the data storage unit. Furthermore, in this data analysis processing system, only when the cause is unknown data is the corresponding data to be processed transmitted from the terminal device to the server device, thereby reducing the amount of communication traffic.

[0241] A fifth invention is the third or fourth invention, wherein the feature analysis processing unit classifies the unknown-factor data stored in the data storage unit into m classes (m: natural number) that share a common cause of an abnormality, and assigns a label of the abnormality cause to each of the classified classes, thereby acquiring new learning data that is a set of m classes of processing target data; The N+1+m class classification trainable model takes new training data as input and has a configuration in which m nodes have been added to the output layer of the N+1 class classification trained model, and performs N+1+m class classification, which classifies data into a normal state class and N+m (N: natural number) abnormal state classes.The N+1+m class classification trainable model has a feature extraction unit and a class classification unit, and has a new feature analysis trained model, which is a model in which the class classification unit of the N+1+m class classification trained model, which is a trained model obtained by performing a learning process using new training data, has been deleted.

[0242] The factor identification data acquisition processing unit further has an i-th factor identification two-class classification trained model that uses i-th abnormal data, which is a set of data classified into the i-th class (i: natural number, N+1≦i≦N+1+m) when new learning data is input into the N+1+m class classification trained model, as learning data and performs two-class classification into whether an abnormal state has occurred or not due to the i-th factor, which is the cause of the abnormality in the data classified into the i-th class.

[0243] As a result, when data on an abnormal state with an unknown cause is acquired, this data analysis processing system can perform additional learning processing and install the trained model after the additional learning processing. Furthermore, this data analysis processing system can detect N+m abnormal states, including the newly added m causes of abnormal occurrence, thereby enabling even more accurate data analysis processing.

[0244] A sixth aspect of the present invention is a terminal device used in the data analysis processing system according to any one of the first to fourth aspects of the present invention.

[0245] This makes it possible to realize a terminal device used in the data analysis processing system according to any one of the first to fourth aspects of the present invention.

[0246] A seventh aspect of the present invention is a server device used in the data analysis processing system of any one of the first to fourth aspects of the present invention.

[0247] This makes it possible to realize a server device used in the data analysis processing system according to any one of the first to fourth aspects of the present invention.

[0248] The eighth invention is a data analysis processing method executed using a data analysis processing system that includes a terminal device and a server device and performs analytical processing on data that can distinguish between normal and abnormal states, and includes a state determination processing step, a feature analysis processing step, a data integration processing step, a first communication step, a second communication step, a cause identification processing step, and a state analysis processing step.

[0249] In the state determination processing step, the terminal device determines the state of the data to be processed, and acquires the result data of the determination as state determination data.

[0250] In the feature analysis processing step, the terminal device acquires feature amount data of the processing data from the processing target data.

[0251] In the data integration processing step, the terminal device acquires data including the state determination data and the feature amount data as integrated data.

[0252] In the first communication step, the terminal device transmits communication data including the integrated data to the server device.

[0253] In the second communication step, the server device receives the communication data transmitted from the terminal device.

[0254] In the factor identification processing step, the server device inputs the feature data contained in the communication data acquired in the second communication step, and acquires factor identification data from the feature data, which is data for identifying the cause of the occurrence of the abnormal state.

[0255] In the state analysis processing step, the server device performs a state analysis processing based on the state determination data and the cause identification data included in the communication data acquired in the second communication step.

[0256] This makes it possible to realize a data analysis processing method that has the same effects as the first aspect of the invention.

[0257] The ninth invention is a program for causing a computer to execute the data analysis processing method of the eighth invention.

[0258] This makes it possible to realize a program for causing a computer to execute a data analysis processing method that has the same effects as the eighth aspect of the invention.

[0259] A tenth invention is a trained model acquisition method for acquiring a trained model to be installed in a system having a first device and a second device, comprising a first step, a second step, a third step, a fourth step, a fifth step, and a sixth step.

[0260] The first step is to perform a learning process for a first trainable model that includes a feature extraction unit and a class classification unit, and obtain a first trained model that is a trained model of the first trainable model.

[0261] In the second step, a model in which the class classification unit is removed from the first trained model is obtained as the first split trained model.

[0262] The third step is to input the input data used in the learning process of the first learnable model into the first divided trained model, and obtain the output from the first divided trained model as the first learning data.

[0263] The fourth step is to perform a learning process for one or more class classification trainable models using the first training data, and obtain a class classification trained model.

[0264] The fifth step is to load the first divided trained model into the first device (for example, a training model in which optimal parameters (parameters obtained by the training process) for realizing the first divided trained model are set is loaded into a predetermined functional unit of the first device).

[0265] The sixth step is to load the class classification trained model into the second device (for example, a training model in which optimal parameters (parameters obtained by the training process) for realizing the class classification trained model are set is loaded into a predetermined functional unit of the second device).

[0266] This makes it possible to perform predetermined processing (processing realized using the first-division trained model and the classification trained model) by installing the first-division trained model in a first device (e.g., a terminal device), transmitting output data (feature data) of the first-division trained model to a second device (e.g., a server device) via a network, and performing processing using the classification trained model installed in the second device. Since the output from the first-division trained model installed in the first device cannot be obtained as meaningful data unless processing is performed using the classification trained model installed in the second device, security of the transmitted data can be ensured when the output data (feature data) of the first-division trained model is transmitted from the first device to the second device.

[0267] The first to sixth steps are each realized using, for example, a processor and a memory accessible from the processor.

[0268] An eleventh invention is a trained model acquisition method for acquiring a trained model to be installed in a system having a first device and a second device, comprising a first step, a second step, a third step, a fourth step, and a fifth step.

[0269] The first step is to perform a learning process for a first trainable model that includes a feature extraction unit and a class classification unit, and obtain a first trained model that is a trained model of the first trainable model.

[0270] In the second step, a model in which the class classification unit is removed from the first trained model is obtained as the first split trained model.

[0271] In the third step, the feature extraction unit is removed from the first trained model to obtain a model as the second split trained model.

[0272] The fourth step is to load the first divided trained model into the first device (for example, a training model in which optimal parameters (parameters obtained by the training process) for realizing the first divided trained model are set is loaded into a predetermined functional unit of the first device).

[0273] The fifth step is to load the second divided trained model into the second device (for example, a training model in which optimal parameters (parameters obtained by the training process) for realizing the second divided trained model are set is loaded into a predetermined functional unit of the second device).

[0274] This makes it possible to perform predetermined processing (processing realized using the first and second divided trained models) by installing the first divided trained model in a first device (e.g., a terminal device), transmitting output data (feature data) of the first divided trained model to a second device (e.g., a server device) via a network, and performing processing using the second divided trained model installed in the second device. Since the output from the first divided trained model installed in the first device cannot be obtained as meaningful data unless processing is performed using the second divided trained model installed in the second device, security of the transmitted data can be ensured when the output data (feature data) of the first divided trained model is transmitted from the first device to the second device.

[0275] The first to fifth steps are each realized using, for example, a processor and a memory accessible from the processor.

[0276] A twelfth invention is a program for causing a computer to execute the trained model acquisition method of the tenth or eleventh invention.

[0277] This makes it possible to realize a program for causing a computer to execute a trained model acquisition method that has the same effects as the tenth or eleventh invention. [Explanation of symbols]

[0278] 1000, 2000, 2000A Data Analysis and Processing System 100, 100A, 100B terminal equipment 200, 200A, 200B server equipment 11. State determination processing unit 12 Feature analysis processing unit 13, 13A, 13B Data integration processing section 14, 14A First communication interface 15. Terminal data storage unit 21, 21A, 21B Second communication interface 22 Data acquisition processing unit for factor identification 23, 23A, 23B Status analysis processing unit DB1 data storage

Claims

1. A data analysis processing system that performs analysis processing on data that can distinguish between a normal state and an abnormal state, A terminal device; a server device; Equipped with The terminal device a state determination processing unit that determines a state of the data to be processed and acquires result data of the determination as state determination data; a feature analysis processing unit that acquires feature amount data of the processing data from the processing target data; a data integration processing unit that acquires data including the state determination data and the feature amount data as integrated data; a first communication interface unit that transmits communication data including the integrated data to the server device; Equipped with The server device a second communication interface unit for receiving communication data transmitted from the terminal device; a factor identification data acquisition processing unit that receives the feature amount data included in the communication data acquired by the second communication interface unit and acquires factor identification data, which is data for identifying a factor of the occurrence of an abnormal state, from the feature amount data; a status analysis processing unit that performs a status analysis process based on the status determination data and the cause identification data included in the communication data acquired by the second communication interface unit; Equipped with Data analysis processing system.

2. The feature analysis processing unit an N+1 class classification trainable model that performs N+1 class classification, which uses the processing target data as input and classifies the data into a normal state class and N (N: natural number) abnormal state classes, and the N+1 class classification trainable model includes a feature extraction unit and a class classification unit, and the N+1 class classification trainable model has a feature analysis trained model that is a model obtained by removing the class classification unit of an N+1 class classification trained model that is a trained model obtained by performing a learning process using normal state data and abnormal state data of the processing target data, The factor identification data acquisition processing unit a kth factor identification two-class classification trained model that performs two-class classification on the kth abnormal data, which is a set of data classified into the kth class (k: natural number, 1≦k≦N) when the abnormal state data of the processing target data is input to the N+1 class classification trained model, as learning data, to determine whether an abnormal state has occurred or not due to a kth factor that is a cause of an abnormality in the data classified into the kth class; The data analysis processing system according to claim 1 .

3. further comprising a data storage unit for storing data; The data integration processing unit If the state determination data is data indicating an abnormal state, data including the state determination data, the feature amount data, and the processing target data is acquired as integrated data; The state analysis processing unit When the state determination data indicates an abnormality, if the cause of the abnormality cannot be identified using the cause identification data acquired by the cause identification data acquisition processing unit, the processing target data included in the communication data acquired by the second communication interface unit is stored in the data storage unit as cause-unknown data. The data analysis processing system according to claim 2 .

4. The terminal device Further comprising a terminal data storage unit, The data integration processing unit If the state determination data indicates an abnormality, the processing target data is stored in the terminal data storage unit; The state analysis processing unit When the state determination data indicates an abnormality and the cause of the abnormality cannot be identified using the cause identification data acquired by the cause identification data acquisition processing unit, a request signal is generated to request the terminal device to transmit the processing target data corresponding to the state determination data indicating an abnormality to the server device; The second communication interface unit transmitting the request signal to the terminal device; When the terminal device receives the request signal, it acquires the processing target data corresponding to the status determination data, which is data indicating an abnormality and is stored in the terminal data storage unit, and transmits communication data including the processing target data to the server device via the first communication interface unit; the status analysis processing unit acquires the processing target data corresponding to the status determination data, which is data indicating an abnormality included in the communication data, and stores the acquired processing target data in the data storage unit as data with an unknown cause; The data analysis processing system according to claim 2 .

5. The feature analysis processing unit classifying the unknown-factor data stored in the data storage unit into m classes (m: natural number) that share a common cause of an abnormality, and assigning a label of the abnormality cause to each of the classified classes, thereby acquiring new learning data that is a collection of m classes of processing target data; the N+1+m class classification trainable model has a configuration in which the new training data is input and the number of nodes in the output layer of the N+1 class classification trained model is added by m, the N+1+m class classification trainable model performs N+1+m class classification that classifies into a normal state class and N+m (N: natural number) abnormal state classes, and the N+1+m class classification trainable model has a new feature analysis trained model that is a model obtained by removing the class classification unit of the N+1+m class classification trained model, which is a trained model obtained by performing a learning process using the new training data, with respect to the N+1+m class classification trainable model that includes a feature extraction unit and a class classification unit; The factor identification data acquisition processing unit The system further includes an i-th factor identification two-class classification trained model that uses i-th abnormal data, which is a set of data classified into the i-th class (i: natural number, N+1≦i≦N+1+m) when the new training data is input to the N+1+m class classification trained model, as training data and performs two-class classification into whether an abnormal state has occurred or not due to an i-th factor, which is a cause of an abnormality in the data classified into the i-th class.

5. The data analysis processing system according to claim 3 or 4.

6. 5. The terminal device used in the data analysis processing system according to claim 1.

7. The server device used in the data analysis processing system according to any one of claims 1 to 4.

8. A data analysis processing method executed by a data analysis processing system that includes a terminal device and a server device and performs analysis processing on data that can distinguish between a normal state and an abnormal state, a status determination processing step in which the terminal device determines the status of the data to be processed and acquires result data of the determination as status determination data; a feature analysis processing step in which the terminal device acquires feature amount data of the processing data from the processing target data; a data integration processing step in which the terminal device acquires data including the state determination data and the feature amount data as integrated data; a first communication step in which the terminal device transmits communication data including the integrated data to the server device; a second communication step in which the server device receives communication data transmitted from the terminal device; a factor identification processing step in which the server device inputs the feature amount data included in the communication data acquired in the second communication step, and acquires factor identification data, which is data for identifying a factor of the occurrence of an abnormal state, from the feature amount data; a state analysis processing step in which the server device performs a state analysis processing based on the state determination data and the cause identification data included in the communication data acquired in the second communication step; A data analysis processing method comprising:

9. A program for causing a computer to execute the data analysis processing method according to claim 8.

10. A trained model acquisition method for acquiring a trained model to be installed in a system including a first device and a second device, a first step of performing a learning process for a first learnable model including a feature extraction unit and a class classification unit, and acquiring a first trained model that is a trained model of the first learnable model; a second step of acquiring a model in which the class classification unit is deleted from the first trained model as a first divided trained model; A third step of inputting input data used in the learning process of the first trainable model into the first divided trained model and obtaining output from the first divided trained model as first learning data; a fourth step of performing a learning process for one or more class classification trainable models using the first learning data to obtain a class classification trained model; a fifth step of loading the first divided trained model into the first device; a sixth step of loading the class classification trained model into the second device; A method for obtaining a trained model comprising:

11. A trained model acquisition method for acquiring a trained model to be installed in a system including a first device and a second device, a first step of performing a learning process for a first learnable model including a feature extraction unit and a class classification unit, and acquiring a first trained model that is a trained model of the first learnable model; a second step of acquiring a model in which the class classification unit is deleted from the first trained model as a first divided trained model; a third step of acquiring a model obtained by deleting the feature extraction unit from the first trained model as a second divided trained model; a fourth step of loading the first divided trained model into a first device; A fifth step of installing the second divided trained model in a second device; A method for obtaining a trained model comprising:

12. A program for causing a computer to execute the trained model acquisition method according to claim 10 or 11.

Images