Learning device, inference device, learning method, and inference method

Abstract

A learning device (AS) comprises an acquisition unit (SH) that acquires a plurality of first data constellations such that each of the data constellations is supposed to include pieces of data and the contents and order of the pieces of data are identical among the plurality of data constellations, and a learning unit (GA) that generates a learned model (GM) in which the plurality of data constellations are assigned with a plurality of first labels (LO, PR) at a first level and the pieces of data are assigned with a plurality of second labels (PR, EV) different from the plurality of first labels (LO, PR, EV) at a second level lower than the first level.

Description

[Technical Field]

[0001] This invention relates to a learning device, an inference device, a learning method, and an inference method. [Previous Technology]

[0002] In patent document 1, which is common in the production of teacher data for constructing a learned model, a notation device is provided for the data arranged in a time series, i.e., the time series data, in order to more accurately attach the desired information to the aforementioned time series data, including a first notation and a second notation. (Prior art documents, patent documents)

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-159819 [Summary of the Invention]

[0004] Summary of the Invention: Problem to be Solved by the Invention

[0005] The above-mentioned annotation device affixes a first mark to the first time interval in the aforementioned time series, and on the other hand, affixes the aforementioned second mark to the second time interval in the aforementioned time series that is located later than the aforementioned first time interval.

[0006] The object of the present invention is to provide a learning device, inference device, learning method, and inference method that can achieve at least one of the following: (1) Simultaneously mastering both the broad and partial mastery of the aforementioned time series data. (2) When some data in the aforementioned time series data are similar to each other, assigning different labels to the data. (3) Distinguishing between whether the labeling of the aforementioned time series data is inappropriate and whether the aforementioned time series data itself is inappropriate. More specifically, for example, even when the classifier has high accuracy, the user must verify the labeling results, which increases the user verification workload when the time series data is quite large. Means for solving the problem

[0007] To solve the above-mentioned problems, the learning device of the present invention includes: an acquisition unit that acquires a first plurality of data groups, each of which should have a plurality of data items and each data item should have a plurality of partial data items, and the content and order of the plurality of data items should be the same among the plurality of data groups; and a learning unit that generates a learned model in which a first plurality of tags are attached to the plurality of data groups at a first level, and a second plurality of tags, different from the first plurality of tags, are attached to the plurality of data items at a lower level, i.e., a second level. Effects of the Invention

[0008] If the learning device according to the present invention is used, at least one of the effects of (1) to (3) above can be achieved.

Implementation Method

[0010] Form used to implement the invention

[0011] An embodiment of the annotation device of the present invention will be described.

[0012] Embodiment 1. <Embodiment 1> The annotation device AS of Embodiment 1 will be described.

[0013] In the following, for ease of explanation and understanding, there will be cases where a single symbol is used to refer to multiple names. For example, there will be cases where a single symbol "lot number LO" is used to refer to the first batch number LO1, the second batch number LO2, the third batch number LO, etc.

[0014] <Function of Embodiment 1> Figure 1 is a functional block diagram of the annotation device AS of Embodiment 1.

[0015] Figure 2 is a timetable showing the time series data of Implementation Form 1.

[0016] Figure 3 shows the hierarchical structure of implementation form 1 (the first one).

[0017] Figure 4 shows the hierarchical structure of implementation form 1 (the second one).

[0018] The function of the annotation device AS in Embodiment 1 will be described with reference to Figures 1 to 4.

[0019] The annotation device AS of Embodiment 1, as shown in Figure 2, assigns a mark to each level of time series data (as shown in Figure 2) under a three-level structure (large level, medium level, and small level), as shown in Figure 3. More specifically, the large level is marked with batch number 1 (LO), batch number 2 (LO), etc.; the medium level is marked with process 1 (PR1), process 2 (PR2), etc.; and the small level is marked with event 1 (EVa1), event 2 (EVa2), etc. To perform the above assignment, the annotation device AS of Embodiment 1 includes, as shown in Figure 1: an acquisition unit SH, a display unit HY, a correct solution generation unit SE, an expansion unit KA, a learning unit GA, an annotation unit AN, and a level structure generation unit KS.

[0020] The annotation part AN further includes, as shown in Figure 1, a middle-level mark prediction part CS, a middle-level mark attachment part CF, a small-level mark attachment part SF, and a large-level mark attachment part DF.

[0021] The annotation device AS corresponds to "learning device" and "inference device".

[0022] The acquisition part SH corresponds to "acquisition part", the learning part GA corresponds to "learning part", the middle-level mark attachment part CF to the upper-level mark attachment part DF corresponds to "attachment part", the level structure generation part KS corresponds to "generation part", the expansion part KA corresponds to "expansion part", and the middle-level mark attachment part CF corresponds to "correction part".

[0023] The time series data, as shown in Figure 2, should consist of batch number 1 (LO1), batch number 2 (LO2), batch number 3 (LO3), ... . Batch number 1 (LO1), as shown in Figure 2, should contain 11 individual processes (PR), that is, it should contain processes PR1 to PR11. The same applies to batch numbers 2 (LO2), 3 (LO3), ... . Note that the time series data is not initially labeled with batch number (LO), process (PR), or event (EV) as described later.

[0024] In addition to the above, the content of processes PR1 to PR11 in a fixed order should be the same between batch numbers LO, i.e., between batch number LO1, batch number LO2, batch number LO3, ... . For example, the content of process PR1 in batch number LO1 should be the same as the content of process PR1 in batch number LO2. In other words, batch numbers LO containing the content of processes PR1 to PR11 should be repeated. In other words, the time series data covers multiple batch numbers LO, and multiple processes PR should have a correct order and be repeated. However, for example, the content of the same process PR does not have to be the same. For example, the content of process PR1 in batch number LO1 does not have to be the same as the content of process PR1 in batch number LO2. More specifically, the value of process PR1 in batch number LO1 does not have to be exactly the same as the value of process PR1 in batch number LO2.

[0025] The hierarchical structure is shown in Figure 3, for example, including three hierarchical levels: "Major Hierarchy (Batch Number LO)," "Middle Hierarchy (Process PR)," and "Minor Hierarchy (Event EV)." As shown in Figure 4, it can also include four hierarchical levels: "Major Hierarchy (Factory FA)," "Major Hierarchy (Batch Number LO)," "Middle Hierarchy (Process PR)," and "Minor Hierarchy (Event EV)." The hierarchical structure can have two or fewer hierarchical levels, or five or more hierarchical levels.

[0026] In the time series data, the batch number LO corresponds to "data group", the process PR corresponds to "data", and the event EV corresponds to "partial data".

[0027] Two of the following: “high class”, “middle class” and “low class”. For example, “high class” and “middle class” correspond to “first class” and “second class”.

[0028] For example, the first batch number LO1, the second batch number LO2, ... correspond to "the first multiple mark", and the first process PR1, the second process PR2, ... correspond to "the second multiple mark".

[0029] Returning to Figure 1, the function of the annotation device AS is explained.

[0030] Obtained information on the time series of data from the Ministry of Industry and Information Technology (MIIT).

[0031] The display unit HY displays time series data in whole or in part as necessary.

[0032] The correct solution generation unit SE generates teacher data by attaching a label (correct solution label) to each level under the multiple-level structure of time series data.

[0033] The Development Department KA has significantly increased the amount of learning materials in order to increase the variety of learning materials.

[0034] The learning unit GA generates a learning model GM (not shown) by sliding a window with a specific window width (for example, as shown in Figure 15) to learn for mid-level classification labels.

[0035] The annotation part AN is marked on multiple levels, namely the high level, the middle level and the low level.

[0036] Within the annotation unit AN, the intermediate-level label inference unit CS uses the learning model GM generated by the learning unit GA to infer labels in the intermediate level.

[0037] In the annotation section AN, the intermediate level mark is attached to the section CF to correct the learning error according to the learning section GA, and the anomaly in the time series data is attached with a mark for the specific anomaly.

[0038] Within the annotation section AN, the sub-level marker attachment section SF obtains the range of the sub-level to which the sub-level belongs, and determines whether each obtained interval has periodicity. For the interval that is presumed to have periodicity, the sub-level marker attachment section SF divides it into periodic units (matching the units of the sub-level) and attaches a marker to the sub-level.

[0039] Within the annotation section AN, the major hierarchy marker is attached to the section DF according to the number of hierarchies contained within the major hierarchy, and the major hierarchy marker is attached to the major hierarchy.

[0040] The hierarchical structure generation unit KS generates the above-mentioned hierarchical structure.

[0041] <Hardware configuration of embodiment 1> Figure 5 shows the hardware configuration of the annotation device AS of embodiment 1.

[0042] In order to achieve the above-mentioned functions, the annotation device AS in the embodiment includes: processor PR, memory ME, and memory medium KI, and further includes: input unit NY and output unit SY as necessary.

[0043] The processor PR is the core of a computer that knows how to make hardware operate based on software. The memory ME is composed of, for example, DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory). The memory media KI is composed of, for example, a hard disk drive (HDD), a solid-state drive (SSD), and ROM (Read Only Memory). The memory media KI stores the program PRG. The program PRG is a group of commands that specifies the processing content that the processor PR should execute.

[0044] The input unit NY may consist of, for example, a camera, microphone, keyboard, mouse, and touch panel. The output unit SY may consist of, for example, an LCD screen, printer, and touch panel.

[0045] Regarding the relationship between the functions and hardware configuration in the annotation device AS, the processor PR executes the program PRG stored in the memory medium KI on the memory ME, and as necessary, the functions of each part from the acquisition unit SH to the hierarchical structure generation unit KS are realized by controlling the operation of the input unit NY and the output unit SY.

[0046] <Operation of Embodiment 1> <Operation of Correct Solution Generator SE> Figure 6 is a flowchart showing the operation of the annotation device AS in Embodiment 1.

[0047] Step ST11: Acquisition Unit SH (shown in Figure 1) acquires time series data (shown in Figure 2). The time series data is acquired, for example, from machines or sensors configured on the manufacturing line in the factory.

[0048] The correct solution generation unit SE (shown in Figure 1) extracts a portion of the acquired time series data (e.g., batch number LO1) and generates teacher data by attaching a mark (the mark of the correct solution) under the hierarchical structure described above.

[0049] The above-mentioned extraction is the same as that known, for the above-mentioned time series data, the user uses a graphical user interface (GUI) to extract according to the specified time (start time, end time) and range (interval), and uses a file that defines the range to be extracted to replace the graphical user interface to extract the specified range.

[0050] <Segmentation Action> Figure 7 shows the timeline of the segmentation action in implementation mode 1.

[0051] Step ST12: As is common practice, as shown in Figure 7, for example, based on the shape (waveform) of the time series data, the range extracted from the time series data is divided into a plurality of intervals. In Figure 7, solid lines indicate the positions that the annotation device AS intends to divide, which are consistent with the positions that should be divided according to control information (e.g., information from the manufacturing line in the factory). On the other hand, dashed lines indicate that the two positions are inconsistent.

[0052] Among them, the control information includes, for example, the number of process PRs in the batch number LO, the start time of the process PR, and the end time of the process PR.

[0053] Figure 8 is a timetable showing the actions of correcting the segmented position in implementation mode 1.

[0054] Step ST13: The position to be segmented by the annotation device AS may not necessarily be consistent with the position to be segmented by the user of the annotation device AS or the aforementioned control information. Therefore, the segmentation position needs to be corrected. The correction is the same as conventional, for example, it can be performed by the user deleting the positions shown by the dotted lines DL1, DL2, DL3, and DL4.

[0055] It can be clearly seen from the comparison of Figure 7 and Figure 8 that the multiple intervals after the correction of the segmentation position are marked, that is, marked as "Process 1 PR1" to "Process 11 PR11".

[0056] Figure 9 shows the timeline for dividing Implementation Form 1 into smaller layers.

[0057] Step ST14: For the above multiple intervals, use autocorrelation graphs, etc., to determine whether there is periodicity. As shown in Figure 8, the intervals determined to be periodic (process 3 PR3, process 7 PR7) are divided into smaller units (event EV units).

[0058] More specifically, as shown in Figure 9, for example, the third process PR3 is divided into a plurality of events EV, namely, events EV1 to EV6. The seventh process PR7 is also divided in the same way.

[0059] Step ST15: By grouping the middle level (process PR) into multiple units (11 process PRs), dividing them into units that match the large level (lot number LO), and attaching the label "lot number LO".

[0060] Figure 10 is an example of the tag table and hierarchical structure JSON file of implementation form 1.

[0061] Step ST16: By integrating the large hierarchy (lot number LO), medium hierarchy (process PR), and small hierarchy (event EV), as shown in Figure 10, a mark table and hierarchy structure are generated.

[0062] In Figure 10, "-1" indicates that the process PR does not have the event EV. Figure 10 shows the hierarchical structure of batch number LO, process PR, and event EV, and also shows the time (start time and end time, e.g., the horizontal axis of Figure 2) and value (e.g., the vertical axis of Figure 2) of batch number LO, process PR, and event EV.

[0063] The generated label table and hierarchical structure are transmitted to the expansion section KA and the learning section GA.

[0064] <Action of the unfolding part KA> Figure 11 is a flowchart showing the action of unfolding data in implementation mode 1.

[0065] When the expansion unit KA receives the mark (the mark of the solution) and the hierarchical structure from the solution generation unit SE, it performs data expansion using the time series data and hierarchical structure used by the solution generation unit SE.

[0066] Step ST21: Obtain the level of the expanded data (e.g., a to c below). "a" refers to the level of the middle layer, "b" refers to the level of the small layer, and "c" refers to the level of the frame. "c" can be specified alone, or when specifying "a" or "b", both "c" and "a" can be specified together, and similarly, both "c" and "b" can be specified together.

[0067] Figure 12 shows the correspondence between the level of unfolding in implementation form 1 and the action of unfolding data.

[0068] Which of the plurality of actions (steps ST21 to ST24) shown in Figure 11 should be performed, as shown in Figure 12, is selected based on the level of expansion.

[0069] Figure 13 shows the range of the solution generation unit SE used in Embodiment 1.

[0070] Step ST22: Randomly select units of the middle level (process PR) from the plurality of large levels (lot numbers LO) used by the solution generation unit SE as shown in Figure 13, and rearrange them according to the order of the middle level (process PR).

[0071] In detail, as shown in Figure 13A, when the range used by the correct solution generation unit SE is batch number LO1 (including process PR1 to process PR11) and batch number LO2 (including process PR1 to process PR11), processes PR1 to PR11 of batch number LO1 and processes PR1 to PR11 of batch number LO2 are randomly selected and combined into processes PR1 to PR11.

[0072] Specifically, in Example 1 (shown in Figure 13B), the first process PR1 in batch 1 LO1, the second process PR2 in batch 2 LO2, the third process PR3 in batch 2 LO2, the fourth process PR4 in batch 2 LO2, the fifth process PR5 in batch 1 LO1, ... the eleventh process PR11 in batch 2 LO2 are combined.

[0073] In Example 2 (shown in Figure 13C), the first process PR1 of the second batch LO2, the second process PR2 of the first batch LO1, the third process PR3 of the first batch LO1, the fourth process PR4 of the second batch LO2, the fifth process PR5 of the second batch LO2, ... the eleventh process PR11 of the first batch LO1 are combined.

[0074] As shown or implied in Figures 13B and 13C, both Expanded Example 1 and Expanded Example 2 are labeled with "Process 1 PR1" to "Process 11 PR11" in this order.

[0075] Step ST23: In the case that there are minor levels (events EV) in the middle level (process PR), randomly select and arrange the minor levels (events EV).

[0076] Specifically, in the third example (shown in Figure 13D), in the third process PR3, the first event EV1 in the first batch LO1, the fifth event EV5 in the second batch LO2, the third event EV3 in the second batch LO2, and the second event EV2 in the first batch LO1 are combined.

[0077] Step ST24: Perform scaling and noise addition, etc., to change the values ​​of the expanded data. The expanded data is then transmitted to the Learning Department (GA).

[0078] <The Actions of the Learning Department GA> Figure 14 is a flowchart showing the learning actions of the Learning Department GA in Implementation Form 1.

[0079] In the learning unit GA, when a portion of the time series data (with tags) extracted by the correct solution generation unit SE and the data (with tags) expanded by the expansion unit KA are received, learning data for the tags of the matching middle level (process PR) is generated according to the sliding window with a specific window width (shown in Figure 15), and the tags for classifying the matching middle level (process PR) are learned.

[0080] Figure 15 shows the timeline of the sliding window in Embodiment 1.

[0081] Step ST31: For each piece of data extracted by the forward solution generation unit SE and the data expanded by the expansion unit KA, as shown in Figure 15, while moving a sliding window with a specific window width, learning data for classifying the middle hierarchy (process PR) is generated.

[0082] Among the multiple markers (e.g., process 3 PR3, process 4 PR4, process 5 PR5) covered by the sliding window, the marker located in the center (process 4 PR4) is the positive value.

[0083] Step ST32: Using the data segmented into each sliding window and the labels of the matched hierarchical levels (process PR), and employing machine learning techniques such as SVM (Support Vector Machine) and neural networks, as is commonly known, a learning model GM is generated. The learning model GM is transmitted to the annotation unit AN.

[0084] <Action of Annotation Section AN> Annotation Section AN is marked for all levels, that is, marked for the large level, medium level and small level (lot number LO, process PR, event EV).

[0085] Figure 16 is a flowchart showing the operation of the annotation part AN in embodiment 1.

[0086] In the annotation unit AN, the intermediate-level label inference unit CS uses the learning model GM generated by the learning unit GA to infer the matching intermediate-level labels (process PR).

[0087] Step ST41: Obtain time series data of the range of markers to be inferred in the middle level, and generate test data while moving a sliding window with a specific window width.

[0088] Step ST42: Apply the learning model GM generated by the learning department GA to the test data, and infer the markers of the middle-level data located in the center of each window (process PR). Transmit the inferred results to the middle-level marker attachment department CF.

[0089] When the middle-level marker appending unit CF receives the marker prediction result of the middle-level based on the middle-level marker prediction unit CS, it corrects the error of the prediction result and appends a marker for the specific anomaly at the anomaly point in the time series data.

[0090] <Actions for error correction> Figure 17 is a flowchart showing the error correction actions in implementation mode 1.

[0091] Step ST51: Using the results of the attached labels generated by the forward solution generator SE and the hierarchical structure, calculate the unit average length of each set of the large and medium hierarchical levels (e.g., the average length of batch number LO, the average length of process PR).

[0092] Step ST52: For the prediction results of the middle-level (process PR) received from the middle-level mark prediction unit CS, extract the places where the mark values ​​are the same and continuous.

[0093] Figure 18 is a timetable showing the error correction actions (1) of implementation mode 1.

[0094] Figure 19 shows the timeline of the error correction actions (2) of implementation mode 1.

[0095] Figure 19 is an enlarged view of the vicinity of m1 in Figure 18.

[0096] In the 100th batch number LO100, as shown in Figure 18, for example, the value marked "1" at c1 is continuous, the value marked "2" at c2 is continuous, and the value marked "3" at c3 is continuous.

[0097] As shown in Figure 19, the value "7" marked at m1 following c3 is continuous. The value "3" marked at c4 following m1 is continuous.

[0098] As shown above, the values ​​of the extracted markers are the same and consecutive.

[0099] Step ST53: Using the unit average length of the aforementioned calculated sets of the large level (lot number LO) and the medium level (process PR), where the value of the aforementioned extracted marker is continuous, extract the points where the marker is suspected to be incorrect, and correct the value of the marker.

[0100] For the consecutive values ​​of the markers c1, c2, c3, m1, c4... (as shown in Figures 18 and 19), the values ​​of the markers are compared sequentially.

[0101] Because the value of the mark at c1 is "1" and the length at c1 is approximately the same as the average length of the first process PR1, c1 is identified as the mark "first process PR1".

[0102] Similarly, since the value of the mark at c2 is "2" and the length at c2 is approximately the same as the average length of the second process PR2, c2 is identified as the mark "second process PR2".

[0103] Figure 20 shows the timeline of the error correction action (3) of implementation mode 1.

[0104] Although the above two identifications were performed on c1 and c2, since the length of c3 following c2 is shorter than the average length of the third process PR3 following the second process PR2, as shown in FIG20, a provisional label indicating an error is attached after c3.

[0105] After the above two identifications for c1 and c2 and the application of the provisional label for c3, since the value of the label at m1 following c3 is "7", under the condition that an error is suspected, the false label of the third process PR3 is applied to m1 as shown in FIG20.

[0106] Although the value of the mark at c4 following m1 is "3", since the length of c4 is shorter than that of the fourth process PR4 following the third process PR3, a provisional mark with an error is attached near c4.

[0107] As shown above, a false label "3rd process PR3" indicating the correct result is attached to the position m1 where it is presumed to be wrong. On the other hand, a provisional label indicating "error" is attached before and after m1, namely at positions c3 and c4.

[0108] When the processing of c1, c2, c3, m1, c4, ... in batch number 100 LO100 is completed, the same processing is then performed on batch number 101 LO101, batch number 102 LO102, ...

[0109] The m1 location (presumably the erroneous location) with the false label "Process 3 PR3" is adjacent to the c3 and c4 locations with the erroneous provisional labels. Therefore, the m1 location is corrected according to the false label "Process 3 PR3". The same correction is made for the other m2, m3, m4, m5... (as shown in Figure 18). For example, the label "Process 7 PR7" is affixed to the m2 and m3 locations in batch number 100 LO100, and the label "Process 3 PR3" is affixed to the m4 location in batch number 101 LO101 and the m5 location in batch number 102 LO102.

[0110] Step ST54: Attach an anomaly marker to the anomaly in the data.

[0111] As shown in Figure 18, the difference between the value "6" marked at e1 and the value "8" marked at e2 is not "1". However, since the length at e1 is approximately the same as the average length of process PR6 in the 6th process, no erroneous provisional label is attached to e1. ​​Similarly, since the length at e2 is approximately the same as the average length of process PR8 in the 8th process, no erroneous provisional label is attached to e2.

[0112] As shown above, although no erroneous provisional flags are attached at e1 and e2, when the difference in the process flags is not "1", it is speculated that there is an anomaly near e1 and e2. Therefore, flags indicating an anomaly are attached at e1 and e2 (e.g., the value "-1" which is not used in the process flags).

[0113] As described above, when there is an error in the mid-level mark appending unit CF in the mid-level mark inference unit CS, the error is corrected, and a mark for the specific abnormality is appended to the abnormality in the data. The result of the mark appending based on the mid-level mark appending unit CF is transmitted to the small-level mark appending unit SF, the large-level mark appending unit DF, and the level structure generation unit KS.

[0114] <Action of the Small-Level Marker Attachment Unit SF> In the annotation unit AN, the small-level marker attachment unit SF uses the hierarchical structure generated by the correct solution generation unit SE and the result of attaching a mark by the middle-level marker attachment unit CF to obtain the range of the middle-level (process PR) to which the small-level belongs, and determines whether each obtained interval is periodic. When the small-level marker attachment unit SF finds that the interval is periodic, it divides the interval into periodic units (units of the small-level) and attaches a mark to the small-level (event EV).

[0115] Figure 21 is a flowchart showing the operation of the small-level mark attachment part SF in implementation form 1.

[0116] Step ST61: Obtain the process PR containing the event EV from the hierarchical construction of the correct solution. Specifically, obtain the third process PR3 containing the event EV and the seventh process PR7 containing the event EV (as shown in Figure 9, for example).

[0117] Step ST62: Receive the labeling results of process PRs attached by the middle-level labeling attachment unit CF, and extract the third process PR3 and the seventh process PR7. The extracted third process PR3 and seventh process PR7 are determined to have periodicity in the same way as conventional methods (e.g., using an autocorrelation graph). If periodicity is present, the process is divided into periodic units (units of event EVs), and each divided interval is labeled with an event EV (e.g., the first event EVa1, the first event EVb1, etc., as shown in Figure 28). The labeling results of the lower-level hierarchy are transmitted to the hierarchy construction generation unit KS.

[0118] <Action of the Large-Level Marker Attachment Unit DF> In the annotation unit AN, when the large-level marker attachment unit DF obtains the number of levels contained in the large-level from the solution generation unit SE, the mark of the large-level is attached.

[0119] Figure 22 is a flowchart showing the operation of the large-level mark attachment part DF in implementation form 1.

[0120] Figure 23 is a timeline showing the results of correcting the errors in Implementation 1.

[0121] Step ST71: Obtain the number of process PRs contained in batch number LO, and form a set from the start mark to the end mark of the process PRs in batch number LO to distinguish each batch number LO.

[0122] Therefore, the number of processes PR within batch number LO is "11". Therefore, when extracting the range from process PR1 to process PR11, as shown in Figure 23, the first interval PE1, the second interval PE2, and the third interval PE3 can be obtained.

[0123] In cases of anomalies in the time series data, there is a possibility that processes PR2 to PR10 (second process) are missing in the middle of batch number LO, process PR1 (first process) is missing at the beginning of batch number LO, or process PR11 (eleventh process) is missing at the end of batch number LO. For example, if only processes PR2 to PR7 (second process) exist in batch number LO, these processes PR2 to PR7 cannot be processed as a set and will remain in this state.

[0124] Step ST72: For the range enclosed in the first interval PE1 to the third interval PE3, i.e. the residual part mentioned above, each becomes a set. For example, the second process PR2 to the seventh process PR7 mentioned above become a set.

[0125] Step ST73: Start from the beginning of the time series data and sequentially attach the batch number LO mark.

[0126] The result of attaching the large hierarchy to the hierarchy construction generation unit KS is transmitted.

[0127] <Operation of the Hierarchical Structure Generator KS> When the hierarchical structure generator KS receives the result of the attachment mark of the middle hierarchical level from the middle hierarchical level attachment part CF, receives the result of the attachment mark of the small hierarchical level from the small hierarchical level attachment part SF, and receives the result of the attachment mark of the large hierarchical level from the large hierarchical level attachment part DF, it generates the hierarchical structure of the large hierarchical level, the middle hierarchical level, and the small hierarchical level.

[0128] In Figure 2, for example, suppose that an abnormal shape occurs in event EVA1. The user can use the above-described hierarchical structure to specify, for example, the third process PR3 containing event EVA1, and the second process PR2 and the fourth process PR4 before and after the third process PR3. In this way, the user can, for example, visually identify and compare the second process PR2 to the fourth process PR4 in batch 1 LO1 (as shown in Figure 2) with the second process PR2 to the fourth process PR4 in batch 100 LO100 (for example, as shown in Figure 25).

[0129] Suppose that without the above-mentioned hierarchical structure, it is impossible to specify the second process PR2 and the fourth process PR4, resulting in the inability to visually identify the range of the second process PR2 to the fourth process PR4.

[0130] Figure 24 is a flowchart showing the operation of the hierarchical structure generation unit KS in implementation form 1.

[0131] Figure 25 is a timetable showing the operation of the hierarchical structure generation unit KS in implementation form 1.

[0132] Step ST81: For each set of the large hierarchy, obtain the middle hierarchy contained in the large hierarchy. As shown in Figure 25, for batch number LO100, obtain the first process PR1 to the eleventh process PR11 contained in the start and end times. The same applies to other batch numbers LO, such as batch number LO101, batch number LO102, etc.

[0133] Step ST82: For each intermediate level already obtained, obtain the sub-levels contained within the intermediate level. More specifically, by extracting the range containing event EV from the range of the already obtained process 1 PR1 to process 11 PR11, process 3 PR3 and process 7 PR7 are obtained (as shown in Figure 9, for example). The same applies to other batch numbers LO.

[0134] Step ST83: Generate a hierarchical structure of large, medium and small classes.

[0135] Figure 26 shows a portion of the hierarchical structure of implementation form 1.

[0136] In Figure 26, the first event EVA1 contained in the third process PR3 (as shown in Figure 2, for example) is denoted as the first event contained in the third process PR3, "event_a_01". Other events EV can also be denoted in the same way.

[0137] <Modified Example> Unlike the first real form 1, which corrects the division based on the correct solution generation unit SE according to the control information (as shown in Figures 7 and 8), it is also possible to perform the division based on the user's instructions (for example, the user records the position to be divided in the file) instead of using control information.

[0138] Different from the implementation form 1 which is based on the so-called large level (lot number LO), medium level (process PR), and small level (event EV), as shown in Figure 4, the so-called large level (factory FA, ​​lot number LO), medium level (process PR), and small level (event EV) can also be used.

[0139] Regarding the hierarchy shown in Figure 4, during step ST73 of Figure 22, the large-level labeling unit DF sequentially labels the layer (batch number LO) closest to the middle level within the large hierarchy (factory FA, ​​batch number LO), starting from the beginning of the object to be inferred, i.e., the time series data (shown in Figure 2). After the aforementioned labeling, using the result of the batch number LO labeling and the hierarchy structure generated by the solution generation unit SE, units that aggregate multiple batch number LOs are distinguished and labeled "factory FA". When aggregating several batch number LOs, the information of the hierarchy structure generated by the solution generation unit SE is used.

[0140] As shown above, for the hierarchy shown in Figure 4, in the larger hierarchy, namely factory FA and batch number LO, from the batch number LO which is closer to the middle hierarchy, namely process PR, to factory FA, ​​each level is marked. The same applies to the middle and smaller hierarchical levels.

[0141] Unlike implementation form 1, which gathers the tags of each level (lot number LO), middle level (process PR), and small level (event EV) into a single file (table), the tags of each level can also be gathered in other output forms.

[0142] Unlike implementation form 1, which uses JSON files to describe hierarchical structures, other file formats can also be used for description.

[0143] Unlike implementation form 1, which generates a hierarchical structure by attaching labels to the large, medium, and small hierarchical levels, it is also possible not to generate a hierarchical structure that aggregates the labels of each level after attaching labels to each level. For example, the hierarchical structure generation unit KS (shown in Figure 1) can be omitted, and the processing of step ST16 of the solution generation unit SE can also be omitted.

[0144] Unlike implementation form 1, which greatly increases the amount of learning data by expanding the data, it is also possible not to expand the data, and the expansion part KA can be omitted (as shown in Figure 1).

[0145] Unlike embodiment 1 which has a display unit HY, the display unit HY can be omitted when the user does not use a graphical user interface (GUI) for processing (as shown in FIG1).

[0146] Unlike embodiment 1, which has a small-level marker appended part SF in the annotation part AN, when the time series data is a periodically repeating waveform, that is, when it does not include the event EV, the small-level marker appended part SF can be omitted (as shown in Fig. 1).

[0147] Figure 27 is a functional block diagram of a modified example of Embodiment 1.

[0148] Unlike implementation form 1, which uses the labeling of the hierarchy (labeling of process PR) in the learning matching of the learning unit GA, an existing learning model GM can also be read in. In Figure 27, the hierarchy labeling inference unit CS obtains the existing learning model GM from memory or the like, and uses the obtained existing learning model GM to infer the labeling of the hierarchy in the matching.

[0149] <Effects of Embodiment 1> Figure 28 is a timeline showing the effects of the annotation device AS in Embodiment 1.

[0150] As shown above, the annotation device AS of Embodiment 1, and as shown in Figure 28, assigns different granularities of markings, such as batch number LO, process PR, and event EV, to each of the three levels of the time series data: the large level, the medium level, and the small level. More specifically, the annotation device AS of Embodiment 1 assigns markings such as batch number 1 LO1, batch number 2 LO2, ... to the large level, batch number 1 PR1, process 2 PR2, ... to the medium level, and event number 1 EVA1, event 1 EVb1, ... to the small level. In this way, the time series data can be simultaneously grasped both broadly and locally.

[0151] In the annotation device AS of Embodiment 1, for example, even if the third process PR3 and the seventh process PR7 are similar to each other, the event EV included in the third process PR3 of the former is assigned the so-called first event EVa1 to the sixth event EVa6, while the event EV included in the seventh process PR7 of the latter is assigned the first event EVb1 to the sixth event EVb6, which are different from the first event EVa1 to the sixth event EVa6. In this way, the first event EVa1 to the sixth event EVa6 of the former and the first event EVb1 to the sixth event EVb6 of the latter can be distinguished, and more specifically, the first event EVa1 of the former and the first event EVb1 of the latter can be distinguished from each other.

[0152] In the annotation device AS of embodiment 1, since the time series data itself is affixed with an inappropriate, i.e., an abnormal, affixation of an abnormal mark, it is possible to distinguish whether the affixing of the mark to the time series data is inappropriate and whether the time series data itself is inappropriate.

[0153] Implementation Form 2. <Implementation Form 2> In Implementation Form 2, after being marked once, it is changed to other marks, resulting in a new hierarchical structure.

[0154] <Function of Embodiment> Figure 29 is a functional block diagram of the annotation device AS in Embodiment 2.

[0155] The annotation device AS of Embodiment 2, as shown in FIG29, has basically the same function as the annotation device AS of Embodiment 1 (shown in FIG1).

[0156] On the other hand, the annotation device AS of Embodiment 2 is different from the annotation device AS of Embodiment 1, and as shown in FIG29, it further includes a marking change unit LH.

[0157] The mark change part LH corresponds to "change part".

[0158] For example, when a user wants to confirm the first event EVA1 belonging to the third process PR3 (as shown in Figure 2), the display unit HY displays the range from the second process PR2 to the fourth process PR4 when the user specifies the third process PR3 and the processes before and after the third process PR3, namely the second process PR2 and the fourth process PR4.

[0159] In the label change unit LH, when the labeling result and hierarchical structure are received from the annotation unit AN, the labels of each level are changed using the existing hierarchical structure, and a new hierarchical structure for the changed labels is generated.

[0160] <Hardware configuration of embodiment 2> The annotation device AS of embodiment 2 has the same hardware configuration as the annotation device AS of embodiment 1 (as shown in FIG5).

[0161] <Operation of Embodiment 2> Figure 30 is a flowchart showing the operation of the annotation device AS in Embodiment 2.

[0162] Step ST91: Set the content to be changed. Specifically, this includes setting the hierarchy of the object to be changed, the type of change (integration, replacement, etc.), and the location of the change.

[0163] For the “level of the object to be changed”, specify the level to be changed (e.g., high level, medium level, low level. If there are more than two levels, specify each level).

[0164] Figure 31 shows the timeline of the integrated actions in implementation mode 2.

[0165] Figure 32 shows the timeline of alternative actions for implementation mode 2.

[0166] For the "type of change (integration, substitution, etc.)", specify "integration" or "substitution" as the type of change. "Integration" is the result of integrating multiple markers. For example, when integrating process 2 PR2 (shown in Figure 8.) and process 3 PR3 (shown in Figure 8.), a new process 2 PR2 is formed (shown as a dashed arrow in Figure 31).

[0167] "Replacement" means to attach the designated mark to other marks. For example, as shown in Figure 8, since the values ​​of processes 5 PR5, 9 PR9 and 11 PR11 change little, the marks "process 5 PR5", "process 9 PR9" and "process 11 PR11" are replaced with the mark "stable part" as shown in Figure 32.

[0168] In the "Change Location", specify the location where changes (integration, replacement, etc.) will be made. For example, specify the second process PR2 (shown in Figure 8) and the third process PR3 (shown in Figure 8) to be integrated as described above, and specify the fifth process PR5, the ninth process PR9, and the eleventh process PR11 (all shown in Figure 8) to be replaced as described above.

[0169] It is also possible to specify changes and substitutions for multiple places, or to combine changes and substitutions and specify them together. The content of the changes and substitutions can be defined according to the file, or the user can set them using a graphical user interface (GUI).

[0170] Step ST92: Obtain the existing attached label results and hierarchical structure, and change the labels. As shown above, for example, by integrating process 2 PR2 (shown in Figure 8.) and process 3 PR3 (shown in Figure 8.), a new process 2 PR2 is formed (illustrated by a dashed arrow in Figure 31). For each process after process 4 PR4 (shown in Figure 8.), the labels are changed sequentially, for example, from label "process 4 PR4" (shown in Figure 8.) to the new label "process 3 PR3" (illustrated in Figure 31.).

[0171] Step ST93: Based on the results of the changed markers, change to a new hierarchical structure. For example, event EVA1 (e.g., shown in Figure 28.), which was originally included in process 3 PR3 (shown in Figure 8.), becomes included in the new process 2 PR2 (shown in Figure 31.). Similarly, event EVb1 (e.g., shown in Figure 28.), which was originally included in process 7 PR7 (shown in Figure 8.), becomes included in the new process 6 PR6 (shown in Figure 31.).

[0172] The changes to the markings and hierarchical structure mentioned above are not limited to the first batch number LO (as shown in Figure 8), but are also made to subsequent batch numbers LO, such as the second batch number LO2, the third batch number LO3, ...

[0173] <Modified Example> Unlike the implementation form 2 of the mark change of the level mark (the mark of process PR) in the mark change section LH, it is also possible to change the mark of the large level (the mark of batch number LO) and the small level (the mark of event EV), and it is also possible to change more than two levels.

[0174] For example, when the setting for marking changes is defined from the file, the display section HY (shown in Figure 29) can be omitted.

[0175] The expansion of the data in the expansion section KA is not necessary, so the expansion section KA can be omitted (as shown in Figure 29).

[0176] <Effects of Embodiment 2> Figure 33 is a timeline showing the effects of the annotation device AS in Embodiment 2.

[0177] As described above, in Embodiment 2, the comprehensive or partial inclusion relationship in the time series data is shown in Figure 33. For example, while maintaining that batch number 1 LO1 includes processes 1 to 11 PR10, and process 3 PR3 includes events EVA1 to EVA6, the markers in multiple layers are changed. Moreover, the marker changes are performed simultaneously, for example, at a large layer, specifically not only for batch number 1 LO1, but also for other batch numbers LO, i.e., batch number 2 LO2, batch number 3 LO3, ... This reduces the difficulty of simultaneously changing the markers of multiple batch numbers LO covering the time series data.

[0178] Instead of the time series data that is the object in Embodiments 1 to 2, other data may be used. Other data can be any data that is ideally implemented in a fixed order and repeated (data that has a correct order of implementation and is repeated). Other data may include text columns, symbol columns, graphics, sounds, images, pictures, etc.

[0179] Without departing from the spirit of the present invention, the above embodiments may be combined, and the constituent elements in each embodiment may be appropriately deleted, modified, or additional. Industrial Applicability

[0180] The annotation device of the present invention can be used to simultaneously acquire both extensive and partial knowledge of a series of data. [Simplified Explanation of the Diagram]

[0009] Figure 1 is a functional block diagram of the annotation device AS in Embodiment 1. Figure 2 is a timetable displaying the time series data of Embodiment 1. Figure 3 shows the hierarchical structure of Embodiment 1 (part 1). Figure 4 shows the hierarchical structure of Embodiment 1 (part 2). Figure 5 shows the hardware configuration of the annotation device AS in Embodiment 1. Figure 6 is a flowchart showing the operation of the annotation device AS in Embodiment 1. Figure 7 shows the timetable of the segmentation operation in Embodiment 1. Figure 8 shows the timetable of the operation of correcting the segmentation position in Embodiment 1. Figure 9 shows the timetable of segmenting into smaller hierarchical levels in Embodiment 1. Figure 10 is an example of the JSON file of the mark table and hierarchical structure in Embodiment 1. Figure 11 is a flowchart showing the operation of expanding data in Embodiment 1. Figure 12 shows the correspondence between the expansion level and the operation of expanding data in Embodiment 1. Figure 13A shows the scope used by the correct solution generation unit SE in Embodiment 1 (part 1). Figure 13B shows the scope used by the correct solution generation unit SE in Embodiment 1 (part 2). Figure 13C shows the scope of the correct solution generation unit SE used in Embodiment 1 (part 3). Figure 13D shows the scope of the correct solution generation unit SE used in Embodiment 1 (part 4). Figure 14 is a flowchart showing the learning operation of the learning unit GA in Embodiment 1. Figure 15 shows the timetable of the sliding window in Embodiment 1. Figure 16 is a flowchart showing the operation of the annotation unit AN in Embodiment 1. Figure 17 is a flowchart showing the error correction operation in Embodiment 1. Figure 18 is a timetable showing the error correction operation (part 1) in Embodiment 1. Figure 19 is a timetable showing the error correction operation (part 2) in Embodiment 1. Figure 20 is a timetable showing the error correction operation (part 3) in Embodiment 1. Figure 21 is a flowchart showing the operation of the small-level mark attachment unit SF in Embodiment 1. Figure 22 is a flowchart showing the operation of the large-level mark attachment unit DF in Embodiment 1. Figure 23 is a timetable showing the results of corrected errors in Embodiment 1. Figure 24 is a flowchart showing the operation of the hierarchical structure generating unit KS in Embodiment 1. Figure 25 is a timetable showing the operation of the hierarchical structure generating unit KS in Embodiment 1. Figure 26 shows a portion of the hierarchical structure in Embodiment 1. Figure 27 is a functional block diagram of a variant of Embodiment 1. Figure 28 is a timetable showing the effects of the annotation device AS in Embodiment 1. Figure 29 is a functional block diagram of the annotation device AS in Embodiment 2. Figure 30 is a flowchart showing the operation of the annotation device AS in Embodiment 2. Figure 31 is a timetable showing the integration operation in Embodiment 2. Figure 32 is a timetable showing the alternative operation in Embodiment 2. Figure 33 is a timetable showing the effects of the annotation device AS in Embodiment 2.

Claims

1. A learning device, characterized by comprising: an acquisition unit that acquires a first plurality of data groups, each data group being a plurality of data, wherein the content and order of the plurality of data are identical among the plurality of data groups; and a learning unit that generates a learned model in which a first plurality of tags are attached to the plurality of data groups at a first level, and a second plurality of tags, different from the first plurality of tags, are attached to the plurality of data groups at a lower level below the first level, namely the second level.

2. An inference apparatus, characterized by comprising: an acquisition unit that acquires a second plurality of data groups that are different from the aforementioned first plurality of data groups; and an assignment unit that, using the learned model described in claim 1, assigns the aforementioned first plurality of tags to the aforementioned second plurality of data groups at the aforementioned first level, and assigns the aforementioned second plurality of tags to a plurality of data in the aforementioned second plurality of data groups at the aforementioned second level.

3. The inference apparatus of claim 2, further comprising: a generation unit that generates a hierarchical structure having the aforementioned first level and the aforementioned second level for the aforementioned second plurality of data groups.

4. The learning device as claimed in claim 1, further comprising: an unfolding unit that significantly increases the learning data by unfolding the aforementioned first plurality of data groups.

5. The learning apparatus of claim 1, further comprising: a modification unit that modifies at least one of the aforementioned first plurality of tags and the aforementioned second plurality of tags attached to the previously learned model generated by the aforementioned learning unit to other tags.

6. The inference device of claim 2, further comprising: a modification unit that changes at least one of the aforementioned first plurality of marks and the aforementioned second plurality of marks attached by the aforementioned attachment unit to other marks.

7. The inference apparatus of claim 2, further comprising: a correction unit that corrects erroneous marks among the aforementioned first plurality of marks and the aforementioned second plurality of marks based on the aforementioned appended unit, and identifies anomalies in the aforementioned first plurality of data groups.

8. The inference apparatus as described in claim 2, wherein, The aforementioned appended part is attached to the aforementioned first plurality of marks in the aforementioned first layer and then to the aforementioned second plurality of marks in the aforementioned second layer, or to the aforementioned first plurality of marks in the aforementioned second layer and then to the aforementioned first layer.

9. A learning method, characterized in that: an acquisition unit acquires a first plurality of data groups, each data group having a plurality of data, wherein the content and order of the aforementioned plurality of data should be the same among the aforementioned plurality of data groups; a learning unit generates a learned model that assigns a first plurality of labels to the aforementioned plurality of data groups at a first level, and assigns a second plurality of labels different from the aforementioned first plurality of labels to the aforementioned plurality of data groups at a lower level below the aforementioned first level, namely the second level.

10. An inference method, characterized in that: the acquisition unit acquires a second plurality of data groups that are different from the aforementioned first plurality of data groups; the assignment unit uses the learned model described in claim 9; the aforementioned first plurality of labels are assigned to the aforementioned second plurality of data groups at the aforementioned first level; and the aforementioned second plurality of labels are assigned to a plurality of data in the aforementioned second plurality of data groups at the aforementioned second level.

Images