Model generation device and model generation method

Abstract

To improve quality of a learning model as compared with a conventional one for performing drawing retrieval.SOLUTION: In a learning device (11 V), a determination unit (115 V) calculates (i) a multi-collinearity evaluation value between two content parameters for each of the combined patterns of the two different content parameters among multiple content parameters included in a content parameter set, and (ii) determines a deletion object content parameter from the multiple content parameters based on the multi-collinearity evaluation value. A learning model generation unit (113 V) generates a learning model for performing a drawing retrieval based on the content parameter set after pruning obtained by deleting the content parameter of the deletion object from the content parameter set.SELECTED DRAWING: Figure 22

Description

[Technical Field]

[0001] One aspect of the present invention relates to a model generation device that generates a learning model for performing drawing search. [Background technology]

[0002] For example, in the field of plant engineering, it is necessary to handle a large number of drawings (e.g., past drawings). For this reason, a technique has been proposed for efficiently searching for a desired drawing from among a large number of drawings.

[0003] As an example, Patent Document 1 discloses a technology aimed at improving user convenience in drawing searches. Specifically, the technology in Patent Document 1 uses machine learning to generate a learning model for drawing searches. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-12413 Summary of the Invention [Problem to be solved by the invention]

[0005] An aspect of the present invention aims to improve the quality of a learning model for performing drawing search compared to conventional techniques. [Means for solving the problem]

[0006] In order to solve the above problem, one embodiment of the model generation device of the present invention is a model generation device that generates a learning model for searching for at least one drawing corresponding to a target drawing from among a plurality of search target drawings, and is equipped with: an acquisition unit that acquires a content parameter set including a plurality of content parameters related to the description content of each of the plurality of search target drawings by analyzing the plurality of search target drawings; (i) a determination unit that calculates a multicollinearity evaluation value between the two content parameters for each combination pattern of two different content parameters from among the plurality of content parameters included in the content parameter set, and (ii) a deletion target content parameter to be deleted from among the plurality of content parameters based on the multicollinearity evaluation value; and a learning unit that generates the learning model based on a pruned content parameter set obtained by deleting the deletion target content parameter from the content parameter set.

[0007] In addition, a model generation method according to one embodiment of the present invention is a model generation method for generating a learning model for searching for at least one drawing corresponding to a target drawing from among a plurality of search target drawings, and includes: an acquisition step of acquiring a content parameter set including a plurality of content parameters related to the description of each of the plurality of search target drawings by analyzing the plurality of search target drawings; (i) a determination step of calculating a multicollinearity evaluation value between two different content parameters from among the plurality of content parameters included in the content parameter set for each combination pattern of the two different content parameters, and (ii) a determination step of determining content parameters to be deleted from among the plurality of content parameters based on the multicollinearity evaluation value; and a learning step of generating the learning model based on a pruned content parameter set obtained by deleting the content parameters to be deleted from the content parameter set. [Effects of the Invention]

[0008] According to one aspect of the present invention, the quality of a learning model for performing drawing search can be improved compared to conventional methods. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a configuration of a main part of an information processing system according to a reference embodiment. [Figure 2] FIG. 10 is a diagram showing an example of a past property / drawing correspondence table in a reference form. [Figure 3] FIG. 10 is a diagram showing an example of a past drawing content parameter table in the reference embodiment. [Figure 4] FIG. 10 is a diagram showing an example of a content parameter setting table in an initial state in the reference embodiment. [Figure 5] FIG. 10 is a diagram showing an example of the number of content parameters after preprocessing in each data set in the reference embodiment. [Figure 6] FIG. 2 is a diagram illustrating a schematic example of the data structure of a data set 1 in the reference embodiment. [Figure 7] 10A and 10B are diagrams showing examples of a normalization data table and a standardization data table in the reference embodiment. [Figure 8] FIG. 10 is a diagram showing an example of a dataset-preprocessing method correspondence table in the reference embodiment. [Figure 9] FIG. 10 is a diagram illustrating some of a plurality of data sets generated by a learning preprocessing unit in the reference embodiment. [Figure 10] FIG. 10 is a diagram showing an example of an evaluation result table in the reference embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a content parameter setting table after updating in the reference embodiment. [Figure 12] FIG. 10 is a diagram showing an example of a new drawing content parameter table in the reference embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a post-preprocessing new drawing content parameter table in the reference embodiment. [Figure 14] FIG. 10 is a diagram illustrating some of a plurality of raw value-label value conversion tables in a reference embodiment. [Figure 15] 10 illustrates an example of a raw value-label value conversion integrated table in a reference embodiment. [Figure 16] 1 is a block diagram showing the configuration of a main part of an information processing system according to a first embodiment. [Figure 17] FIG. 10 is a diagram showing an example of a table of coefficients of determination and the number of high-risk coefficients of determination in the initial state in the first embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a coefficient of determination / number of high-risk coefficients of determination table obtained by updating the coefficient of determination / number of high-risk coefficients of determination table of FIG. 17. [Figure 19] FIG. 20 is a diagram showing an example of a coefficient of determination / number of high-risk coefficients of determination table obtained by updating the coefficient of determination / number of high-risk coefficients of determination table in FIG. 18. [Figure 20] FIG. 10 is a diagram showing an example of a final table of coefficients of determination and the number of high-risk coefficients of determination obtained as a result of repeated updates. [Figure 21] FIG. 21 is a diagram showing an example of a pruned dataset 1 corresponding to the table of coefficient of determination and number of high-risk coefficient of determination in FIG. 20. [Figure 22] FIG. 10 is a block diagram showing the configuration of a main part of an information processing system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Reference form] Prior to describing the information processing system 100 of the first embodiment, an information processing system 100s will be described as a reference embodiment. For convenience of description, components having the same functions as those described in the reference embodiment will be denoted by the same reference numerals in the following embodiments, and their descriptions will not be repeated. For simplicity, descriptions of matters similar to those in known technologies will also be omitted as appropriate.

[0011] Please note that the numerical values ​​described below in this specification are merely examples. In this specification, the expression "A to B" for two numbers A and B means "greater than or equal to A and less than or equal to B" unless otherwise specified.

[0012] (Outline of Information Processing System 100s) 1 is a block diagram showing the configuration of a main part of an information processing system 100s. The information processing system 100s includes an information processing device 1s, a past property plan DB (Database) 91, and a new property plan DB 92.

[0013] The information processing device 1s includes a control device 10s, an input unit 71, a display unit 72, and a storage unit 80. The control device 10s includes a learning device 11s and a drawing search device 12. In this specification, the term "property" refers to, for example, a "site" in plant engineering.

[0014] The information processing device 1 s may be communicably connected to the past property plan DB 91 and the new property plan DB 92. Therefore, unlike the example of Fig. 1, at least one of the past property plan DB 91 and the new property plan DB 92 may be provided inside the information processing device 1 s.

[0015] Drawings to be searched by the information processing device 1s (more specifically, the control device 10s) (search target drawings) include, for example, specification drawings, design drawings, and production drawings. Search target drawings may also include specifications, design documents, and estimates. In this way, search target drawings are not limited to documents containing the name "drawing." As an example, search target drawings include any type of document related to project plans in the plant engineering field.

[0016] However, as will be apparent to those skilled in the art, the information processing device according to an aspect of the present invention can also be applied to searching for drawings in fields other than plant engineering. The drawings according to an aspect of the present invention may be drawings from which content parameters can be obtained by the information processing device.

[0017] The control device 10s comprehensively controls each unit of the information processing device 1s. The storage unit 80 stores various data and programs used in the processing of the control device 10s. As described below, the control device 10s uses machine learning to search for at least one drawing corresponding to the target drawing (e.g., drawing ND) from multiple search target drawings (e.g., past drawings a1 to MN).

[0018] The input unit 71 accepts user operations (user operations). The display unit 72 displays various data. As an example, the display unit 72 may display data indicating search results by the control device 10s. The input unit 71 and the display unit 72 may be provided as an integrated unit. For example, the input unit 71 and the display unit 72 can be integrated by using a touch panel.

[0019] (Previous property drawing DB91) The past property drawing DB91 stores drawings (strictly speaking, drawing data) for each past property (each existing property). In the following explanation, "drawing data of drawing A (a certain drawing)" will be abbreviated as simply "drawing A" as appropriate. Also, "drawing number of drawing A" will be abbreviated as simply "drawing A" as appropriate.

[0020] In the reference embodiment, the past property drawing DB 91 stores drawings for each of a plurality of M different properties. M is an integer equal to or greater than 1. Hereinafter, the jth property will also be referred to as "property j." j is an integer equal to or greater than 1 and equal to or less than M.

[0021] Furthermore, the past property drawing DB91 stores N different drawings (N types) for each of properties 1 to M. N is an integer equal to or greater than 1. Hereinafter, the i-th drawing for property j will also be referred to as "drawing (i, j)." Furthermore, the i-th drawing (i-th type) for each property will also be collectively referred to as drawing i. i is an integer equal to or greater than 1 and equal to or less than N.

[0022] As described above, a total of T drawings are stored in the past property drawing DB 91. In the reference example, T = M x N. T is an integer equal to or greater than 2. In other words, at least one of M and N is equal to or greater than 1.

[0023] Specifically, in the past property drawing DB91, similar to Patent Document 1, each type of drawing number is listed by property number in the form of the past property / drawing correspondence table TB1 shown in Figure 2. Hereinafter, the past property / drawing correspondence table TB1 will also be abbreviated as "TB1." Other elements will also be abbreviated in the same way as appropriate. The cell in the i-th row and j-th column of TB1 indicates the drawing number of drawing (i, j).

[0024] In the example of Figure 2, for convenience, properties 1 to 3 are also referred to as properties A to C, respectively. In the example of Figure 2, the first type of drawing (Drawing 1) is an outline drawing, the second type of drawing (Drawing 2) is an assembly drawing, the third type of drawing (Drawing 3) is a foundation drawing, and the Nth type of drawing (Drawing N) is a configuration drawing.

[0025] For simplicity, the drawings (1,1) to (N,1) in the example of FIG. 2 (i.e., the N types of drawings for property A) will be referred to as drawings a1 to aN below. For example, drawings a1 to aN in the example of FIG. 2 refer to the outline drawing to the configuration drawing of property A, respectively. The drawings for other properties will be referred to in the same way.

[0026] As described above, the past property drawing DB91 stores a total of T drawings, from drawing a1 to drawing MN. Hereinafter, drawings a1 to MN will also be collectively referred to as past drawings. Past drawings are an example of drawings to be searched. For this reason, drawings a1 to MN will also be referred to as a group of drawings to be searched.

[0027] In addition, in this specification, any one of multiple past drawings (drawings to be searched) is also referred to as a candidate drawing. As an example, we will mainly illustrate the processes when drawing a1 (exterior drawing of property A) is the candidate drawing. The processes for other drawings will be omitted as appropriate, but are the same as those for drawing a1.

[0028] (New property drawing DB92) The new property drawing DB 92 stores drawings (hereinafter also collectively referred to as new drawings) relating to new properties (e.g., at least one property that is scheduled to be constructed) as a new property data set. In this specification, one new property (property T) included in the new property data set will be described.

[0029] As an example, the new property drawing DB92 stores N different drawings (outline drawings to configuration drawings) of the same type as previous properties for property T. In this specification, the outline drawing of property T is illustrated as an example of a new drawing. Hereinafter, the outline drawing of property T will be referred to as drawing ND. Drawing ND in the reference embodiment is assumed to be the same as in Patent Document 1.

[0030] (Learning Device 11s) The learning device 11s includes a past drawing data acquisition unit 111, a past drawing content parameter acquisition unit 112 (candidate drawing content parameter acquisition unit, search target drawing content parameter acquisition unit, acquisition unit), a learning preprocessing unit 114 (preprocessing unit), and a learning model generation unit 113s. The learning device 11s generates a learning model for drawing search by the drawing search device 12s based on the drawings a1 to MN. For this reason, the learning device 11s may be referred to as a model generation device. An example of the processing flow of the learning device 11s will be described below.

[0031] (Obtaining candidate drawings) In the past property drawing DB91, drawings a1 to MN are pre-sorted by drawing type according to TB1. Therefore, for example, the past property drawing data acquisition unit 111 acquires each exterior drawing from the past property drawing DB91 in the order of "drawing a1 → b1 → ... → M1". Next, the past property drawing data acquisition unit 111 acquires each assembly drawing from the past property drawing DB91 in the order of "drawing a2 → b2 → ... → M2". Then, finally, the past property drawing data acquisition unit 111 acquires each configuration drawing from the past property drawing DB91 in the order of "drawing aN → bN → ... → MN". These drawings a1 to MN in the reference embodiment are assumed to be the same as those in Patent Document 1.

[0032] In the above example, the past drawing data acquisition unit 111 first references the cell in row 1, column 1 of TB1. Then, the past drawing data acquisition unit 111 acquires the drawing (1,1) corresponding to the above cell, i.e., drawing a1, from the past property drawing DB 91. The past drawing data acquisition unit 111 supplies the acquired drawing a1 to the past drawing content parameter acquisition unit 112.

[0033] (Getting the content parameter set corresponding to the candidate drawing) The past drawing content parameter acquisition unit 112 acquires a content parameter set corresponding to the drawing a1 by analyzing the drawing a1 (more specifically, by performing a syntax analysis on the drawing a1 after OCR processing, focusing on the k-th specific character string described below), in the same manner as in Patent Document 1. The content parameter set is a data set indicating the k-th content parameter (hereinafter, Ak) associated with the k-th specific character string. The first to L-th content parameters are also collectively referred to as content parameters.

[0034] In this specification, a specific character string that is set in advance for each drawing is referred to as a specific character string. In the reference embodiment, it is assumed that L (L is an integer of 2 or more) different kth specific character strings are set in advance. Hereinafter, the kth specific character string will be referred to as the kth specific character string. k is an integer of 1 or more and L or less. In the following explanation, an example will be given in which the first specific character string is set as "voltage value", the second specific character string is set as "current value", the third specific character string is set as "OR", and the Lth specific character string is set as "open".

[0035] The content parameter is a quantity associated with the content of a drawing (specifically, the content of a specific character string). Therefore, the content parameter can be said to be a type of data that quantifies (quantifies) the content of the drawing. For this reason, the content parameter is used as an index indicating the content of a drawing.

[0036] In order to distinguish from the target drawing content parameters described later, the content parameters of the search target drawing (past drawing) are also referred to as search target drawing content parameters. Also, the kth content parameter of the search target drawing is also referred to as the kth content parameter of the search target drawing. However, in the following description, unless otherwise specified, content parameters refer to past drawing content parameters. Similarly, unless otherwise specified, content parameter set refers to past drawing content parameter set.

[0037] In this specification, Ak of ​​drawing (i, j) is also written as Ak(i, j). As described above, the past drawing content parameter acquisition unit 112 sets Ak(i, j) based on the analysis result for drawing (i, j) (more specifically, the detection result of a specific character string in drawing (i, j)).

[0038] As described above, the past drawing content parameter acquisition unit 112 sets A1 to AL for drawing a1. As described above, the past drawing content parameter acquisition unit 112 acquires content parameters of the candidate drawing by analyzing the candidate drawing. For this reason, the past drawing content parameter acquisition unit 112 is also referred to as a candidate drawing content parameter acquisition unit.

[0039] The past drawing content parameter acquisition unit 112 also performs the same process for the other past drawings. That is, the past drawing content parameter acquisition unit 112 sets A1 to AL for the drawings a1 to MN, respectively.

[0040] Thereafter, the past drawing content parameter acquisition unit 112 generates a past drawing content parameter table TB2 indicating A1 to AL of the drawings a1 to MN, respectively, that is, A1(1,1) to AL(M,N).

[0041] TB2 includes the ith sub-table TB2-i in the past drawing content parameter table. FIG. 3 shows TB2-1 as an example. TB2-i is a table showing A1 to AL of each of drawing i. TB2-1 shows A1 to AL of each of drawings a1 to M1 (exterior drawings of properties A to M). TB2 is made up of a set of N sub-tables, TB2-1 to TB2-N. In this way, in the reference embodiment, a sub-table is created for each drawing type. These sub-tables in the reference embodiment are assumed to be the same as those in Patent Document 1.

[0042] TB2 indicates the correspondence between the number (an example of an identifier) ​​of each of a plurality of past drawings (e.g., drawing a1) and each of A1 to AL of the plurality of past drawings. Therefore, the learning model generation unit 113s acquires TB2 as training data. As an example, the learning model generation unit 113s may generate a learning model by performing multinomial logistic regression using the training data. However, as will be clear from the description below, the machine learning algorithm according to one embodiment of the present invention is not limited to this example, and other well-known algorithms may be applied. Note that the numbers (identifiers) of each past drawing indicated in TB2 are used as correct answer data in machine learning.

[0043] In the reference embodiment, the past drawing content parameter acquisition unit 112 acquires the kth content parameter and also acquires information indicating the variable type of the kth content parameter (kth content parameter variable type information). The kth content parameter variable type information is information indicating the variable type (data type) of the kth content parameter. Specifically, the kth content parameter variable type information is information indicating whether the kth content parameter is a qualitative variable (hereinafter referred to as VL) or a quantitative variable (hereinafter referred to as VN). Hereinafter, the first to kth content parameter variable type information will be collectively referred to as content parameter variable type information.

[0044] As an example, the past drawing content parameter acquisition unit 112 may acquire content parameter variable type information based on the results of the above-described syntax analysis. For example, as described above, the past drawing content parameter acquisition unit 112 determines, as a result of the syntax analysis, that the number "90" contained in the character string "90V" following the first specific character string "voltage value" represents the magnitude of the voltage value. In this case, the past drawing content parameter acquisition unit 112 determines that the first content parameter is VN. Similarly, the past drawing content parameter acquisition unit 112 determines that the second content parameter is VN.

[0045] Furthermore, the past drawing content parameter acquisition unit 112 determines, as a result of syntax analysis, that the third specific character string "OR" is not followed by a character corresponding to a quantitative variable. In this case, the past drawing content parameter acquisition unit 112 determines that the third content parameter is VL. Similarly, the past drawing content parameter acquisition unit 112 determines that the Lth content parameter is VL.

[0046] (An example of processing in the learning preprocessing unit 114) The learning preprocessing unit 114 acquires content parameter sets corresponding to the drawings a1 to MN (the first to Lth content parameters of each of the drawings a1 to MN) and content parameter variable type information from the past drawing content parameter acquisition unit 112. Then, the learning preprocessing unit 114 generates a content parameter setting table TB3 according to the acquired content parameter variable type information.

[0047] The following explanation mainly describes each process based on the content parameter sets of drawings a1 to M1 (outline drawings) (the first to Lth content parameters of drawings a1 to M1, respectively). Therefore, the tables and data in each drawing described below are generated based on the content parameter sets of the outline drawings. Please note that these tables and data are also generated for other types of drawings shown in FIG. 2 (e.g., assembly drawings, foundation drawings, and configuration drawings) using a process flow similar to that described below for outline drawings. Thus, in the reference embodiment, these tables and data are generated for each type of drawing shown in FIG. 2.

[0048] FIG. 4 shows an example of TB3 in its initial state. As shown in FIG. 4, TB3 is a table showing the correspondence between (i) the variable types (data types) of the first to Lth content parameters and (ii) the preprocessing methods to be applied to the first to Lth content parameters. The learning preprocessing unit 114 records the acquired content parameter variable type information in the "variable" field of TB3. Note that the preprocessing methods to be applied to the first to Lth content parameters have not yet been determined at this stage. Therefore, in the initial state of TB3, all of the "preprocessing methods" fields are set as blank fields. In this specification, TB3 in its initial state is referred to as TB3init.

[0049] Next, the learning preprocessing unit 114 applies a plurality of types of preprocessing methods to the k-th content parameter according to the k-th content parameter variable type information. Specifically, in the reference embodiment, when the k-th content parameter is VL, the learning preprocessing unit 114 applies the following (i) to (iv) to the k-th parameter: (i) Processing using raw values ​​as is (identity processing) (hereinafter referred to as [R]); (ii) One hot encoding (hereafter referred to as [O]) ; (iii) Normalization (hereafter referred to as [N]); (iiii) Standardization (hereinafter referred to as [S]); Four types of preprocessing methods are applied:

[0050] When the k-th content parameter is VL, the k-th content parameter can also be understood as a value derived by label encoding (hereinafter referred to as [L]) of the k-th specific character string by the past drawing content parameter acquisition unit 112. Therefore, when the k-th content parameter is VL, [R] can be read as [L]. In this way, when the k-th content parameter is VL, [R] and [L] are equivalent preprocessing methods. An example of the preprocessing method [L] will be described later.

[0051] On the other hand, when the k-th content parameter is VN, the learning preprocessing unit 114 performs the following (i) to (v) on the k-th parameter: (i) [R]; (ii) [L]; (iii) [O]; (iv) [N]; (v) [S]; Five types of preprocessing techniques are applied. However, as will be apparent to those skilled in the art, the preprocessing technique according to one embodiment of the present invention is not limited to these examples. The preprocessing technique according to one embodiment of the present invention may be any preprocessing technique applicable to VL or VN.

[0052] As described above, the learning preprocessing unit 114 applies multiple types of preprocessing methods to each content parameter included in the content parameter set according to the k-th content parameter variable type information, thereby expanding (data augmenting) the content parameter set. Hereinafter, the numbers of VL and VN included in the content parameter set will be denoted as FL and FN, respectively.

[0053] As is clear from the above explanation, in the example of the reference embodiment, the total number of combinations of preprocessing patterns for the content parameter set is 4.FL x5 FN Therefore, the learning preprocessing unit 114 processes one content parameter set into four FL x5 FN The following four content parameter sets are FL x5 FN These four content parameter sets are collectively referred to as the expanded content parameter sets. FL x5 FN Each of the content parameter sets is Dataset 1, Dataset 2, ..., Dataset 4 FL x5 FN The post-enhancement content parameter set may be referred to as a post-preprocessing content parameter set. For example, data set 1 may be referred to as data set No. 1.

[0054] The past drawing content parameter set may be referred to as a search target drawing content parameter set. Therefore, the expanded content parameter set may be referred to as a past drawing expanded content parameter set (or a search target drawing expanded content parameter set). Similarly, the pre-processed content parameter set may be referred to as a pre-processed past drawing content parameter set (or a pre-processed search target drawing content parameter set). Each piece of data included in the pre-processed content parameter set may be referred to as a pre-processed content parameter (more specifically, a pre-processed search target drawing content parameter).

[0055] 5 shows an example of the number of post-preprocessing content parameters in each data set (hereinafter referred to as the number of post-preprocessing content parameters). The number of post-preprocessing content parameters in one data set (e.g., data set 1) can also be expressed as the number of data included in the data set. Therefore, for example, the number of post-preprocessing content parameters in data set 1 may be referred to as the number of dimensions (number of elements) of data set 1. Hereinafter, the kth content parameter after preprocessing will be referred to as the kth post-preprocessing content parameter.

[0056] As will be apparent from Fig. 6, the number of post-preprocessing content parameters may vary depending on the pre-processing method applied to each content parameter included in the content parameter set. For example, the more content parameters [O] is applied to, the more the number of post-preprocessing content parameters tends to increase (see also Fig. 8).

[0057] Fig. 6 is a diagram illustrating a schematic example of the data structure of Data Set 1. In the example of Fig. 6, Data Set 1 is a data structure (e.g., a data frame) generated by applying [S] to (i) the first content parameter, [S] to (ii) the second content parameter, [O] to (iii) the third content parameter, and [L] to (iv) the Lth content parameter (see also Fig. 8 described below).

[0058] In the example of Figure 6, the third content parameter (the content parameter corresponding to the third specific string "OR") is one-hot encoded to generate four pre-processed content parameters corresponding to the third content parameter: "OR_0", "OR_1", "OR_2", and "OR_3".

[0059] As an example for convenience of explanation, consider a case where the minimum value of the third content parameter (A3) included in the content parameter set is 0 and the maximum value is 3. That is, consider a case where, in the content parameter set, A3 takes four discrete values ​​from 0 to 3. In this case, each value of A3 can be represented by a four-dimensional one-hot vector (more specifically, a four-bit one-hot vector).

[0060] For example, if A3=0, OR_0=(1,0,0,0); OR_1=(0,0,0,0); OR_2=(0,0,0,0); OR_3=(0,0,0,0); is.

[0061] Also, if A3=1, OR_0=(0,0,0,0); OR_1=(0,1,0,0); OR_2=(0,0,0,0); OR_3=(0,0,0,0); is.

[0062] Also, if A3=2, OR_0=(0,0,0,0); OR_1=(0,0,0,0); OR_2=(0,0,1,0); OR_3=(0,0,0,0); is.

[0063] Also, if A3=3, OR_0=(0,0,0,0); OR_1=(0,0,0,0); OR_2=(0,0,0,0); OR_3=(0,0,0,1); As described above, A3=i (where i is any natural number from 0 to 3) is expressed by a one-hot vector OR_i, which has a component “1” only in the i-th element.

[0064] Next, an example of the preprocessing method [N] will be described. In the following, a case where the learning preprocessing unit 114 normalizes the k-th content parameter (Ak) will be described as an example. First, the learning preprocessing unit 114 acquires the maximum value (Akmax) and the minimum value (Akmin) of Ak from the content parameter set.

[0065] Then, the learning preprocessing unit 114 performs the following: Ak_Normalized=(Ak-Akmin) / (Akmax-Akmin) …(1) Ak_Normalized is calculated as follows. Ak_Normalized is referred to as the normalized k-th content parameter (more specifically, the normalized k-th past drawing content parameter). The normalized 1st to Lth content parameters are collectively referred to as the normalized content parameters (more specifically, the normalized past drawing content parameters). The normalized content parameters are an example of preprocessed content parameters.

[0066] As described above, the learning preprocessing unit 114 normalizes Ak to Ak_Normalized according to equation (1). Normalization is also called min-max scaling. As can be seen from equation (1), in the data set generated by [N], the minimum value of the normalized content parameter is 0 and the maximum value is 1.

[0067] Next, an example of the preprocessing method [S] will be described. The following illustrates a case where the training preprocessing unit 114 standardizes Ak. First, the training preprocessing unit 114 derives the mean value (Akmean) and standard deviation (Aksd) of Ak from the content parameter set.

[0068] Next, the learning preprocessing unit 114 performs the following: Ak_Standardized is calculated as follows: Ak_Standardized=(Ak-Akmean) / Aksd (2). Ak_Standardized is referred to as the standardized k-th content parameter (more specifically, the standardized k-th content parameter of the previous drawing). The standardized first to L-th content parameters are collectively referred to as the standardized content parameters (more specifically, the standardized content parameters of the previous drawing). The standardized content parameters are an example of preprocessed content parameters.

[0069] As described above, the learning preprocessing unit 114 standardizes Ak to Ak_Standardized according to equation (2). As can be seen from equation (2), in the data set generated by [S], the mean value of the standardized content parameters is 0 and the standard deviation is 1.

[0070] The training preprocessing unit 114 may generate a table (standardization data table) that records Akmax and Akmin obtained in the process [S]. Similarly, the training preprocessing unit 114 may generate a table (normalization data table) that records Akmean and Aksd obtained in the process [N].

[0071] 7 are examples of a normalization data table and a standardization data table in the reference embodiment. Specifically, the normalization data table 700A and the standardization data table 700B are tables generated in conjunction with the generation of dataset 1.

[0072] As can be understood from the above explanation, Akmax and Akmin are not obtained for the kth content parameter to which [N] is not applied. For this reason, in normalization data table 700A, dummy values ​​(e.g., 0) are assigned to Akmax and Akmin for the kth content parameter to which [N] is not applied.

[0073] In the above example, the first to third post-preprocessing content parameters and the Lth post-preprocessing content parameter in data set 1 are all derived by a preprocessing method other than [N]. Therefore, as shown in Fig. 7, in normalization data table 700A, for the first to third post-preprocessing content parameters and the Nth post-preprocessing content parameter, a dummy value of 0 is recorded as Akmax and Akmin.

[0074] In the reference embodiment, a case where A1mean=71.9, A1sd=10.5, A2mean=2.4, and A2sd=0.6 are obtained by the training preprocessing unit 114 is exemplified. In this case, the training preprocessing unit 114 records these values ​​in the first content parameter and second content parameter items of the standardization data table 700B.

[0075] Note that for the k-th content parameter to which [S] is not applied, Akmean and Aksd are not obtained in the first place. Therefore, in the standardization data table 700B, for the k-th content parameter to which [S] is not applied, a dummy value (e.g., 0) is assigned to Akmean and Aksd. In the example of FIG. 7, both the post-preprocessing third content parameter and the preprocessing L-th content parameter are derived by a preprocessing method other than [S]. Therefore, in the standardization data table 700B, for the post-preprocessing third content parameter and the preprocessing L-th content parameter, a dummy value of 0 is recorded as Akmax and Akmin.

[0076] The learning preprocessing unit 114 may generate a table (dataset-preprocessing method correspondence table) that indicates the correspondence between each dataset and each preprocessing method. Table TB4 in FIG. 8 is an example of the dataset-preprocessing method correspondence table. Note that in the example of FIG. 8, the preprocessing method corresponding to dataset 1 is hatched, and the reason for this will be described later.

[0077] 9 illustrates some of the multiple data sets generated by the learning preprocessing unit 114. In FIG. 9, reference numeral 900A indicates data set 1, reference numeral 900B indicates data set 2, and reference numeral 900C indicates data set 4. FL x5 FN As can be understood from the above explanation, the first and second preprocessed content parameters in dataset 1 are the first and second standardized content parameters, respectively. The first and second standardized content parameters are values ​​obtained by standardizing the first and second content parameters according to the above-mentioned formula (2).

[0078] As can be seen from Fig. 8 above, Dataset 2 in the example of Fig. 9 is generated by applying [L] to the third content parameter, unlike Dataset 1. The preprocessing methods for the other content parameters are the same as those in the example of Dataset 1.

[0079] As described above, unlike Dataset 1, Dataset 2 does not apply [O] to the third content parameter. Therefore, the number of dimensions of Dataset 2 is smaller than the number of dimensions of Dataset 1. Specifically, as shown in Figure 5 above, the number of dimensions of Dataset 1 is 50, and the number of dimensions of Dataset 2 is 28.

[0080] As can be seen from FIG. 8 above, data set 4 in the example of FIG. FL x5 FN Unlike Datasets 1 and 2, Dataset 4 is generated by applying [O] to all content parameters. FL x5 FN The number of dimensions of dataset 4 is larger than that of datasets 1 and 2. Specifically, as shown in Figure 5, FL x5 FN The number of dimensions is 151.

[0081] (Learning phase in the learning model generation unit 113s) The processing in the learning model generation unit 113s can be roughly divided into a learning phase and a verification phase. First, the learning phase will be described. The learning model generation unit 113s receives the expanded content parameter sets (data set 1 to data set 4) from the learning preprocessing unit 114. FL x5 FN ) is acquired. Then, the learning model generation unit 113s acquires the data set 1 to the data set 4. FL x5 FN Each of the data is divided into training data and validation data.

[0082] The learning model generation unit 113s uses data sets 1 to 4 FL x5 FN The learning model generation unit 113s generates multiple learning models by applying each of a predetermined number of types of machine learning algorithms to any one of the data sets (for convenience, referred to as a data set of interest). Specifically, the learning model generation unit 113s generates multiple learning models by applying each of a predetermined number of types of machine learning algorithms using training data of the data set of interest.

[0083] As an example, the learning model generation unit 113s generates multiple learning models corresponding to the dataset 1 by applying each of a predetermined number of types of machine learning algorithms using the training data of the dataset 1. In this way, the learning model generation unit 113s generates multiple learning models corresponding to the dataset of interest.

[0084] In the example of the reference form, the multiple types of machine learning algorithms include a gradient-based machine learning algorithm and a distance-based machine learning algorithm. The gradient-based machine learning algorithm is a machine learning algorithm that uses gradient descent. Specific examples of gradient-based machine learning algorithms include DT (Decision Tree), LR (Logistic Regression), and NN (Neural Network). The above-mentioned multinomial logistic regression is an example of RS.

[0085] More precisely, the DT according to an embodiment of the present invention refers to a GBDT (Gradient Boosting DT, gradient boosting decision tree). Therefore, examples of the DT according to an embodiment of the present invention include XGBoost (eXtreme Gradient Boosting) and LightGBM (Light Gradient Boosting Machine).

[0086] A distance-based machine learning algorithm is a machine learning algorithm that evaluates (e.g., classifies) each piece of input data based on the distance between the pieces of input data in a metric space that represents the distribution of one or more parameters contained in each piece of input data. Specific examples of distance-based machine learning algorithms include SVM (Support Vector Machine) and multiple regression.

[0087] As described above, the learning model generation unit 113s generates the learning model for the data sets 1 to 4. FLx5 FN Multiple learning models can be generated by exhaustively applying each machine learning algorithm to each of the above. This allows a sufficient number of learning models to be evaluated (verified) in the validation phase described below.

[0088] Additionally, the learning model generation unit 113s may change each hyperparameter set of a certain machine learning algorithm. In this case, the learning model generation unit 113s generates a further learning model by applying the same machine learning algorithm using the changed hyperparameter set. In this way, the learning model generation unit 113s may generate multiple learning models by comprehensively applying the hyperparameter sets. This allows for the generation of even more learning models (see FIG. 10, described later).

[0089] (Verification phase in the learning model generation unit 113s) Next, the verification phase will be described. The learning model generation unit 113s evaluates the quality of each of the multiple learning models generated in the learning phase using the data sets 1 to 4. FL x5 FN Using each of the above (more specifically, data sets 1 to 4) FL x5 FN The evaluation is performed using the respective validation data.

[0090] As an example, consider dataset j as the dataset of interest. j is a subscript indicating the column number of TB5 (see FIG. 10), which will be described later. The learning model generation unit 113s obtains, for each of a plurality of learning models corresponding to dataset j, an index value indicating the prediction accuracy (determination accuracy) of each of the plurality of learning models using the validation data of dataset j. For example, the learning model generation unit 113 inputs the validation data of dataset j to model (i, j), which will be described later, to output (derive) the index value to model (i, j). The index value can also be expressed as an index value indicating the quality of model (i, j).

[0091] In the reference embodiment, the learning model generation unit 113s acquires Accuracy (correct answer rate) as the index value. Therefore, the prediction accuracy in the reference embodiment may be referred to as search accuracy (more specifically, search accuracy of past drawings) (see FIG. 10 ). However, as will be apparent to those skilled in the art, the index value according to one embodiment of the present invention is not limited to the above example, and other index values ​​known in the field of machine learning may be used. Therefore, for example, the learning model generation unit 113s may acquire Precision (precision rate) or Recall (recall rate) as the index value. Alternatively, the learning model generation unit 113s may acquire an F-score as the index value. As is well known, the F-score is the harmonic mean of Precision and Recall.

[0092] The learning model generation unit 113s selects the best learning model according to the acquired multiple index values. In the example of the reference embodiment, the learning model generation unit 113s identifies the maximum value (maximum index value) among the multiple index values. Then, the learning model generation unit 113s selects the learning model having the maximum index value as the best learning model.

[0093] The learning model generation unit 113s may generate a table (evaluation result table) showing the evaluation results in the evaluation phase. Table TB5 in Fig. 10 is an example of the evaluation result table. In TB5, one index value is recorded so as to correspond one-to-one with one dataset, one machine learning algorithm, and one hyperparameter set (e.g., Para1).

[0094] In the example of Figure 10, Para1 and Para2 each represent a hyperparameter set (a series of hyperparameters) applied to a certain machine learning algorithm (e.g., DT). · DT Para1: Data splitting method="gni", Max depth=3, ... · DT Para2: Data division method="entropy", max depth=3, ... · LR Para1: Regularization type="l2", Regularization coefficient=1.0, … · LR Para2: Regularization type="l2", Regularization coefficient=0.5, … NN Para1: Batch size = 256, Maximum number of training iterations = 1000, ... NN Para2: Batch size = 128, Maximum number of training iterations = 1000, ... ·SVM Para1: Kernel type="rbf", Regularization coefficient=1.0, … SVM Para2: Kernel type="rbf", Regularization coefficient=0.5, ... As will be apparent to those skilled in the art, the number of hyperparameter sets is not limited to two. For example, five hyperparameter sets, Para1 to Para5, may be assigned to each machine learning algorithm.

[0095] Hereinafter, the component in the i-th row and j-th column of TB5 will be denoted as TB5(i,j). The learning model corresponding to TB5(i,j) will be referred to as model(i,j). In TB8, machine learning algorithms and hyperparameter sets for the machine learning algorithms are arranged in the row direction (i direction). Datasets are arranged in the column direction (j direction). Therefore, as an example, TB8(1,1) in FIG. 10 is the search accuracy obtained when Para1 is applied in DT. In the example of FIG. 10, TB5(i,j) = 42.1%. As mentioned above, TB5(1,1) can also be considered an index value indicating the quality of the learning model generated by DT to which Para1 is applied, i.e., model(1,1).

[0096] For convenience of explanation, the row number corresponding to Para1 of NN is denoted as im in the example of Fig. 10. In the example of Fig. 10, TB5(im,1) = 81.6% is the maximum value among each TB5(i,j) (see the hatched cells in TB5).

[0097] As described above, the learning device 11s generates multiple learning models based on the content parameter set (for convenience, referred to as the first drawing type content parameter set) of drawings a1 to M1 (outline drawings).The learning device 11 then evaluates the quality of each of the multiple learning models generated based on the first drawing type content parameter set (more specifically, derives an index value for each of the multiple generated learning models).

[0098] As in the above example for the outline drawing, the learning device 11s generates multiple learning models for each type of drawing based on the content parameter set of the drawing, and then evaluates the quality of each of the multiple learning models generated based on the content parameter set.

[0099] As one example, the learning device 11s generates multiple learning models based on a content parameter set (for convenience, referred to as a second content drawing type parameter set) for drawings a2 to M2 (assembly drawings). Then, the learning device 11 evaluates the quality of each of the multiple learning models generated based on the second drawing type content parameter set. As another example, the learning device 11s generates multiple learning models based on a content parameter set (for convenience, referred to as an Nth drawing type content parameter set) for drawings aN to MN (configuration drawings). Then, the learning device 11s evaluates the quality of each of the multiple learning models generated based on the Nth drawing type content parameter set.

[0100] In the reference embodiment, as described above, among all the index values ​​derived based on the first drawing type content parameter set to the Nth drawing type content parameter set, TB5(im,1) is assumed to be the maximum value. Therefore, in the reference embodiment, the learning model generation unit 113s identifies TB5(im,1) as the maximum index value. Then, the learning model generation unit 113s selects the learning model having the maximum index value, i.e., model (im,1), as the best learning model. As described above, in the example of the reference embodiment, the learning model generation unit 113s selects the learning model with the highest quality as the best learning model among the multiple learning models generated in the learning phase. Note that in this specification, the machine learning algorithm corresponding to the best learning model is referred to as the best machine learning algorithm. In the example of FIG. 10, the best machine learning algorithm is NN.

[0101] As will be apparent to those skilled in the art, the method for selecting the best learning model is not limited to the above example. The learning model generation unit 113s may select the best learning model from among multiple learning models based on multiple index values. For example, the learning model generation unit 113s may derive a statistical value based on multiple index values ​​and select the best learning model based on the statistical value. The maximum index value in the reference embodiment is an example of a statistical value.

[0102] (Additional information about the learning phase) However, unlike gradient-based machine learning algorithms, distance-based machine learning algorithms are known to be affected by the so-called "curse of dimensionality." For this reason, when a dataset of interest has a high number of dimensions, a learning model generated by a distance-based machine learning algorithm (hereinafter referred to as a distance-based learning model) tends to be lower quality than a learning model generated by a gradient-based machine learning algorithm (hereinafter referred to as a gradient-based learning model). For this reason, when a dataset of interest has a high number of dimensions, it is considered that the distance-based learning model generated using the dataset of interest is unlikely to be selected as the best learning model.

[0103] Therefore, in a reference embodiment, it is preferable that the learning model generation unit 113 stops generating a learning model using a distance-based machine learning algorithm with a data set of interest when the number of dimensions of the data set of interest is equal to or greater than a predetermined dimension threshold Dth. This can prevent the generation of a learning model that is expected to be of low quality, thereby reducing the computational cost in the learning phase. In addition, it can also reduce the computational cost in the subsequent evaluation phase.

[0104] In the field of machine learning, it is empirically known that when the number of dimensions of a dataset is 30 or more, the quality of a distance-based learning model tends to decrease. Therefore, for example, Dth may be set to a predetermined value of 30 or more. In the reference embodiment, a case where Dth is set to 30 is exemplified.

[0105] As described above, in the example of the reference embodiment, the number of dimensions of Dataset 1 is 50, and Dataset 4 is 50. FL x5 FN The number of dimensions of Data Set 1 and Data Set 4 is 151. Therefore, in the example of FIG. 10, the learning model generation unit 113s FL x5 FN For this, we do not generate a learning model using a distance-based machine learning algorithm (e.g., SVM) (see cells marked with an "X" in TB5).

[0106] From the above, in the example of FIG. 10, the learning model generation unit 113s generates the learning model for the dataset 1 and the dataset 4. FL x5 FN The learning model generation unit 113s applies only gradient-based machine learning algorithms (e.g., DT, LR, and NN) to the dataset 1 and the dataset 4 to generate a learning model. FL x5 FN For , we generate only a gradient-based learning model without generating a distance-based learning model.

[0107] On the other hand, when the number of dimensions of the dataset of interest is less than Dth, the learning model generation unit 113s may apply a distance-based machine learning algorithm to generate a learning model using the dataset. In the example of the reference embodiment, the number of dimensions of dataset 2 is 28. Therefore, in the example of FIG. 10, the learning model generation unit 113 generates a gradient-based learning model for dataset 2 and also generates a distance-based learning model.

[0108] (Processing after the verification phase in the learning model generation unit 113s) The learning model generation unit 113s selects one dataset corresponding to the best learning model as the best dataset from among the datasets included in TB5. In the example of Fig. 10, the learning model generation unit 113s selects dataset 1 as the best dataset.

[0109] Next, the learning model generation unit 113s selects the preprocessing method corresponding to the best data set as the best preprocessing method. In the example of the reference embodiment, the learning model generation unit 113s refers to the above-mentioned TB4 and reads out the preprocessing method corresponding to data set 1 as the best preprocessing method (see the hatched area in Figure 8). As can be understood from the above explanation, the learning model generation unit 113s selects the preprocessing method corresponding to the best learning model as the best preprocessing method.

[0110] Next, the learning model generation unit 113s updates TB3init by recording the best preprocessing method in the "preprocessing method" field in the above-mentioned TB3init. In this specification, the updated content parameter setting initial table is referred to as TB3new. FIG. 11 shows an example of TB3new. In the example of FIG. 11, the preprocessing method corresponding to dataset 1 (in other words, the preprocessing method corresponding to the best learning model) is recorded in the "preprocessing method" field as the best preprocessing method.

[0111] (Drawing search device 12) Next, a description will be given of the drawing search device 12. The drawing search device 12 includes a new drawing data acquisition unit 121, a new drawing content parameter acquisition unit 122 (target drawing content parameter acquisition unit), a search preprocessing unit 125, and a search unit 126.

[0112] The drawing search device 12 searches for at least one drawing by matching the target drawing with each of a plurality of search target drawings using a learning model (in the example of the reference embodiment, the best learning model) generated by the learning device 11s. Drawing ND in the reference embodiment is an example of a target drawing. As will be described below, the drawing search device 12 uses the learning model to match drawing ND with each of drawings a1 to MN.

[0113] (Getting new drawings) The new drawing data acquisition unit 121 is a functional unit paired with the past drawing data acquisition unit 111. As an example, the new drawing data acquisition unit 121 acquires a predetermined new drawing (e.g., drawing ND) included in the new property data set of the new property drawing DB 92 when the input unit 71 receives a predetermined user operation. The new drawing data acquisition unit 121 supplies the acquired drawing ND to the new drawing content parameter acquisition unit 122.

[0114] (Getting the content parameter set corresponding to the new drawing) The new drawing content parameter acquisition unit 122 is a functional unit paired with the past drawing content parameter acquisition unit 112. The new drawing content parameter acquisition unit 122 acquires a content parameter set corresponding to the drawing ND by the same processing as the past drawing content parameter acquisition unit 112. That is, the new drawing content parameter acquisition unit 122 acquires the content parameters of the drawing ND by analyzing the drawing ND by the same analysis method as the past drawing content parameter acquisition unit 112.

[0115] Hereinafter, the k-th content parameter of the drawing ND will also be referred to as Ck. Note that, to distinguish from the above-mentioned search target drawing content parameters, the content parameters of the target drawing (drawing ND) will also be referred to as target drawing content parameters. Furthermore, the k-th content parameter of the target drawing will also be referred to as target drawing k-th content parameter. The target drawing content parameter may also be referred to as new drawing content parameter. Therefore, the target drawing k-th content parameter may also be referred to as new drawing k-th content parameter.

[0116] As described above, the new drawing content parameter acquisition unit 122 sets C1 to CL by performing the same processing on the drawing ND as the past drawing content parameter acquisition unit 112. Thereafter, the new drawing content parameter acquisition unit 122 may generate a new drawing content parameter table TB-ND indicating C1 to CL. An example of TB-ND is shown in Fig. 12.

[0117] (An example of processing in the search preprocessing unit 125) The search preprocessing unit 125 acquires a content parameter set (for convenience, referred to as a new drawing content parameter set) corresponding to the drawing ND from the new drawing content parameter acquisition unit 122. Specifically, the new drawing content parameter set means a data set including the first to Lth content parameters (C1 to CL) of the drawing ND. As an example, the search preprocessing unit 125 acquires the above-mentioned TB-ND from the new drawing content parameter acquisition unit 122.

[0118] Furthermore, the search preprocessing unit 125 acquires the best preprocessing technique from the learning model generation unit 113s. As an example, the search preprocessing unit 125 acquires TB3 from the learning model generation unit 113s and reads out the best preprocessing technique from TB3.

[0119] Next, the search preprocessing unit 125 generates a preprocessed new drawing content parameter set by performing preprocessing on the new drawing content parameter set according to the best preprocessing method. That is, the search preprocessing unit 125 generates a preprocessed new drawing content parameter set by performing the same preprocessing method as the best preprocessing method on each of C1 to CL. In the example of the reference embodiment, the search preprocessing unit 125 (i) performs [S] on C1, (ii) performs [S] on C2, (iii) performs [O] on C3, and (iv) performs [L] on CL. vinegar.

[0120] The new drawing content parameter set may also be referred to as a target drawing content parameter set. Accordingly, the pre-processed new drawing content parameter set may also be referred to as a pre-processed target drawing content parameter set.

[0121] The search preprocessing unit 125 may generate a table indicating the post-preprocessing new drawing content parameter set (post-preprocessing new drawing content parameter table). TB-NDP shown in Fig. 13 is an example of the post-preprocessing new drawing content parameter table in the reference embodiment. As is clear from the above description, the post-preprocessing new drawing content parameter set has the same data structure as dataset 1 (see also Fig. 6 above).

[0122] An example of the preprocessing method [S] in the search preprocessor 125 is as follows: In the following, a case where Ck (the k-th content parameter of the drawing ND) is standardized by the search preprocessor 125 will be described.

[0123] First, the search preprocessing unit 125 references the standardization data table 700B and obtains Akmean and Aksd. Ck_Standardized=(Ck-Akmean) / Aksd …(3) Ck_Standardized is calculated as follows. Ck_Standardized is the standardized k-th content parameter in the drawing ND. Ck_Standardized is also referred to as the standardized new drawing k-th content parameter. The standardized new drawing k-th content parameter is an example of the preprocessed new drawing k-th content parameter.

[0124] An example of the preprocessing method [N] in the search preprocessing unit 125 will be described as follows. First, the search preprocessing unit 125 references the normalization data table 700A described above to obtain Akmax and Akmin. Next, the search preprocessing unit 125 Ck_Normalized=(Ck-Akmin) / (Akmax-Akmin) …(4) Ck_Normalized is calculated as follows. Ck_Normalized is the kth content parameter after normalization in drawing ND. Ck_Normalized is also referred to as the new drawing kth content parameter after normalization. The new drawing kth content parameter after normalization is another example of the new drawing kth content parameter after preprocessing.

[0125] (Search phase in search unit 126) The search unit 126 acquires the post-preprocessing new drawing content parameter set from the search preprocessing unit 125. The search unit 126 also acquires the best learning model from the learning model generation unit 113. The search unit 126 inputs the post-preprocessing new drawing content parameter set to the best learning model. Then, the search unit 126 acquires the output of the best learning model corresponding to the post-preprocessing new drawing content parameter set from the best learning model.

[0126] As an example, consider a case where each learning model in the reference embodiment is a learning model trained to output (derive) a relevance score, which is a score (index) indicating the degree of relevance of each past drawing (drawings a1 to MN) to drawing ND. For an example of a method for deriving the relevance score, please refer to Patent Document 1.

[0127] In this case, the search unit 126 inputs the preprocessed new drawing content parameter set into the best learning model, causing the best learning model to output a relevance score corresponding to the preprocessed new drawing content parameter set. Then, the search unit 126 searches for at least one past drawing corresponding to the drawing ND based on the output (e.g., relevance score) of the best learning model. For an example of a method for searching for the past drawing based on the relevance score, see Patent Document 1. As in Patent Document 1, the search unit 126 may cause the display unit 72 to display data indicating the search results.

[0128] (Effect of reference form) According to the information processing system 100s (information processing device 1s) of the reference embodiment, similar to the prior art (the technology of Patent Document 1), it is possible to improve user convenience in drawing searches more than ever before. In addition, according to the learning device 11s, unlike the prior art, multiple types of preprocessing methods are comprehensively applied to the search target drawing content parameter set (past drawing content parameter set), thereby expanding the search target drawing content parameter set. In other words, multiple preprocessed search target drawing content parameter sets are generated.

[0129] Next, by applying multiple machine learning algorithms, multiple preprocessed search target drawing content parameter sets (e.g., datasets 1 to 4) are generated. FL x5 FN ) to generate multiple learning models. Then, the best learning model is selected from the multiple learning models based on an index value indicating the quality of each of the multiple learning models (e.g., retrieval accuracy of past drawings). In other words, the best machine learning algorithm is selected from multiple machine learning algorithms based on the index value. Then, the best preprocessing method corresponding to the best learning model is selected.

[0130] In general, the quality of a learning model generated by a machine learning algorithm may vary depending on the preprocessing method applied to the learning data (e.g., the set of drawing content parameters to be searched). In addition, the quality of the learning model may also vary depending on the type of machine learning algorithm applied to the preprocessed learning data.

[0131] When an appropriate preprocessing method and machine learning algorithm are selected, it is possible to generate a high-quality learning model. However, there are a wide variety of preprocessing methods and machine learning algorithms that can be applied in the field of machine learning. For this reason, it is not necessarily easy for a user to manually select a combination of a preprocessing method and a machine learning algorithm that is suitable (ideally, optimal) for improving the quality of a learning model.

[0132] Therefore, as described above, the learning device 11s comprehensively evaluates each of the generated learning models based on the index values ​​to select the best learning model and the best preprocessing method. That is, the best learning model and the best preprocessing method are selected by performing a grid search on the generated learning models.

[0133] According to the above configuration, the best learning model and the best preprocessing method can be identified without manual selection by the user. That is, the learning device 11 can automatically select a combination of a preprocessing method and a machine learning algorithm that is expected to be optimal (or at least suitable) for improving the quality of the learning model.

[0134] Thereafter, in the drawing search device 12, preprocessing is performed on the target drawing content parameter set (new drawing content parameter set) according to the best preprocessing technique selected by the learning device 11s. That is, according to the best preprocessing technique, a preprocessed target drawing content parameter set (preprocessed new drawing content parameter set) is generated.

[0135] According to the above configuration, a preprocessed target drawing content parameter set is generated as an input data set having a data structure suitable for the best learning model. Therefore, by inputting the preprocessed target drawing content parameter set into the best learning model, it is possible to obtain a learning model output (e.g., a relevance score derived by the best learning model) with higher accuracy than that of the prior art.

[0136] As described above, the information processing system 100s can cause the drawing search device 12 to perform a search using (i) the best preprocessing method selected in advance by the learning device 11s and (ii) the best learning model generated in advance by the learning device 11s. As a result, the information processing system 100s can achieve even higher search accuracy than the prior art.

[0137] (Supplementary information in reference format) An example of the preprocessing method [L] will be described below. In a reference embodiment, the learning preprocessing unit 114 may label-encode raw values ​​(content parameters acquired by the past drawing content parameter acquisition unit 112) according to a raw value-label value conversion table. Specifically, the learning preprocessing unit 114 may convert raw values ​​into label values ​​according to a table indicating the correspondence between a certain raw value and the label value corresponding to the raw value (hereinafter referred to as the raw value-label value conversion table).

[0138] As an example, in the information processing system 100s, individual raw value-label value conversion tables are set in advance for the first to Lth content parameters. Hereinafter, the raw value-label value conversion table corresponding to the kth content parameter will be referred to as the kth raw value-label value conversion table.

[0139] Some of the multiple raw value-label value conversion tables are illustrated in Fig. 14. In Fig. 14, (i) reference numeral 1400-1 represents the first raw value-label value conversion table, (ii) reference numeral 1400-2 represents the second raw value-label value conversion table, (iii) reference numeral 1400-3 represents the third raw value-label value conversion table, and (iv) reference numeral 1400-L represents the Lth raw value-label value conversion table.

[0140] 14, the training preprocessing unit 114 may label encode the k-th content parameter according to the k-th raw value-to-label value conversion table. For example, the training preprocessing unit 114 label encodes the first content parameter according to the first raw value-to-label value conversion table. Also, the training preprocessing unit 114 label encodes the L-th content parameter according to the L-th raw value-to-label value conversion table.

[0141] As described above, the preprocessing method [L] may be applied to the k-th content parameter regardless of the variable type of the k-th content parameter (i.e., whether the k-th content parameter is VL or VN). Note that, as described above, when the k-th content parameter is VL, [L] can be said to be a preprocessing method equivalent to [R].

[0142] As another example, in the information processing system 100s, a table that combines the first to Lth raw value-label value conversion tables (hereinafter referred to as an integrated raw value-label value conversion table) may be created in advance. Reference numeral 1500 in Fig. 15 denotes an example of the integrated raw value-label value conversion table.

[0143] 15, the kth row of the integrated raw value-label value conversion table corresponds to the kth raw value-label value conversion table. Therefore, the learning preprocessing unit 114 may refer to the kth row of the integrated raw value-label value conversion table and label encode the kth content parameter. For example, the learning preprocessing unit 114 refers to the second row of the integrated raw value-label value conversion table and label encodes the second content parameter.

[0144] [Embodiment 1] 16 is a block diagram showing the configuration of the main parts of the information processing system 100 of embodiment 1. The information processing device of the information processing system 100 is referred to as the information processing device 1. The control device of the information processing device 1 is referred to as the control device 10. The learning device of the control device 10 is referred to as the learning device 11 (model generation device).

[0145] Unlike the learning device 11s, the learning device 11 further includes a determination unit 115. Furthermore, the learning device 11 includes a learning model generation unit 113 (learning unit) instead of the learning model generation unit 113s of the learning device 11s. Before describing the operation of each unit of the learning device 11, possible improvements in the reference embodiment will be described below.

[0146] As described above, in the reference embodiment, the content parameter set is expanded by applying multiple types of preprocessing methods to the k-th content parameter according to the k-th content parameter variable type information. Next, the best learning model and the best preprocessing method are selected through generation and verification (evaluation) of a learning model using the expanded content parameter set.

[0147] However, in the reference embodiment, depending on the preprocessing method applied to the k-th content parameter, multicollinearity may occur between multiple explanatory variables included in the expanded content parameter set (between multiple preprocessed content parameters). As can be understood by those skilled in the art, if a learning model is generated using multiple explanatory variables in which multicollinearity occurs, the quality of the learning model may be degraded.

[0148] Furthermore, as will be understood by those skilled in the art, the risk of multicollinearity increases as the number of dimensions of training data for generating a certain training model (in the example of the reference embodiment, the number of post-preprocessing content parameters) increases. As described above, one of the multiple types of preprocessing methods may be [O] (one-hot encoding). As can be understood from the explanation in the reference embodiment, [O] can be said to be a typical example of a preprocessing method that results in an increase in the number of post-preprocessing content parameters.

[0149] From this, it is feared that the risk of multicollinearity increases as [O] is applied to more k-th content parameters. Therefore, in order to further improve the quality of the learning model (for example, in order to obtain a higher-quality best learning model), it is considered preferable to introduce a measure for eliminating multicollinearity. The learning device 11 of Embodiment 1 was newly created by the inventors of the present application based on this idea.

[0150] (An example of the processing of the determination unit 115) Hereinafter, a series of processes of the determination unit 115 for one data set obtained by expanding the content parameter set (that is, the above-mentioned target data set) will be described. Hereinafter, the case where the data set 1 shown by the reference numeral 900A in FIG. 9 above is the target data set will be described. The determination unit 115 calculates the coefficient of determination for each of two different pre-processed content parameter sets included in the data set 1.

[0151] Specifically, the determination unit 115

Equation

[0152] In equation (5), Ik is the i-th preprocessed content parameter (one preprocessed content parameter) of the P preprocessed content parameters included in dataset 1. Jk is the j-th preprocessed content parameter (another preprocessed content parameter) of the P preprocessed content parameters included in dataset 1. Q is the number of items (number of columns) in the dataset of interest. In the example of embodiment 1, Q=M. Iave is the average value of I1 to IQ. Jave is the average value of J1 to JQ.

[0153] As described above, the determination unit 115 may calculate the coefficient of determination between any two different preprocessed content parameters included in the data set of interest. In the first embodiment, the determination unit 115 calculates the coefficient of determination according to equation (5): P C2 combinations of Rij are calculated. That is, the determining unit 115 calculates Rij for each of the combination patterns of two different drawing content parameters to be searched included in the data set of interest.

[0154] As shown in equation (5), Rij can be expressed as the square of the correlation coefficient sij between Ik and Jk. Therefore, Rij can be used as an evaluation value (multicollinearity evaluation value) indicating the degree (strength) of multicollinearity between Ik and Jk. Rij in embodiment 1 is an example of a multicollinearity evaluation value.

[0155] The determination unit 115 compares each calculated coefficient of determination with a predetermined threshold Rth (coefficient of determination threshold). The coefficient of determination threshold is an example of a threshold for the multicollinearity evaluation value (multicollinearity threshold). In this specification, a multicollinearity evaluation value that satisfies the condition that "the multicollinearity evaluation value is equal to or greater than the multicollinearity threshold" is referred to as a high-risk multicollinearity evaluation value. Therefore, in the first embodiment, a coefficient of determination that satisfies the condition that "the coefficient of determination is equal to or greater than Rth" is referred to as a high-risk coefficient of determination.

[0156] In the first embodiment, the determination unit 115 compares each coefficient of determination with Rth for each preprocessed content parameter, extracts high-risk coefficients of determination from the coefficients of determination, and counts up the number of extracted high-risk coefficients of determination for each preprocessed content parameter.

[0157] In the field of data science, when the absolute value of the correlation coefficient between two explanatory variables (hereinafter referred to as the absolute value of the correlation coefficient) is 0.7 or more, it is often evaluated that there is a strong correlation (or a moderately strong correlation) between the two explanatory variables. For this reason, as an example, Rth is 0.49 (= 0.7 2 It is preferable that Rth is set to a predetermined value equal to or greater than 1 and equal to or less than 1. In the first embodiment, a case where Rth is set to 0.49 is exemplified. However, as will be apparent to those skilled in the art, Rth is not limited to the above example.

[0158] The determination unit 115 generates a table (coefficient of determination vs. number of high-risk coefficients of determination table) showing the correspondence between each Rij and the number of high-risk coefficients of determination. TB6 in FIG. 17 is an example of the coefficient of determination vs. number of high-risk coefficients of determination table. As will be described below, TB6 is updated through a series of processes in the determination unit 115. For this reason, in this specification, TB6 in the initial state is also referred to as TB6init.

[0159] An example of TB6init is shown in Figure 17. The coefficient of determination Rij between the ith preprocessed content parameter (i.e., Ik) and the jth preprocessed content parameter (i.e., Jk) is recorded in the i-th row and j-th column of TB6init. The rightmost column (P+1 column) of TB6init records the number of high-risk determination coefficients corresponding to Ik (for convenience of explanation, also referred to as "the number of high-risk determination coefficients possessed by Ik"). In the example of Figure 17, high-risk determination coefficients are hatched.

[0160] As an example, the determination unit 115 may determine, among the pre-processing content parameters in TB6 (in TB6init in the example of FIG. 17), the pre-processing content parameter having the largest number of high-risk determination coefficients (the largest number of high-risk determination coefficients) as the pre-processing content parameter to be deleted. In the example of FIG. 17, among the pre-processing content parameters included in TB6init, the pre-processing content parameter "OR_2" in the fifth row has the largest number of high-risk determination coefficients (6) (see the dotted rectangle in "Number of High-Risk Determination Coefficients" in FIG. 17). Therefore, the determination unit 115 determines the pre-processing content parameter "OR_2" as the pre-processing content parameter to be deleted. Then, the determination unit 115 updates TB6 by deleting the row and column corresponding to "OR_2" (i.e., the fifth row and fifth column) from TB6init.

[0161] TB6a in Fig. 18 is an example of TB6 obtained by updating TB6init as described above. The determination unit 115 deletes the row and column corresponding to "OR_2" from TB6init, and then updates the number of high-risk determination coefficients corresponding to Ik that was recorded in the rightmost column of TB6init. Therefore, the rightmost column of TB6a records the number of high-risk determination coefficients after the update.

[0162] In the example of Fig. 18, among the pre-processed content parameters included in TB6a, the pre-processed content parameter "current value" in the second row and the pre-processed content parameter "OR_1" in the fourth row each have the most number of high risk determination coefficients (3) (see the rectangle illustrated by the dotted line in "Number of high risk determination coefficients" in Fig. 18). In this way, it is also possible that multiple pre-processed content parameters in TB6 have the same number of most high risk determination coefficients.

[0163] Therefore, in such a case, the determination unit 115 may determine the multiple pre-processed content parameters having the largest number of high-risk determination coefficients as the deletion candidate post-processed content parameters. Next, the determination unit 115 may determine the deletion candidate post-processed content parameter having the largest number of determinations among the multiple deletion candidate post-processed content parameters as the deletion candidate post-processed content parameter. Note that the deletion candidate post-processed content parameter may also be referred to as a non-learning candidate post-processed content parameter. Similarly, the deletion candidate post-processed content parameter may also be referred to as a non-learning candidate post-processed content parameter.

[0164] In the example of FIG. 18, the maximum value of the coefficient of determination for the preprocessed content parameter "current value" in the second row is assumed to be greater than the maximum value of the coefficient of determination for the preprocessed content parameter "OR_1" in the fourth row. Therefore, the determination unit 115 determines the preprocessed content parameter "current value" in the second row as the preprocessed content parameter to be deleted (see the hatched rectangle in "Number of high risk determination coefficients" in FIG. 18). Then, the determination unit 115 updates TB6 by deleting the row and column corresponding to "current value" (i.e., the second row and second column) from TB6a.

[0165] TB6b in FIG. 19 is an example of TB6 obtained by updating TB6a as described above. After deleting the row and column corresponding to "current value" from TB6a, the determination unit 115 updates the number of high-risk determination coefficients corresponding to Ik, which was recorded in the rightmost column of TB6a. Therefore, the rightmost column of TB6b records the number of high-risk determination coefficients after the update. Thereafter, the determination unit 115 repeats updating TB6 as described above until the numbers of all high-risk determination coefficients in TB6 become 0. Note that in the example of FIG. 19, the pre-processed content parameter "OR_1" in the third row is determined as the pre-processed content parameter candidate for deletion (see the dotted rectangular portion in "Number of high-risk determination coefficients" in FIG. 19). However, in the example of FIG. 19, a pre-processed content parameter (not shown) is determined as the pre-processed content parameter to be deleted.

[0166] TB6end in Figure 20 is an example of the final TB6 obtained as a result of the repeated updating described above. As shown in Figure 20, in TB6end, the numbers of all high-risk determination coefficients are 0. The pre-processed content parameters included in TB6end are the pre-processed content parameters obtained by excluding each pre-processed content parameter to be deleted (in the range illustrated in the example of embodiment 1) found in the TB6 update process from the multiple pre-processed content parameters included in TB6init. For this reason, the pre-processed content parameters included in TB6end may be referred to as residual pre-processed content parameters.

[0167] As is clear from the above explanation, no high-risk coefficient of determination occurs between the multiple residual preprocessed content parameters. That is, the risk of multicollinearity occurring between the multiple residual preprocessed content parameters is sufficiently reduced. Therefore, the determination unit 115 may process (more specifically, prune) the dataset 1 so that only the residual preprocessed content parameters are used as explanatory variables in the learning phase.

[0168] Specifically, the determination unit 115 may generate pruned dataset 1 (pruned dataset corresponding to dataset 1) by deleting the post-preprocessing content parameters to be deleted from dataset 1. Reference numeral 900AP ​​in FIG. 21 represents an example of pruned dataset 1. As is clear from the above description, pruned dataset 1 in the example of FIG. 21 is generated by deleting "OR_2" and "current value" from dataset 1.

[0169] As described above, the determining unit 115 may generate a pruned dataset of interest corresponding to the dataset of interest by deleting the post-preprocessing content parameters to be deleted in the dataset of interest from the dataset of interest. Therefore, for example, the determining unit 115 may generate a pruned dataset 2 by deleting the post-preprocessing content parameters to be deleted in dataset 2 from dataset 2, in the same manner as in the above example regarding dataset 1. Furthermore, the determining unit 115 may generate a pruned dataset 2 by deleting the post-preprocessing content parameters to be deleted in dataset 2 from dataset 2. FL x5 FN The content parameters to be deleted after preprocessing are in dataset 4. FL x5 FN After pruning, the dataset 4 FL x5 FN may be generated.

[0170] In this way, the determination unit 115 determines the pruned first drawing type content parameter sets (pruned data set 1 to pruned data set 4) by deleting the post-preprocessing content parameters to be deleted in the post-preprocessing first drawing type content parameter set (datasets obtained by applying each of the multiple patterns of preprocessing to the first drawing type content parameter set) from the post-preprocessing first drawing type content parameter set. FL x5 FN (a set consisting of

[0171] Similar to the above example for the first drawing type content parameter set, the determination unit 115 may generate the pruned post-preprocessing second drawing type content parameter set by deleting, from the post-preprocessing second drawing type content parameter set, post-preprocessing content parameters to be deleted in the post-preprocessing second drawing type content parameter set (a data set obtained by applying each of a plurality of patterns of preprocessing to the second drawing type content parameter set). Also, the determination unit 115 may generate the pruned post-preprocessing N-th drawing type content parameter set by deleting, from the preprocessing N-th drawing type content parameter set, post-preprocessing content parameters to be deleted in the preprocessing N-th drawing type content parameter set (a data set obtained by applying each of a plurality of patterns of preprocessing to the N-th drawing type content parameter set).

[0172] (An example of processing in the learning model generation unit 113) In the first embodiment, the learning model generation unit 113 acquires the pruned, preprocessed, first drawing type content parameter set from the determination unit 115. Similar to the reference embodiment, the learning model generation unit 113 generates a plurality of learning models corresponding to the pruned, preprocessed, first drawing type content parameter set based on the pruned, preprocessed, first drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models.

[0173] Specifically, the learning model generation unit 113 divides each data set included in the pruned and preprocessed first drawing type content parameter set into training data and validation data. Then, the learning model generation unit 113 uses the training data to generate multiple learning models corresponding to the training data. Next, the learning model generation unit 113 uses the validation data to evaluate the quality of each of the multiple learning models. More specifically, the learning model generation unit 113 uses the validation data to derive index values ​​for each of the multiple learning models.

[0174] Similarly, the learning model generation unit 113 obtains a post-pruning, post-preprocessing, second drawing type content parameter set from the determination unit 115. The learning model generation unit 113 generates a plurality of learning models corresponding to the post-pruning, post-preprocessing, second drawing type content parameter set based on the post-pruning, post-preprocessing, second drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models. Furthermore, the learning model generation unit 113 obtains a post-pruning, post-preprocessing, N-th drawing type content parameter set from the determination unit 115. The learning model generation unit 113 generates a plurality of learning models corresponding to the post-pruning, post-preprocessing, N-th drawing type content parameter set based on the post-pruning, post-preprocessing, N-th drawing type content parameter set. Next, the learning model generation unit 113 evaluates the quality of each of the plurality of learning models.

[0175] In the first embodiment, the learning model generation unit 113 selects the best learning model from among the multiple generated learning models based on index values ​​derived based on the pruned, preprocessed, first drawing type content parameter set to the pruned, preprocessed, Nth drawing type content parameter set. For example, the learning model generation unit 113 selects the learning model with the largest index value from among the multiple learning models as the best learning model. Subsequently, the learning model generation unit 113 selects the preprocessing method corresponding to the best learning model as the best preprocessing method, similar to the reference embodiment. The subsequent processing in the first embodiment is similar to that in the reference embodiment.

[0176] (Effects of the First Embodiment) According to the learning device 11 of the first embodiment, it is possible to generate multiple learning models after excluding preprocessing parameters to be deleted (preprocessing parameters that are feared to have a high risk of causing multicollinearity) from multiple preprocessing parameters, and then to find the best learning model from among the multiple learning models.

[0177] In this specification, a data set obtained by deleting the post-preprocessing content parameters to be deleted in a certain post-preprocessing content parameter set (more specifically, a post-preprocessing drawing content parameter set to be searched) from the content parameter set is referred to as a post-pruning post-preprocessing content parameter set (more specifically, a post-pruning post-preprocessing drawing content parameter set to be searched). The above-mentioned post-pruning post-preprocessing first drawing type content parameter set to post-pruning post-preprocessing Nth drawing type content parameter set are all examples of post-pruning post-preprocessing content parameter sets.

[0178] As described above, the learning device 11 can generate multiple learning models based on multiple pruned and preprocessed content parameter sets. Then, the best learning model can be found from among the multiple learning models. Therefore, the learning device 11 can obtain a learning model from which the effects of multicollinearity have been eliminated as the best learning model. Therefore, a best learning model with even higher quality than the reference embodiment can be obtained. In this way, the learning device 11 can improve the quality of learning models for drawing search compared to conventional methods.

[0179] Furthermore, the learning device 11 can generate multiple learning models so that only the residual preprocessed content parameters are used as explanatory variables in the learning phase. That is, multiple learning models can be generated after reducing the number of dimensions of each data set compared to the reference form. Therefore, the computational cost required for the learning phase can be reduced compared to the reference form. For example, a high-quality best learning model can be obtained in a shorter computation time than the reference form.

[0180] Furthermore, the learning device 11 can select, as the best preprocessing method, a preprocessing method that corresponds to the best learning model from which the effects of multicollinearity have been eliminated. Therefore, it is expected that a more effective preprocessing method for improving the accuracy of drawing retrieval can be selected as the best preprocessing method.

[0181] The drawing search device 12 in the first embodiment can perform drawing search using the best learning model and the best preprocessing method selected by the learning device 11 as described above. As a result, drawing search can be performed with higher search accuracy than in the reference embodiment.

[0182] [Modification] In the first embodiment, the coefficient of determination (Rij) is used as the multicollinearity evaluation value. However, as will be apparent to those skilled in the art, the multicollinearity evaluation value is not limited to the above example. For example, the learning device 11 may use the above-mentioned absolute value of the correlation coefficient as the multicollinearity evaluation value.

[0183] In this case, the determining unit 115 may calculate the absolute value of the correlation coefficient between any two different preprocessed content parameters included in the data set of interest, in the same manner as in the first embodiment. P C2 different absolute values ​​of the correlation coefficient are calculated. In this way, the determination unit 115 calculates the absolute value of the correlation coefficient (that is, |sij|) for each preprocessed content parameter.

[0184] The determining unit 115 then compares each calculated absolute value of the correlation coefficient with a predetermined threshold sth (correlation coefficient absolute value threshold). The correlation coefficient absolute value threshold is another example of a multicollinearity threshold. In this specification, an absolute value of the correlation coefficient that satisfies the condition that "the absolute value of the correlation coefficient is equal to or greater than sth" is referred to as a high-risk absolute value of the correlation coefficient.

[0185] In this modification, the determination unit 115 compares each correlation coefficient absolute value with sth for each preprocessed content parameter, extracts high-risk correlation coefficient absolute values ​​from each correlation coefficient absolute value, and counts the number of high-risk correlation coefficient absolute values ​​extracted for each preprocessed content parameter.

[0186] As is clear from the correspondence with the above formula (5), for example, sth=Rth 1 / 2 …(6) Therefore, sth is preferably set to a predetermined value equal to or greater than 0.7 and equal to or less than 1. As an example, sth may be set to 0.7. However, as will be apparent to those skilled in the art, sth is not limited to the above example.

[0187] In this modification, the determination unit 115 generates a table (a correlation coefficient absolute value / high risk correlation coefficient absolute value number table) indicating the correspondence between each correlation coefficient absolute value and the number of high risk correlation coefficient absolute values. This correlation coefficient absolute value / high risk correlation coefficient absolute value number table corresponds to the above-mentioned TB6. Then, similar to the first embodiment, the determination unit 115 repeatedly updates the correlation coefficient absolute value / high risk correlation coefficient absolute value number table until the number of all high risk correlation coefficient absolute values ​​in the correlation coefficient absolute value / high risk correlation coefficient absolute value number table becomes 0. The subsequent processing is similar to that of the first embodiment.

[0188] As is clear from the above description, in one aspect of the present invention, the determination unit 115 calculates a multicollinearity evaluation value for each preprocessed content parameter. The determination unit 115 then compares each calculated multicollinearity evaluation value with a predetermined multicollinearity threshold. Specifically, the determination unit 115 compares each multicollinearity evaluation value with the multicollinearity threshold and extracts high-risk multicollinearity evaluation values ​​from among the multicollinearity evaluation values. The determination unit 115 counts the number of high-risk multicollinearity evaluation values ​​extracted for each preprocessed content parameter.

[0189] Next, the determination unit 115 generates a table (multicollinearity evaluation value / high-risk multicollinearity evaluation value count table) indicating the correspondence between each multicollinearity evaluation value and the number of high-risk multicollinearity evaluation values. This multicollinearity evaluation value / high-risk multicollinearity evaluation value count table corresponds to the above-mentioned TB6. Then, as described above, the determination unit 115 repeatedly updates the multicollinearity evaluation value / high-risk multicollinearity evaluation value count table until the counts of all high-risk multicollinearity evaluation values ​​in the multicollinearity evaluation value / high-risk multicollinearity evaluation value count table become zero.

[0190] As described above, the determination unit 115 may be configured to determine preprocessed content parameters to be deleted from among the plurality of preprocessed content parameters based on the plurality of multicollinearity evaluation values ​​derived for the plurality of preprocessed content parameters. This allows the determination unit 115 to delete the preprocessed content parameters to be deleted from the plurality of preprocessed content parameters. As a result, as described above, a learning model from which the influence of multicollinearity has been eliminated can be obtained as the best learning model.

[0191] As will be apparent to those skilled in the art, the method for selecting the preprocessed content parameters to be deleted is not limited to the example of embodiment 1. When the multicollinearity evaluation value between two different preprocessed content parameters is equal to or greater than the multicollinearity threshold, the determination unit 115 may determine one of the two search target drawing content parameters as the preprocessed content parameter to be deleted.

[0192] [Embodiment 2] 22 is a block diagram showing the configuration of the main parts of an information processing system 100V according to the second embodiment. The information processing device of the information processing system 100V is referred to as an information processing device 1V. The control device of the information processing device 1V is referred to as a control device 10V. The learning device and drawing search device of the control device 10 are referred to as a learning device 11V (model generation device) and a drawing search device 12V, respectively.

[0193] Unlike the learning device 11 of embodiment 1, the learning device 11V does not have a learning preprocessing unit 114. The determination unit and learning model generation unit of the learning device 11V are referred to as a determination unit 115V and a learning model generation unit 113V (learning unit), respectively. Also, unlike the drawing search device 12 of embodiment 1, the drawing search device 12V does not have a search preprocessing unit 125. The search unit of the drawing search device 12V is referred to as a search unit 126V.

[0194] As is clear from FIG. 22, unlike the learning device 11, the learning device 11V does not expand each content parameter set through preprocessing. Therefore, in the second embodiment, the risk of multicollinearity occurring between multiple explanatory variables (between multiple content parameters in the example of the second embodiment) is expected to be lower than in the first embodiment. However, depending on the content of each past drawing, for example, it is conceivable that multicollinearity may occur between multiple content parameters. Therefore, in the second embodiment, a determination unit 115V is provided to eliminate the influence of multicollinearity.

[0195] (An example of processing in the learning device 11V) In the second embodiment, the determination unit 115V acquires the first drawing type content parameter set to the Nth drawing type content parameter set from the past drawing content parameter acquisition unit 112. The determination unit 115V may calculate a multicollinearity evaluation value between any two different content parameters included in the first drawing type content parameter set.

[0196] In the second embodiment, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the first drawing type content parameter set (the first to Lth content parameters in the first drawing type content parameter set). That is, in the second embodiment, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the first drawing type content parameter set. L In this way, the determining unit 115 calculates C2 combinations of multicollinearity evaluation values ​​for each of the combination patterns of two different drawing content parameters to be searched included in the first drawing type content parameter set.

[0197] Next, the determination unit 115V compares each calculated multicollinearity evaluation value with a multicollinearity threshold, extracts high-risk multicollinearity evaluation values ​​from each multicollinearity evaluation value, and counts the number of extracted high-risk multicollinearity evaluation values ​​for each content parameter.

[0198] Next, the determination unit 115V generates a multicollinearity evaluation value / high-risk multicollinearity evaluation value count table, and repeats updating the multicollinearity evaluation value / high-risk multicollinearity evaluation value count table until the counts of all high-risk multicollinearity evaluation values ​​in the multicollinearity evaluation value / high-risk multicollinearity evaluation value count table become zero.

[0199] In this way, the determination unit 115V repeatedly updates the multicollinearity evaluation value / high-risk multicollinearity evaluation value count table to identify deletion target content parameters to be deleted from among the first to Lth content parameters in the first drawing type content parameter set. The deletion target content parameters can be said to be content parameters that are feared to have a high risk of causing multicollinearity. Note that the deletion target content parameters may also be referred to as non-learning content parameters.

[0200] As is clear from the above explanations, the determination unit 115V may determine one of the two search target drawing content parameters as a deletion target content parameter when the multicollinearity evaluation value is equal to or greater than a predetermined multicollinearity threshold. Then, the determination unit 115V generates a pruned first drawing type content parameter set by deleting the deletion target content parameter from the first to Lth content parameters in the first drawing type content parameter set.

[0201] As an example, consider a case where the second content parameter "current value" is identified as a content parameter to be deleted among the first to Lth content parameters in the first drawing type content parameter set. In this case, the determination unit 115V generates a pruned first drawing type content parameter set by deleting the series corresponding to the second content parameter "current value" (the second row in the example of FIG. 3) from the first drawing type content parameter set.

[0202] Similarly, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the second drawing type content parameter set (the first to Lth content parameters in the second drawing type content parameter set).Then, the determination unit 115 prunes the second drawing type content parameter set based on each multicollinearity evaluation value, thereby generating a pruned second drawing type content parameter set.

[0203] Furthermore, the determination unit 115V calculates a multicollinearity evaluation value for each content parameter included in the Nth drawing type content parameter set (the 1st to Lth content parameters in the Nth drawing type content parameter set).The determination unit 115V then generates a pruned Nth drawing type content parameter set by pruning the Nth drawing type content parameter set based on each multicollinearity evaluation value.The determination unit 115V supplies the pruned 1st to Nth drawing type content parameter sets generated as described above to the learning model generation unit 113V.

[0204] In the second embodiment, the learning model generation unit 113V generates a learning model based on the pruned first drawing type content parameter set to the pruned Nth drawing type content parameter set. Specifically, the learning model generation unit 113V generates a learning model based on the pruned first drawing type content parameter set to the pruned Nth drawing type content parameter set by executing a predetermined machine learning algorithm. The learning model generation unit 113V supplies the generated learning model to the drawing search device 12V (more specifically, the search unit 126V).

[0205] The machine learning algorithm in embodiment 2 may be any one of the multiple types of machine learning algorithms exemplified in embodiment 1. Therefore, as an example, the learning model generation unit 113V may generate a learning model based on the pruned first drawing type content parameter set to the pruned Nth drawing type content parameter set by multinomial logistic regression.

[0206] As described above, in the second embodiment, each content parameter set is not expanded by preprocessing. Therefore, in the second embodiment, the best learning model and the best preprocessing method are not determined. As is clear from this, in one aspect of the present invention, the steps of generating multiple learning models and selecting the best learning model from the multiple learning models are not required. Similarly, in one aspect of the present invention, the step of determining the best preprocessing method from the multiple preprocessing methods is not required.

[0207] (An example of processing in the drawing search device 12V) The search unit 126V (i) acquires a new drawing content parameter set from the new drawing content parameter acquisition unit 122, and (ii) acquires the learning model generated by the learning model generation unit 113V from the learning device 11V.

[0208] The search unit 126V inputs the new drawing content parameter set into the learning model, and then acquires the output of the learning model corresponding to the new drawing content parameter set from the learning model.

[0209] As an example, the search unit 126V inputs a new drawing content parameter set into the learning model, causing the learning model to output a relevance score according to the new drawing content parameter set. Then, the search unit 126V searches for at least one past drawing corresponding to the drawing ND based on the output (e.g., relevance score) of the learning model.

[0210] (Effects of the second embodiment) According to the learning device 11V in the second embodiment, a learning model can be generated after excluding content parameters to be deleted (content parameters that are feared to have a high risk of causing multicollinearity) from a plurality of content parameters. That is, according to the learning device 11V, a learning model can be generated based on a plurality of content parameters excluding the content parameters to be deleted.

[0211] In this specification, the data set obtained by deleting the deletion target content parameters from the content parameter set (more specifically, the search target drawing content parameter set) is also referred to as the pruned content parameter set (more specifically, the pruned search target drawing content parameter set). As described above, the learning device 11V can generate a learning model based on the pruned content parameter set. Therefore, the learning device 11V can obtain a learning model from which the influence of multicollinearity is eliminated. Therefore, the quality of the learning model can be improved compared to the prior art (e.g., the technology of Patent Document 1).

[0212] The drawing search device 12V in the second embodiment can perform drawing search using the learning model generated by the learning device 11V as described above. As a result, drawing search can be performed with higher search accuracy than before.

[0213] As described above, in the second embodiment, unlike the first embodiment, the selection of the best learning model and the best preprocessing method is not performed. Therefore, according to the second embodiment, the final learning model (operational model) to be used when operating the drawing search device can be obtained in a shorter time than in the first embodiment.

[0214] However, as is clear from the above description, according to the first embodiment, the best learning model can be selected as the operational model. That is, according to the first embodiment, an operational model of higher quality can be obtained compared to the second embodiment. In addition, according to the first embodiment, the best preprocessing method can be selected as the preprocessing method to be used during operation of the drawing search device (preprocessing method during operation). By applying the best preprocessing method during operation, the drawing search accuracy of the drawing search device can be further improved. Therefore, whether the configuration of the information processing device of the first or second embodiment is to be adopted may be appropriately determined by the designer of the information processing device, for example, depending on the specifications required for the information processing device.

[0215] [Software implementation example] The functions of the information processing system 100s·100·100V (hereinafter referred to as the "device" for convenience) can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (in particular, each part included in the control device 10s·10·10V).

[0216] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0217] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0218] In addition, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of one aspect of the present invention. In addition, the functions of the control blocks can be realized by, for example, a quantum computer.

[0219] As is clear from the above explanations, each process described in each of the above embodiments can be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0220] [Additional Notes] One aspect of the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of one aspect of the present invention. [Explanation of symbols]

[0221] 1,1V Information Processing Device 10,10V control device 11,11V Learning device (model generation device) 100,100V Information Processing System 111 Past Drawing Data Acquisition Department 112 Past drawing content parameter acquisition unit (acquisition unit) 113,113V Learning model generation unit (learning unit) 114 Learning preprocessing section (preprocessing section) 115, 115V decision section

Claims

1. A model generation device that generates a learning model for searching at least one drawing corresponding to a target drawing from among a plurality of search target drawings, an acquisition unit that analyzes the plurality of search target drawings to acquire a content parameter set including a plurality of content parameters related to the description content of each of the plurality of search target drawings; (i) a determination unit that calculates a multicollinearity evaluation value between two different content parameters for each combination pattern of the plurality of content parameters included in the content parameter set, and (ii) determines content parameters to be deleted from the plurality of content parameters based on the multicollinearity evaluation value; a learning unit that generates the learning model based on a pruned content parameter set obtained by deleting the deletion target content parameters from the content parameter set, The model generating device a preprocessing unit that generates a plurality of preprocessed content parameter sets each including a plurality of preprocessed content parameters by preprocessing each of the plurality of content parameters included in the content parameter set according to a combination of a predetermined plurality of types of preprocessing techniques; The determination unit (i) calculates a multicollinearity evaluation value between two different pre-processed content parameters for each combination pattern of the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets, and (ii) determines, based on the multicollinearity evaluation value, a post-processed content parameter to be deleted from the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets; The learning section is generating a plurality of learning models using a plurality of pruned post-preprocessing content parameter sets obtained by deleting the post-preprocessing content parameters to be deleted from each of the plurality of post-preprocessing content parameter sets by applying each of a plurality of predetermined types of machine learning algorithms; selecting a best learning model from the plurality of learning models based on a plurality of index values ​​indicating the quality of each of the plurality of learning models, the index values ​​being obtained by verifying each of the plurality of learning models using each of the plurality of post-pruning and post-preprocessing content parameter sets; Among the plurality of types of preprocessing methods, a preprocessing method corresponding to the best learning model is selected as a best preprocessing method; One of the multiple preprocessing techniques is one-hot encoding. Model generation device.

2. 2. The model generating device according to claim 1, wherein the learning unit selects, from the plurality of learning models, a learning model having the highest index value as the best learning model.

3. 3. The model generating device according to claim 1, wherein the determination unit determines one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets as the preprocessed content parameter to be deleted when the multicollinearity evaluation value is equal to or greater than a multicollinearity threshold.

4. The determination unit: calculating a coefficient of determination between the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets as the multicollinearity evaluation value; 4. The model generating device according to claim 3, wherein, when the coefficient of determination is equal to or greater than a coefficient of determination threshold as the multicollinearity threshold, one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets is determined to be the preprocessed content parameter to be deleted.

5. 5. The model generating device according to claim 4, wherein the coefficient of determination threshold is set to a predetermined value not less than 0.49 and not more than 1.

6. The determination unit: calculating, as the multicollinearity evaluation value, an absolute value of a correlation coefficient between the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets; 4. The model generating device according to claim 3, wherein, when the absolute value of the correlation coefficient is equal to or greater than a correlation coefficient absolute value threshold as the multicollinearity threshold, one of the two preprocessed content parameters included in each of the plurality of preprocessed content parameter sets is determined to be the preprocessed content parameter to be deleted.

7. 7. The model generating device according to claim 6, wherein the correlation coefficient absolute value threshold is set to a predetermined value not less than 0.7 and not more than 1.

8. A model generation method in which a model generation device generates a learning model for searching at least one drawing corresponding to a target drawing from among a plurality of search target drawings, the method comprising: The above model generation method is as follows: an acquisition step in which the model generation device analyzes the plurality of search target drawings to acquire a content parameter set including a plurality of content parameters related to the description content of each of the plurality of search target drawings; the model generating device performs a determination step of (i) calculating a multicollinearity evaluation value between two different content parameters for each combination pattern of the plurality of content parameters included in the content parameter set, and (ii) determining a deletion target content parameter to be deleted from the plurality of content parameters based on the multicollinearity evaluation value; a learning step in which the model generation device generates the learning model based on a pruned content parameter set obtained by deleting the deletion target content parameters from the content parameter set, The above model generation method is as follows: a preprocessing step of generating a plurality of preprocessed content parameter sets each including a plurality of preprocessed content parameters by preprocessing each of the plurality of content parameters included in the content parameter set according to a combination of a plurality of predetermined types of preprocessing techniques; the determining step further includes a step in which the model generating device (i) calculates a multicollinearity evaluation value between two different pre-processed content parameters for each combination pattern of the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets, and (ii) determines, based on the multicollinearity evaluation value, a pre-processed content parameter to be deleted from the plurality of pre-processed content parameters included in each of the plurality of pre-processed content parameter sets; The learning process is as follows: The model generating device generates a plurality of learning models using a plurality of pruned post-preprocessing content parameter sets obtained by deleting the post-preprocessing content parameters to be deleted from each of the plurality of post-preprocessing content parameter sets by applying each of a plurality of predetermined types of machine learning algorithms; a step in which the model generation device selects a best learning model from among the plurality of learning models based on a plurality of index values ​​indicating the quality of each of the plurality of learning models, the index values ​​being obtained by verifying each of the plurality of learning models using each of the plurality of post-pruning and post-preprocessing content parameter sets; the model generation device further includes a step of selecting, from the plurality of types of preprocessing methods, a preprocessing method corresponding to the best learning model as a best preprocessing method; One of the multiple preprocessing techniques is one-hot encoding. Model generation method.

Images