Search device, search method, and search program

Abstract

To enable improvement of search target data search performance.SOLUTION: A search device 100 disclosed herein identifies a vector of a predetermined dimension corresponding to a data search task on the basis of information regarding the data search task. The search device 100 searches for search target data according to the identified vector of the predetermined dimension.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a search device, a search method, and a search program. [Background technology]

[0002] In document searches, techniques are known for improving search speed and accuracy when searching for target documents. For example, in determining the similarity of texts, in order to minimize the impact of misrecognition in optical character recognition (OCR), a conventional technique is known in which similar characters are pre-classified into groups, and each character recognized by optical character recognition is mapped to a representative character of each character group optimized for the characteristics of each OCR engine, and a match is determined based on the mapping (see, for example, Patent Document 1). The above-described conventional technique enables similarity determination that is highly resistant to misrecognition based on the characteristics of each OCR engine, thereby enabling more robust searches. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-323415 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technologies have problems in improving the search performance of the search target data. For example, the conventional technologies require a process of switching normalization tables optimized for each OCR engine, which can make it difficult to improve search speed and accuracy when the documents and situations to be searched change. [Means for solving the problem]

[0005] Therefore, in order to solve the above-mentioned problems and achieve the object, the search device of the present invention is characterized by having an identification unit that identifies a vector of a predetermined number of dimensions corresponding to the data search task based on information regarding the data search task, and a search unit that searches for the data to be searched based on the vector of the predetermined number of dimensions identified by the identification unit. [Effects of the Invention]

[0006] The present invention has the effect of enabling improvement in search performance of data to be searched. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an overall image of processing by a search device according to this embodiment. [Figure 2] FIG. 2 is a diagram illustrating the evaluation results of search performance for each number of dimensions. [Figure 3] FIG. 3 is a diagram showing an example of the configuration of a search device according to this embodiment. [Figure 4] FIG. 4 is a table showing an example of mapping information according to this embodiment. [Figure 5] FIG. 5 is a table showing an example of search data according to this embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the search process according to this embodiment. [Figure 7] FIG. 7 is a diagram showing an example of the search process according to this embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of the procedure of the search process according to this embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the procedure of the search process according to this embodiment. [Figure 10] FIG. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the search device according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment (hereinafter referred to as "embodiment") will be described with reference to the drawings. In the following description, common components are given the same reference numerals, and repeated description will be omitted. Furthermore, the description of the embodiment described below does not limit the search device, search method, and search program according to the present invention.

[0009] <Introduction> First, an introduction to this embodiment will be given. FIG. 1 is a diagram illustrating an overview of processing by a search device 100 according to this embodiment. The search device 100 shown in FIG. 1 is an example of a computer that provides a technology for searching data using vectors, by selecting vectors for each search task, each vector having a different number of dimensions depending on the characteristics of the search task, and searching for data using the selected vector. Note that this embodiment will describe an example in which the search device 100 performs a "document search" as a "data search."

[0010] (background) Semantic search is known as a technology for more efficiently searching for electronically managed documents. In semantic search, by extracting search results that match the meaning of the natural language input, it is possible to efficiently search for the target document even if it contains spelling variations. There are also known methods for more efficiently executing the above-mentioned semantic search.

[0011] For example, in determining the similarity of sentences in document search, a reference technology is known in which, in order to minimize the impact of OCR misrecognition, each OCRed character optimized for the characteristics of the OCR engine is mapped to a representative character for each character group, and a match or mismatch is determined based on that.

[0012] However, the above-mentioned reference techniques may have difficulty in improving search speed and accuracy when the documents and situations to be searched change. Therefore, another known technique for improving the search performance of similarity searches is to divide the natural language sentence to be searched, vectorize the divided natural language sentences, and use the vectors to search for related information. By vectorizing a sentence as described above, the semantic features of the sentence can be expressed in a numerical format (an array of numerical values), which can be used for similarity searches, etc.

[0013] On the other hand, vectors have characteristics depending on the number of dimensions, so it is necessary to select an appropriate vector when performing a similarity search. For example, the higher the number of dimensions of a vector, the more information it can hold, which can improve search accuracy. On the other hand, the lower the number of dimensions of a vector, the faster the search speed and the easier it is to interpret, such as ease of handling when visualizing or analyzing, and the less memory required.

[0014] An example of the characteristics of vectors for each number of dimensions will now be described with reference to Figure 2. Figure 2 is a diagram illustrating the evaluation results of search performance for each number of dimensions. The graph shown in (1) of Figure 2 is a graph in which the vertical axis indicates the search speed (ms) for each model based on vectors obtained by vectorizing a given document based on a Bidirectional Encoder Representations from Transformers (BERT)-based model and performing dimension reduction using PCA (Principal Component Analysis).

[0015] The graph shown in Figure 2 (1) shows the search speed for each vector for a model (model_128) related to a vector with a dimension of 128, a model (model_384) related to a vector with a dimension of 384, and a model (model_768) related to a vector with a dimension of 768. As shown in Figure 2 (1), the search speed tends to decrease as the number of dimensions of the vector decreases.

[0016] On the other hand, Figure 2 (2) is a table showing the evaluation results of the STS (Semantic Textual Similarity) benchmark for each model based on vectors with different dimensionalities. The table shown in Figure 2 (2) shows cosine similarity, Euclidean distance, Manhattan distance, and dot product, which are used as indices of search accuracy. For each indices, indices showing correlations such as Bearson correlation and Spearman correlation are also shown.

[0017] As shown in (2) of Figure 2, even if the search speed decreases due to the increase in the vector dimension, the search accuracy does not necessarily improve, and the impact of the vector dimension on the overall search performance can be complex. Therefore, to optimize the overall search performance, it is necessary to select the number of dimensions appropriate for the task objectives.

[0018] (Overall Overview of Processing by Search Device 100) To solve the above-mentioned problems, the search device 100 according to this embodiment selects an appropriate vector based on the type and characteristics of the search task, and searches for data using the selected vector. Returning to FIG. 1, the processing flow of the search device 100 will now be described.

[0019] The search device 100 identifies a vector with a predetermined number of dimensions (hereinafter, sometimes referred to as a "search vector") according to the data search task, based on information about the data search task input by a user or the like ((1) in FIG. 1). Then, the search device 100 searches for the data to be searched, based on the identified search vector ((2) in FIG. 1).

[0020] Through the above-described processing, the search device 100 according to this embodiment selects an appropriate vector according to the search task and performs a search, thereby achieving a data search that achieves both search speed and search accuracy. As a result, the search device 100 has the effect of enabling improvement in search performance for target data when a search task is executed.

[0021] <Explanation of Search Device 100> Next, a detailed description will be given of the functions of the search device 100 according to this embodiment. Fig. 3 is a diagram showing an example of the configuration of the search device 100 according to this embodiment.

[0022] (Search device 100) 3, the search device 100 has a communication unit 110, a storage unit 120, and a control unit 130. The search device 100 also has an input unit (not shown) such as a keyboard or a touch panel for receiving input from a user or the like, and a display unit (not shown) such as a display or a printer for displaying the results of information processing by the search device 100 to the user or the like.

[0023] (Communication unit 110) The communication unit 110 performs communication related to outputting data search results and inputting information related to search tasks. The communication unit 110 is realized by a NIC (Network Interface Card) or the like. The communication unit 110 is connected to a network via wire or wirelessly as necessary, and can transmit and receive information bidirectionally.

[0024] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores data and programs used for various processes by the control unit 130. As shown in FIG. 3 , the storage unit 120 has a mapping information DB 121, a search data DB 122, and a learning model DB 123.

[0025] (Mapping information DB121) The mapping information DB 121 is a database that stores mapping information that associates information related to a search task with information that identifies a vector with a number of dimensions according to the search task.

[0026] The mapping information DB121 stores mapping information for each search task, and for example, for the above-mentioned mapping information, information for identifying a vector with a number of dimensions corresponding to the search performance required for one search task is stored in association with one search task. Note that the above-mentioned mapping information may be stored in the mapping information DB121 by an administrator or the like, with the search task and the vector with a number of dimensions corresponding to the search performance required for the search task being associated with each other.

[0027] An example of mapping information stored in the mapping information DB 121 will now be described with reference to Fig. 4. Fig. 4 is a table diagram showing an example of mapping information according to this embodiment. As shown in Fig. 4, the mapping information DB 121 stores "field names" that identify individual pieces of information included in the mapping information, as well as "types" that are the formats of the data identified by the field names and "descriptions" that are detailed contents.

[0028] For example, the mapping information DB 121 stores information such as a "unique numerical value or character string" that identifies a search task under the field name "taskId."

[0029] The mapping information DB 121 also stores the field name "taskName" as information such as "search task name" that identifies the search task. The mapping information DB 121 also stores the field name "taskDescription" and information about the description item that stores information such as "task characteristics" that are indicators of the real-time nature and search accuracy of the search task.

[0030] Furthermore, the mapping information DB 121 stores the field name "searchApiId" as information such as information for identifying a vector with a number of dimensions according to the search performance required for the search task. The field name identified by the above-mentioned "searchApiId" corresponds to the field name "embedding_n (n is substituted with an arbitrary number)" stored in the search data DB 122, which will be described later.

[0031] (Search data DB122) The search data DB 122 is a database that stores search data in which data to be searched is associated with vectors of multiple dimensions according to the purpose of the search.

[0032] An example of search data stored in the search data DB 122 will now be described with reference to Fig. 5. Fig. 5 is a table diagram showing an example of search data according to this embodiment. As shown in Fig. 5, the search data DB 122 stores a "field name" that identifies individual information included in the search data, a "type" that is the format of the data identified by the field name, and a "description" that is detailed content.

[0033] The search data DB 122 stores search data corresponding to each piece of divided document data, and for example, stores, as the above-mentioned search data, a vector with a number of dimensions corresponding to the search performance required for multiple search tasks and information identifying the vector with that number of dimensions, in association with one piece of divided document data. Note that the above-mentioned search data may be stored in the search data DB 122 by an administrator or the like, in which one piece of divided document data is associated with a vector with a number of dimensions corresponding to the search performance required for multiple search tasks and information identifying the vector with that number of dimensions.

[0034] For example, the search data DB 122 stores information such as a "unique numerical value or character string" that identifies individual search data under the field name "id."

[0035] The search data DB 122 also stores vectors with multiple dimensions set according to the purpose of the search task, in association with the field names "embedding_1," "embedding_2," and "embedding_3," respectively. The field name identified by the above-mentioned "embedding_n (n is substituted with an arbitrary number)" corresponds to the field name "searchApiId" stored in the mapping information DB 121.

[0036] The search data DB 122 also stores information such as divided document data to be searched, in association with the field name "guidance."

[0037] (Learning model DB123) The learning model DB 123 is a database that stores predetermined learning models. Specifically, the learning model DB 123 stores learning models that are trained to receive information about a data search task as input and output information about the data search performance required for the data search task.

[0038] (control unit 130) The control unit 130 is realized by a processor, a micro processing unit (MPU), a central processing unit (CPU), or the like executing various programs stored in the storage unit 120 using RAM as a work area. The control unit 130 is also realized by an integrated circuit (IC) such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). As shown in FIG. 3 , the control unit 130 includes a receiving unit 131, a storage unit 132, an identifying unit 133, a searching unit 134, and an output unit 135.

[0039] (Reception Department 131) The receiving unit 131 receives information about a search task input by a user or the like via the input unit (not shown) or the communication unit 110 described above.

[0040] (storage section 132) The storage unit 132 stores information such as document data to be searched and vectors for search, which are input in advance, in the storage unit 120 in association with each other.

[0041] Specifically, the storage unit 132 divides and vectorizes natural language sentences included in the data to be searched based on semantic segmentation. Next, the storage unit 132 selects vectors with a predetermined number of dimensions according to the search performance required for the data search task for searching the natural language sentences included in the divided and vectorized data to be searched.

[0042] Here, the storage unit 132 associates information identifying the selected vector of a predetermined number of dimensions with the natural language sentence contained in the divided and vectorized data to be searched, and stores them in the memory unit 120 (search data DB 122).

[0043] As an example, the storage unit 132 associates "embedding_1" corresponding to the selected high-dimensional vector, "embedding_2" corresponding to the medium-dimensional vector, "embedding_3" corresponding to the low-dimensional vector, and "guidance" that identifies the divided sentence to be searched, and stores them in the search data DB 122.

[0044] In addition, the storage unit 132 stores information identifying the selected search vector in the memory unit 120 (mapping information DB 121) in association with at least one of information identifying the data search task and information regarding the data search performance required for the data search task.

[0045] For example, the storage unit 132 associates "taskName," which is information for identifying the input task, with "searchApiId," which corresponds to the selected dimension vector, and stores them in the mapping information DB 121. Furthermore, the storage unit 132 associates "taskDescription," which is information regarding search performance required for a task search related to the input task, with "searchApiId," which corresponds to the selected dimension vector, and stores them in the mapping information DB 121.

[0046] (Specific Section 133) The specifying unit 133 specifies a vector with a predetermined number of dimensions according to the search performance required for the data search task, the characteristics of the search task, and the like, based on information related to the data search task.

[0047] Specifically, as the process of identifying the vector of the predetermined number of dimensions described above, the identification unit 133 executes a process of obtaining information identifying the vector of the predetermined number of dimensions identified based on information related to the data search task from the memory unit 120 (mapping information DB 121).

[0048] For example, the identification unit 133 uses information identifying the data search task ("taskName") included in the information related to the input data search task to identify information ("searchApiId") identifying a target vector with a predetermined number of dimensions that is previously stored in the storage unit 120 (mapping information DB 121) in association with the information identifying the data search task ("taskName"). Then, the identification unit 133 acquires the information ("searchApiId") identifying the identified vector with the predetermined number of dimensions from the storage unit 120 (mapping information DB 121). Note that an example of the processing of the identification unit 133 described above will be described later in the section "First Example."

[0049] Furthermore, for example, the identification unit 133 receives information about a data search task as input and generates information about data search performance (“taskDescription”) based on a learning model that has been trained to output information about data search performance required for the data search task, such as cosine similarity, Euclidean distance, Manhattan distance, and dot product. Next, the identification unit 133 uses the generated information about data search performance to identify information (“searchApiId”) that identifies a target vector with a predetermined number of dimensions that has been previously stored in the storage unit 120 (mapping information DB 121) in association with the information about data search performance (“taskDescription”). Then, the identification unit 133 acquires the information (“searchApiId”) that identifies the identified vector with a predetermined number of dimensions from the storage unit 120 (mapping information DB 121). Note that an example of the processing by the identification unit 133 described above will be described in the section “Second Example” below.

[0050] (Search section 134) The search unit 134 searches for data to be searched, using information identifying the search vector identified by the identification unit 133. Specifically, the search unit 134 matches the information identifying the vector of the predetermined number of dimensions acquired by the identification unit 133 with information identifying a vector of the predetermined number of dimensions for search that has been set in advance. Then, the search unit 134 searches for data to be searched that is associated with the matched information identifying the vector of the predetermined number of dimensions for search. Note that examples of the processing by the search unit 134 described above will be described in the sections "First Example" and "Second Example" below.

[0051] (output unit 135) The output unit 135 outputs the target document data searched by the search unit 134 to a user, etc. Specifically, the output unit 135 outputs the target document data to a display unit, etc., provided in the information processing device operated by the user or the search device 100.

[0052] (Example of processing) Next, the search processing realized by the search device 100 according to this embodiment will be described with reference to "First example: search using taskName" and "Second example: search using a learning model" using Figures 6 and 7. Figures 6 and 7 are diagrams showing an example of the search processing according to this embodiment.

[0053] (First example) First, as a first example, a "search using taskName" will be described with reference to Fig. 6. In the first example shown in Fig. 6, the search device 100 uses "taskName," which is information for identifying a search task, to identify a vector with a predetermined number of dimensions for search that corresponds to the search task. Then, the search device 100 uses the identified vector with the predetermined number of dimensions for search to search for data such as documents to be searched.

[0054] First, the search device 100 receives an input search task ((1) in FIG. 6). Next, the search device 100 performs a process of identifying a search vector based on information identifying the search task ((2) in FIG. 6). Specifically, the search device 100 uses the taskName (information identifying the search task) of the input task ((2-1) in FIG. 6) to identify a search API Id (information identifying a target vector of a predetermined number of dimensions) stored in the storage unit 120 (mapping information DB 121).

[0055] For example, the search device 100 executes processing linked to a learning model based on the evaluation results of search performance for each vector of a predetermined number of dimensions shown in Fig. 2. As a specific example, if the search device 100 determines that Euclidean distance (Spearman correlation) is important due to the nature of the task, the search device 100 determines to perform a search using a 384-dimensional vector because there is no difference in the evaluation values ​​related to Euclidean distance (Spearman correlation) between 384 dimensions and 768 dimensions in the search performance evaluation results shown in Fig. 2 and the search speed for 384 dimensions is fast. Then, based on the determination result, the search device 100 specifies information identifying the 384-dimensional vector as a search API ID.

[0056] Then, the search device 100 acquires the identified search API Id from the storage unit 120 (mapping information DB 121) ((2-2) in FIG. 6).

[0057] The search device 100 uses the acquired search API ID to search for the documents to be searched ((3) in FIG. 6), and then outputs the searched documents.

[0058] Through the above-described processing, the search device 100 searches for documents using a vector with a number of dimensions that is set in advance according to the characteristics of the search task and the required performance related to the search as a search vector. Therefore, the search device 100 can realize a search process that achieves both search speed and search accuracy according to the search task.

[0059] (Second example) Next, as a second example, "search using a learning model" will be described with reference to FIG. 7. In the second example shown in FIG. 7, the search device 100 inputs information related to the name of a search task and the content of the search task into a trained learning model, and identifies a vector for search with a predetermined number of dimensions corresponding to the search task. Then, the search device 100 uses the identified vector for search with the predetermined number of dimensions to search data, etc. of the document to be searched.

[0060] First, the search device 100 receives an input search task ((1) in FIG. 7). Next, the search device 100 performs a process of identifying a search vector based on information identifying the search task ((2) in FIG. 7). Specifically, the search device 100 receives input such as taskDescrition, and generates information related to the search performance of the data based on a learning model that has been trained to output cosine similarity, Euclidean distance, Manhattan distance, dot product, etc. ((2-1) in FIG. 7).

[0061] Next, the search device 100 uses the information about the search performance of the generated data to identify the target search API ID stored in the storage unit 120 (mapping information DB 121). Then, the search device 100 acquires the identified search API ID from the storage unit 120 (mapping information DB 121) ((2-2) in FIG. 7).

[0062] The search device 100 searches for the documents to be searched using the acquired search API Id ((3) in FIG. 7), and then outputs the searched documents.

[0063] Through the above-described process, the search device 100 searches for documents using a vector with the number of dimensions selected based on the trained learning model as a search vector. Therefore, the search device 100 can realize a search process that achieves both search speed and search accuracy according to the search task.

[0064] (Processing Procedure) Next, the processing procedure by the search device 100 according to this embodiment will be described with reference to Fig. 8 and Fig. 9. Fig. 8 and Fig. 9 are flowcharts showing an example of the procedure of the search process according to this embodiment. In this section, "search using information that identifies a search task" and "search using information related to the search performance of data generated by a learning model" will be described as processing procedures by the search device 100.

[0065] First, the "search using information for identifying a search task" performed by the search device 100 will be described with reference to FIG.

[0066] The receiving unit 131 receives information related to a data search task (S101). Next, the identifying unit 133 extracts information for identifying the data search task from the information related to the data search task (S102). Next, the identifying unit 133 uses the information for identifying the data search task to obtain information for identifying a search vector (S103).

[0067] The search unit 134 searches for the target data using the acquired search vector (S104), and the search device 100 then ends the process.

[0068] Next, a "search using information related to the search performance of data generated by a learning model" performed by the search device 100 will be described with reference to FIG.

[0069] The receiving unit 131 receives information about a data search task (S201). Subsequently, the identifying unit 133 inputs the information about the data search task into a learning model and generates information about data search performance (S202). Next, the identifying unit 133 uses the information that identifies the data search task to obtain information that identifies a search vector (S203).

[0070] The search unit 134 searches for the target data using the acquired search vector (S204), and the search device 100 then ends the process.

[0071] (effect) Next, we will explain the effects achieved by the search device 100 according to this embodiment. As described above, the vectors used when executing search tasks such as similarity search tend to have higher search accuracy as the number of dimensions of the vector increases, and tend to have lower search accuracy as the number of dimensions of the vector decreases.

[0072] For example, (1) in Figure 2 above shows that the search speed of a model based on a 768-dimensional vector tends to be slower than that of a model based on a 128-dimensional vector. On the other hand, when comparing search accuracy using the Pearson correlation of cosine similarity as an example, the Pearson correlation of the model based on a 768-dimensional vector shows a higher value than the Pearson correlation of the model based on a 128-dimensional vector in Figure 2 (2), indicating a tendency for search accuracy to be higher.

[0073] However, although the fluctuations in search speed and accuracy due to the variation in the number of vector dimensions show a certain tendency, there is not necessarily a uniform trade-off relationship. Therefore, in order to optimize the performance of the entire search task, it is necessary not only to simply vary the number of vector dimensions, but also to select an appropriate number of dimensions depending on the purpose of the search task.

[0074] Therefore, the specifying unit 133 of the search device 100 according to this embodiment specifies a vector with a predetermined number of dimensions according to the data search task, based on information about the data search task. The search unit 134 of the search device 100 searches for the data to be searched, based on the vector with the predetermined number of dimensions specified by the specifying unit 133.

[0075] Therefore, the search device 100 of this embodiment has the effect of enabling improvement in search performance. That is, the search device 100 selects a vector with an appropriate number of dimensions according to an input search task, and executes the search task using the selected vector with the appropriate number of dimensions, thereby achieving both search performance and search speed for the search task and optimizing the execution of the search task.

[0076] Furthermore, in order to achieve the above-mentioned effects, the search device 100 executes the specific processing described below.

[0077] For example, the specification unit 133 acquires information identifying a vector of a predetermined number of dimensions specified based on information related to a data search task from the storage unit 120 (mapping information DB 121). Next, the search unit 134 matches the information identifying the vector of the predetermined number of dimensions acquired by the specification unit 133 with information identifying a vector of the predetermined number of dimensions set for search. Then, the search unit 134 searches for data to be searched that is associated with the matched information identifying the vector of the predetermined number of dimensions for search.

[0078] Through the above-described processing, the search device 100 can select a vector with an appropriate number of dimensions in accordance with the characteristics of the search task and the required search performance, and execute the search task. As a result, the search device 100 has the effect of enabling the execution of a search task to search for target data more efficiently than before, by performing a search using a selected vector with an appropriate number of dimensions.

[0079] Furthermore, for example, the identification unit 133 uses information identifying a data search task included in the information related to the input data search task to identify information identifying a target vector of a predetermined number of dimensions that is stored in advance in the storage unit 120 (mapping information DB 121) in association with the information identifying the data search task. Next, the identification unit 133 acquires information identifying the identified target vector of a predetermined number of dimensions from the storage unit 120 (mapping information DB 121). Then, the search unit 134 searches for the data to be searched using the information identifying the vector of a predetermined number of dimensions for search identified by the identification unit 133.

[0080] Through the above-described process, the search device 100 can more efficiently select a vector with an appropriate number of dimensions by using information that identifies a search task that has been previously associated with a vector with a predetermined number of dimensions. As a result, the search device 100 has the effect of enabling the search of target data with faster processing speed than conventional methods for search tasks that have been previously associated with known search tasks.

[0081] Furthermore, for example, the identification unit 133 receives information about a data search task as input and generates information about data search performance based on a learning model that has been trained to output information about data search performance required for the data search task. Next, the identification unit 133 uses the generated information about data search performance to identify information for identifying a target vector with a predetermined number of dimensions that has been previously stored in the storage unit 120 (mapping information DB 121) in association with the information about the data search performance. Then, the identification unit 133 acquires information for identifying the identified target vector with a predetermined number of dimensions from the storage unit 120 (mapping information DB 121). Then, the search unit 134 searches for the data to be searched using the information for identifying the vector with a predetermined number of dimensions for search identified by the identification unit 133.

[0082] Through the above-described process, the search device 100 can more efficiently select a vector with an appropriate number of dimensions based on information identifying a vector with a predetermined number of dimensions generated by a trained learning model. As a result, the search device 100 can select a vector with an overall appropriate number of dimensions using information such as the characteristics of the search task and the performance required for the search. Therefore, the search device 100 has the effect of enabling a search for target data with higher accuracy than conventional methods.

[0083] In addition, the search device 100 can store in advance in a memory unit document data relating to the divided data to be searched, vectors of a predetermined number of dimensions according to the search task when searching the divided document data, information relating to the search performance required for the search task, and the like.

[0084] Specifically, the storage unit 132 divides and vectorizes natural language sentences included in the data to be searched based on semantic segmentation. Next, the storage unit 132 selects vectors with a predetermined number of dimensions according to the search performance required for the data search task for searching the natural language sentences included in the divided and vectorized data to be searched.

[0085] Then, the storage unit 132 associates the information identifying the selected vectors of the predetermined number of dimensions with the natural language sentences included in the divided and vectorized data to be searched, and stores them in the memory unit 120 (search data DB 122).The storage unit 132 also associates the information identifying the selected vectors of the predetermined number of dimensions with at least one of information identifying a data search task and information related to data search performance required for the data search task, and stores them in the memory unit 120 (mapping information DB 121).

[0086] Through the above-described processing, the search device 100 stores vectors of a predetermined number of dimensions in association with each other in advance based on the characteristics of the search task, the required performance of the search, etc., thereby achieving the effect of enabling searches of target data that achieve both search accuracy and search speed.

[0087] Furthermore, the search device 100 can speed up processing by reducing search processing that exceeds the performance required for the search task by selecting a vector with a number of dimensions according to the search task that requires search speed. On the other hand, the search device 100 can execute search processing that emphasizes search accuracy, such as preventing search omissions, by selecting a vector with a number of dimensions according to the search task that requires search accuracy.

[0088] <Modification> The following describes modified examples realized by the search device 100 according to this embodiment.

[0089] (Data, etc.) The search task, vector with a predetermined number of dimensions, search vector, information about the search task, information for identifying a vector with a predetermined number of dimensions, information for identifying a vector with a predetermined number of dimensions for search, information about the search performance of data, names of functional parts of the search device 100, steps, processes, names of steps or processes, etc. used in the description of the above embodiment are merely examples and can be changed as desired.

[0090] For example, it has been explained that the mapping information DB121 stores a "field name" that identifies individual information contained in the mapping information, a "type" that is the format of the data identified by the field name, and a "description" that is the detailed content, but this is not limited to this.

[0091] For example, it has been explained that the search data DB122 stores a "field name" that identifies individual information contained in the search data, a "type" that is the format of the data identified by the field name, and a "description" that is the detailed content, but this is not limited to this.

[0092] For example, it has been explained that the learning model DB123 stores a learning model that has been trained to take information about a data search task as input and output information about the data search performance required for the data search task, but the type of the learning model described above is not particularly limited.

[0093] (Another example of data) In the present embodiment, an example has been described in which the retrieval device 100 performs a "document search" as a "data search," but the present invention is not limited to this. For example, the "data" may be data that can be vectorized, such as still images, moving images, natural language sentences, numerical values, and mathematical formulas. In other words, the retrieval device 100 can select and search vectors with an appropriate number of dimensions for various data.

[0094] (Flowcharts, etc.) The steps in the flowcharts may be interchanged as long as there is no contradiction, and some steps may not be performed. In addition, conjunctions such as "next," "continue," "further," "at this time," and "on this occasion" used in the explanation of the flowcharts do not limit the order or timing of the execution of the processes in the flowcharts.

[0095] (others) Of the processes described in the above embodiments and variations, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information, including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings, can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

[0096] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0097] The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0098] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, a control section can be read as control means or a control circuit.

[0099] Although some of the embodiments have been described in detail above with reference to the drawings, these are merely examples, and it is possible to implement the present embodiments in other forms that have undergone various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

[0100] <Hardware configuration> The search device 100 according to this embodiment is realized, for example, by a computer 1000 configured as shown in Fig. 10. Fig. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the search device 100 according to this embodiment. The computer 1000 has a configuration in which a CPU 1100, a RAM 1200, a ROM 1300, an auxiliary storage device 1400, a communication I / F (interface) 1500, and an input / output I / F (interface) 1600 are connected by a bus 1800.

[0101] The CPU 1100 operates and controls each unit based on a program stored in the ROM 1300 or the auxiliary storage device 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0102] The auxiliary storage device 1400 stores programs executed by the CPU 1100, data used by the programs, etc. The communication I / F 1500 receives data from other devices via a predetermined communication network NW (including closed-area wireless communication in this embodiment) and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network NW. The CPU 1100 controls output devices such as displays and printers, and input / output devices 1700 such as keyboards and mice, via the input / output I / F 1600. The CPU 1100 acquires data from the input / output devices 1700 via the input / output I / F 1600. The CPU 1100 also outputs generated data to the input / output devices 1700 via the input / output I / F 1600.

[0103] For example, when the computer 1000 functions as various devices according to the present embodiment, the CPU 1100 of the computer 1000 executes a program loaded onto the RAM 1200 to realize the functions of the control unit. [Explanation of symbols]

[0104] 100 Search Device 110 Communications Department 120 Storage section 121 Mapping Information DB 122 Search Data DB 123 Learning Model DB 130 Control Unit 131 Reception 132 Storage area 133 Specific part 134 Search Section 135 Output section

Claims

1. an identification unit that identifies a vector with a predetermined number of dimensions according to a data search task based on information about the data search task; a search unit that searches for data to be searched based on the vector of the predetermined number of dimensions identified by the identification unit; A search device comprising:

2. The identification unit obtaining, from a storage unit, information for identifying the vector of the predetermined number of dimensions, the vector being specified based on information about the data search task; The search unit comparing information for identifying the vector of the predetermined number of dimensions acquired by the identification unit with information for identifying a vector of the predetermined number of dimensions set in advance for search; searching for search target data associated with information identifying the matched search vector with the predetermined number of dimensions; 2. The search device according to claim 1.

3. The identification unit using information for identifying the data retrieval task included in the information about the data retrieval task to be input, identifying information for identifying the target vector of the predetermined number of dimensions that is stored in advance in a storage unit in association with the information for identifying the data retrieval task; acquiring information identifying a vector of the predetermined number of dimensions of the identified target from the storage unit; 3. The search device according to claim 1 or 2.

4. The identification unit Generate information about the search performance of the data based on a learning model that is trained to receive information about a data search task as input and output information about the search performance of the data required for the data search task; Using the generated information on the search performance of the data, identify information for identifying the target vector of the predetermined number of dimensions, which information is stored in advance in a storage unit in association with the information on the search performance of the data, acquiring information identifying a vector of the predetermined number of dimensions of the identified target from the storage unit; 3. The search device according to claim 1 or 2.

5. The natural language sentences contained in the data to be searched are divided and vectorized based on semantic segmentation, selecting a vector of the predetermined number of dimensions according to a search performance required for a data search task for searching natural language sentences included in the divided and vectorized data to be searched, storing the information for identifying the selected vectors of the predetermined number of dimensions in a storage unit in association with the natural language sentences included in the divided and vectorized data to be searched; a storage unit that stores, in a storage unit, information identifying the selected vector of the predetermined number of dimensions and at least one of information identifying the data retrieval task and information related to data retrieval performance required for the data retrieval task in association with each other; 3. The search device according to claim 1 or 2.

6. A search method to be executed by a search device, a specifying step of specifying a vector having a predetermined number of dimensions according to the data search task based on information about the data search task; a search step of searching for data to be searched based on the vector of the predetermined number of dimensions identified in the identification step; A search method comprising:

7. a step of identifying a vector having a predetermined number of dimensions according to a data search task based on information about the data search task; a search procedure for searching for data to be searched based on the vector with the predetermined number of dimensions identified by the identification procedure; A search program characterized by causing a computer to execute the above.

Images