Information processing device, search method, and search program

Abstract

An information processing device (100) comprises: an acquisition unit (130) that acquires search content, a user category, a replacement dictionary (215) indicating a correspondence relationship between a replacement target word, a category, and replacement information that is information indicating a replacement, a vectorization model, a plurality of sets of document data, and a plurality of sets of vector data; a determination unit (140) that uses the replacement dictionary (215) to determine whether or not the search content contains a replacement target word; a replacement unit (150) that, when the search content contains a replacement target word, replaces the replacement target word with replacement information by using the user category and the replacement dictionary (215); a vector calculation unit (160) that uses the vectorization model to vectorize the replaced search content, which is search content containing the replacement information, and calculates the degree of similarity between the vectorized search content and each of the plurality of sets of vector data; and an output unit (170) that sorts the plurality of sets of document data corresponding to the plurality of sets of vector data in descending order of the degree of similarity, and outputs the plurality of sets of document data as a search result.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a search method, and a search program. [Background technology]

[0002] When searching many documents, distributed representations may be used. With distributed representations, accurate searches can be performed if the search keywords are not ambiguous. However, the search keywords are generated by the user. Therefore, the search keywords may be ambiguous. Therefore, a technique for evaluating the ambiguity of keywords has been proposed (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-045196 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above technology, ambiguity is evaluated. Even if ambiguity is evaluated, the user may not obtain the search results he or she desires. For example, "machine learning" is input as a search keyword. "Machine learning" itself has no ambiguity. However, when the user desires search results related to "machine learning" in "image processing", the search results may be output as search results related to "machine learning" in "natural language processing". In this way, the user may not obtain the search results he or she desires.

[0005] An objective of the present disclosure is to output search results that a user desires. [Means for solving the problem]

[0006] According to an aspect of the present disclosure, there is provided an information processing device, which is a device for processing a search result, a user category that is a user's category, a replacement dictionary that indicates a correspondence relationship between a word to be replaced, a category, and replacement information that is information indicating a replacement, a vectorized model, a plurality of document data, and the plurality of document data. Using the substitution dictionary the vector calculation unit uses the vectorization model to vectorize a replacement search content, which is the search content including the replacement information, and calculates a similarity between the vectorized replacement search content and each of the plurality of vector data; and an output unit arranges the plurality of document data corresponding to the plurality of vector data in order of decreasing similarity, and outputs the plurality of document data as a search result. Effect of the Invention

[0007] According to the present disclosure, it is possible to output search results that the user desires. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a search system. [Diagram 2] FIG. 2 is a diagram illustrating hardware included in an information processing device. [Diagram 3] FIG. 2 is a block diagram showing the functions of a generating device. [Figure 4] FIG. 11 is a diagram illustrating an example of category information. [Diagram 5] FIG. 11 is a diagram illustrating an example of user information. [Figure 6] FIG. 11 is a diagram illustrating an example of model information. [Figure 7] FIG. 11 is a diagram showing an example of replacement target word information. [Figure 8] FIG. 13 is a diagram illustrating an example of a replacement dictionary. [Figure 9] FIG. 13 is a diagram illustrating an example of a document table. [Figure 10] FIG. 11 is a diagram illustrating an example of vector information. [Figure 11] 13 is a flowchart illustrating an example of a generation process of a vectorized model. [Figure 12] 13 is a flowchart illustrating an example of a process executed by the generating device. [Figure 13] FIG. 2 is a block diagram showing functions of a terminal device. [Figure 14] FIG. 2 is a block diagram showing functions of the information processing device. [Figure 15] FIG. 11 is a diagram showing an example of search content information. [Figure 16] 11 is a flowchart illustrating an example of a process executed by an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. The following embodiment is merely an example, and various modifications are possible within the scope of the present disclosure.

[0010] Embodiment 1 is a diagram showing a search system. The search system includes an information processing device 100, a generating device 200, and a terminal device 300. The information processing device 100, the generating device 200, and the terminal device 300 communicate with each other via a network 10. The network 10 is a wired network or a wireless network. The information processing device 100 and the generating device 200 may be realized by a single device.

[0011] The information processing device 100 is a device that executes a search method. For example, the information processing device 100 is a personal computer (PC), a server, a smartphone, a tablet device, or the like. The generating device 200 is a device that generates various information. The terminal device 300 is a device that is used by a user.

[0012] Next, the hardware of the information processing device 100 will be described. 2 is a diagram showing hardware included in the information processing device 100. The information processing device 100 includes a processor 101, a volatile storage device 102, a non-volatile storage device 103, a communication IF (Interface) 104, an input / output IF 105, and a media IF .

[0013] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU), a field programmable gate array (FPGA), a digital signal processor (DSP), or the like. The processor 101 may be a multiprocessor. The information processing device 100 may also have a processing circuit. Furthermore, the information processing device 100 may have a microcomputer or a system on chip (SoC).

[0014] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The non-volatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the non-volatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD). The communication IF 104 communicates with the generating device 200 and the terminal device 300. The input / output IF 105 connects to an input device (e.g., a keyboard) and an output device (e.g., a display). The media IF 106 communicates with a recording medium. For example, the recording medium is a CD (Compact Disc), a DVD (Digital Versatile Disc), or a flash memory.

[0015] Like the information processing device 100, the generation device 200 and the terminal device 300 each include a processor, a volatile storage device, a non-volatile storage device, a communication IF, an input / output IF, and a media IF.

[0016] Next, before describing the functions of the information processing device 100, the functions of the generation device 200 and the terminal device 300 will be described. 3 is a block diagram showing the functions of the generating device 200. The generating device 200 has a storage unit 210, a communication unit 220, and a control unit 230. The control unit 230 includes an acquisition unit 231, a determination unit 232, a replacement unit 233, a preprocessing unit 234, a generating unit 235, and a vectorization unit 236.

[0017] The storage unit 210 may be realized as a storage area secured in a volatile storage device or a non-volatile storage device included in the generating device 200. A part or all of the communication unit 220 and the control unit 230 may be realized by a processing circuit included in the generating device 200. In addition, a part or all of the communication unit 220 and the control unit 230 may be realized as a program module executed by a processor included in the generating device 200.

[0018] The storage unit 210 stores category information 211, user information 212, model information 213, replacement target word information 214, a replacement dictionary 215, a document table 216, and vector information 217. Details of the category information 211, user information 212, model information 213, replacement target word information 214, the replacement dictionary 215, the document table 216, and vector information 217 will be described later.

[0019] The communication unit 220 communicates with the information processing device 100 and the terminal device 300 . The acquisition unit 231 acquires the category information 211 from the storage unit 210. An example of the category information 211 is shown below.

[0020] 4 is a diagram showing an example of category information 211. The category information 211 has items of category ID (identifier) ​​and category.

[0021] The acquisition unit 231 acquires the user information 212 from the storage unit 210 .

[0022] 5 is a diagram showing an example of user information. The user information 212 has items such as a user ID, attribute information, etc. For example, the attribute information indicates the user's field of expertise.

[0023] The determination unit 232 uses the category information 211 and the user information 212 to determine a category corresponding to the attribute information. For example, the determination unit 232 determines a category "image processing" corresponding to the attribute information based on "image" indicated by the attribute information. The determination unit 232 associates the determined category with the attribute information. For example, the determination unit 232 associates the category "image processing" with the attribute information "image".

[0024] Furthermore, the determination unit 232 may estimate a category corresponding to the attribute information by using a trained model. The trained model is included in the model information 213. An example of the model information 213 is shown below.

[0025] 6 is a diagram showing an example of model information. The model information 213 has items of model ID, model name, and trained model. The determination unit 232 may estimate a category corresponding to the attribute information by using the trained model, which is an attribute category estimation model. The determination unit 232 associates the estimated category with the attribute information.

[0026] The acquiring unit 231 acquires the replacement target word information 214 from the storage unit 210. An example of the replacement target word information 214 is shown below.

[0027] 7 is a diagram showing an example of replacement target word information. The replacement target word information 214 has items of replacement target word ID and replacement target word. Here, the replacement target word may be called an ambiguous word.

[0028] The replacement unit 233 generates the replacement dictionary 215 by using the categories of the user information 212 and the replacement target word information 214. An example of the replacement dictionary 215 is shown below.

[0029] FIG. 8 is a diagram showing an example of a replacement dictionary. The replacement dictionary 215 is information indicating a correspondence relationship between a replacement target word, a category, and replacement information. Specifically, the replacement dictionary 215 has items of a replacement target word, a category, and replacement information. The replacement information may be generated based on the replacement target word and the category. For example, the replacement unit 233 generates replacement information "machine learning#image processing" based on the replacement target word "machine learning" and the category "image processing". A case where replacement information is generated based on a replacement target word and a category has been described. The replacement information may be represented by different information. That is, the replacement information may be any information as long as it indicates a replacement.

[0030] The acquisition unit 231 acquires the document table 216 from the storage unit 210. An example of the document table 216 is shown below.

[0031] 9 is a diagram showing an example of a document table 216. The document table 216 has items such as document ID and document data.

[0032] The preprocessing unit 234 performs morphological analysis on the document data. The preprocessing unit 234 registers the document data after the morphological analysis in the document table 216. The preprocessing unit 234 may also perform preprocessing such as normalization.

[0033] The preprocessing unit 234 estimates a category corresponding to the document data by using a document category estimation model registered in the model information 213. The preprocessing unit 234 associates the estimated category with the document data. As a result, the category is registered in the document table 216. The user may also analyze a category corresponding to the document data, and the preprocessing unit 234 may associate the category obtained by the analysis with the document data.

[0034] The replacement unit 233 replaces the replacement target word included in the document data after the morphological analysis with the replacement information by using the replacement dictionary 215. For example, the replacement unit 233 replaces the replacement target word "machine learning" included in the document data after the morphological analysis with the replacement information "machine learning#natural language processing".

[0035] The generation unit 235 generates a trained model using the replaced document data (hereinafter, replaced document data) included in the document table 216. The generation unit 235 registers the generated trained model in the model information 213. Note that the name of the trained model is a vectorized model.

[0036] The vectorization unit 236 vectorizes the replaced document data using a vectorization model registered in the model information 213. The vectorization unit 236 registers vector data obtained by vectorization in the vector information 217. An example of the vector information 217 is shown below.

[0037] 10 is a diagram showing an example of vector information 217. Vector information 217 has items of document ID and vector data.

[0038] Next, the process of generating a vectorized model will be described with reference to a flowchart. FIG. 11 is a flowchart illustrating an example of a process for generating a vectorized model. (Step S11 ) The acquiring unit 231 acquires the document table 216 from the storage unit 210 . (Step S12) The preprocessing unit 234 performs a morphological analysis on the document data. (Step S13) The preprocessing unit 234 uses the document category estimation model to estimate a category corresponding to the document data. (Step S14) The replacing unit 233 uses the replacement dictionary 215 to replace the words to be replaced that are included in the document data after the morphological analysis with replacement information. (Step S15) The generating unit 235 generates a vectorized model by using the replaced document data included in the document table 216.

[0039] Next, the process executed by generation device 200 will be described with reference to a flowchart. FIG. 12 is a flowchart illustrating an example of a process executed by the generating device. (Step S21) The acquisition unit 231 acquires the category information 211 and the user information 212 from the storage unit 210. (Step S22) The determination unit 232 uses the category information 211 and the user information 212 to determine a category corresponding to the attribute information. (Step S23) The determining unit 232 associates the determined category with the attribute information. (Step S24) The acquiring unit 231 acquires the document table 216 from the storage unit 210.

[0040] (Step S25) The vectorization unit 236 selects one piece of replaced document data. (Step S26) The vectorization unit 236 vectorizes the selected replaced document data using the vectorization model. (Step S27) The vectorization unit 236 registers the vector data obtained by the vectorization in the vector information 217. (Step S28) The vectorization unit 236 determines whether or not all of the replaced document data has been selected. If all of the replaced document data has been selected, the process ends. If all of the replaced document data has not been selected, the process proceeds to step S25.

[0041] Next, the functions of the terminal device 300 will be described. 13 is a block diagram showing the functions of a terminal device 300. The terminal device 300 has a communication unit 310, a control unit 320, an input unit 330, a display unit 340, and a storage unit 350. The control unit 320 includes an acquisition unit 321, a search content receiving unit 322, a transmission unit 323, a result receiving unit 324, and a display control unit 325.

[0042] A part or all of the communication unit 310 and the control unit 320 may be realized by a processing circuit included in the terminal device 300. Also, a part or all of the communication unit 310 and the control unit 320 may be realized as a program module executed by a processor included in the terminal device 300. For example, the input unit 330 is realized by a touch panel or the like. For example, the display unit 340 is realized by a display. The storage unit 350 may be realized as a storage area secured in a volatile storage device or a non-volatile storage device included in the terminal device 300.

[0043] The communication unit 310 communicates with the information processing device 100 and the generating device 200 . The acquiring unit 321 acquires a user ID. For example, the acquiring unit 321 acquires a user ID input by a user. The acquiring unit 321 stores the user ID in the storage unit 350. The search content receiving unit 322 receives the search content input by the user. The search content includes the search keyword and the meaning of the query. The search content receiving unit 322 stores the search content in the storage unit 350. The transmission unit 323 transmits the user ID and the search details to the information processing device 100 via the communication unit 310. As described below, the result receiving unit 324 receives the search results from the information processing device 100. The result receiving unit 324 stores the search results in the storage unit 350. The display control unit 325 displays the search results on the display unit 340 .

[0044] Next, functions of the information processing device 100 will be described. 14 is a block diagram showing the functions of the information processing device 100. The information processing device 100 includes a storage unit 110, a communication unit 120, an acquisition unit 130, a determination unit 140, a substitution unit 150, a vector calculation unit 160, and an output unit 170.

[0045] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103 . A part or all of the communication unit 120, the acquisition unit 130, the determination unit 140, the replacement unit 150, the vector calculation unit 160, and the output unit 170 may be realized by a processing circuit. Also, a part or all of the communication unit 120, the acquisition unit 130, the determination unit 140, the replacement unit 150, the vector calculation unit 160, and the output unit 170 may be realized as a module of a program executed by the processor 101. For example, the program executed by the processor 101 is also called a search program. For example, the search program is recorded on a recording medium.

[0046] The storage unit 110 may store the user information 212, the model information 213, the replacement dictionary 215, and the vector information 217. For example, the user information 212, the model information 213, the replacement dictionary 215, and the vector information 217 are acquired from the generation device 200 by the acquisition unit 130. Then, the acquisition unit 130 stores the user information 212, the model information 213, the replacement dictionary 215, and the vector information 217 in the storage unit 110.

[0047] The storage unit 110 may store search content information 111. The search content information 111 will be described later. The communication unit 120 communicates with the generation device 200 and the terminal device 300 .

[0048] The acquiring unit 130 acquires the user ID and the search content from the terminal device 300. Here, the user may input the user ID and the search content to the information processing device 100. The acquiring unit 130 may acquire the user ID and the search content through an input operation by the user.

[0049] The acquisition unit 130 registers the user ID and the search content in the search content information 111. The acquisition unit 130 also registers the date and time when the search content was acquired in the search content information 111. The acquisition unit 130 refers to the user information 212 and acquires a category corresponding to the user ID. The acquisition unit 130 registers the acquired category in the search content information 111. Here, the category corresponding to the user ID is also called a user category. In other words, the user category is a category of the user. The user may be expressed as a searcher. The acquisition unit 130 may acquire the user category by an input operation by the user.

[0050] The acquisition unit 130 acquires the replacement dictionary 215 from an external device such as the generation device 200 or the storage unit 110 .

[0051] The determination unit 140 determines whether or not the search content contains a word to be replaced by using the replacement dictionary 215. The determination unit 140 may also perform a morphological analysis on the search content and determine whether or not the search content after the morphological analysis contains a word to be replaced.

[0052] When the search content includes the word to be replaced, the replacement unit 150 replaces the word to be replaced with replacement information by using the category corresponding to the user ID and the replacement dictionary 215. For example, when the word to be replaced is "machine learning" and the category corresponding to the user ID is "image processing", the replacement unit 150 replaces the word to be replaced "machine learning" with the replacement information "machine learning#image processing". The replacement unit 150 registers the replacement search content, which is the search content including the replacement information, in the search content information 111. Here, an example of the search content information 111 is shown.

[0053] 15 is a diagram showing an example of search content information 111. The search content information 111 has items such as a search ID, a user ID, a category, a search content, a replacement search content, and a date and time.

[0054] The acquisition unit 130 acquires the vectorized model from an external device such as the generation device 200 or the storage unit 110 .

[0055] The acquiring unit 130 acquires the vector information 217 from an external device such as the generating device 200 or the storage unit 110. That is, the acquiring unit 130 acquires a plurality of vector data. As described above, the plurality of vector data is data obtained by replacing and vectorizing a plurality of document data. In detail, the plurality of vector data is data obtained by replacing a part of the plurality of document data and vectorizing it.

[0056] The acquiring unit 130 acquires a plurality of document data corresponding to a plurality of vector data. For example, the acquiring unit 130 acquires the plurality of document data from an external device such as the generating device 200. For example, the acquiring unit 130 may acquire the plurality of document data based on a document ID included in the vector information 217.

[0057] The vector calculation unit 160 vectorizes the replacement search content using a vectorization model. The vector calculation unit 160 calculates the similarity between the vectorized replacement search content and each of the multiple vector data included in the vector information 217. For example, when the similarity is calculated, cosine similarity is used. Specifically, the vector calculation unit 160 calculates the similarity between the vectorized replacement search content and the vector data "DOCVE1" using the cosine similarity. In addition, the vector calculation unit 160 calculates the similarity between the vectorized replacement search content and the vector data "DOCVE2" using the cosine similarity. In this way, the vector calculation unit 160 calculates the similarity corresponding to each of the multiple vector data.

[0058] The output unit 170 arranges the multiple document data corresponding to the multiple vector data in descending order of similarity. The output unit 170 outputs the multiple document data as search results. For example, the output unit 170 outputs the multiple document data to the terminal device 300. Also, for example, the output unit 170 outputs the multiple document data to a display of the information processing device 100. The output unit 170 may output, as search results, document data corresponding to vector data having a similarity equal to or greater than a predetermined threshold value, among the multiple document data corresponding to the multiple vector data. This allows the information processing device 100 to output search results more desired by the user.

[0059] Next, the process executed by the information processing device 100 will be described with reference to a flowchart. FIG. 16 is a flowchart illustrating an example of processing executed by the information processing device. (Step S31) The acquiring unit 130 acquires a user ID and search details from the terminal device 300. (Step S32) The acquiring unit 130 refers to the user information 212 and acquires the category corresponding to the user ID. (Step S33) The determination unit 140 determines whether or not the search content includes the word to be replaced using the replacement dictionary 215. If the search content includes the word to be replaced, the process proceeds to step S34. If the search content does not include the word to be replaced, the process proceeds to step S37.

[0060] (Step S34) The replacing unit 150 uses the category corresponding to the user ID and the replacement dictionary 215 to replace the word to be replaced with the replacement information. (Step S35) The vector calculation unit 160 vectorizes the replacement search content, which is the search content including replacement information, by using the vectorization model. (Step S36) The vector calculation unit 160 calculates the similarity between the vectorized replacement search content and each of the multiple vector data included in the vector information 217. Then, the process proceeds to step S39.

[0061] (Step S37) The vector calculation unit 160 vectorizes the search content. (Step S38) The vector calculation unit 160 calculates the similarity between the vectorized search content and each of the multiple vector data included in the vector information 217. (Step S39) The output unit 170 arranges the multiple document data corresponding to the multiple vector data in descending order of similarity, and outputs the multiple document data to the terminal device 300 as a search result. This allows the terminal device 300 to display the search results.

[0062] According to the embodiment, the information processing device 100 outputs search results based on user categories, so that the information processing device 100 can output search results that the user desires. [Explanation of symbols]

[0063] 10 network, 100 information processing device, 101 processor, 102 volatile storage device, 103 non-volatile storage device, 104 communication IF, 105 input / output IF, 106 media IF, 110 memory unit, 111 search content information, 120 communication unit, 130 acquisition unit, 140 judgment unit, 150 replacement unit, 160 vector calculation unit, 170 output unit, 200 generation device, 210 memory unit, 211 category information, 212 user information, 213 model information, 214 replacement target word information, 215 replacement dictionary, 216 document table, 217 vector information, 220 communication unit, 230 control unit, 231 acquisition unit, 232 determination unit, 233 replacement unit, 234 preprocessing unit, 235 generation unit, 236 vectorization unit, 300 terminal device, 310 communication unit, 320 control unit, 321 acquisition unit, 322 search content receiving unit, 323 transmission unit, 324 result receiving unit, 325 display control unit, 330 input unit, 340 display unit, 350 storage unit.

Claims

1. an acquisition unit that acquires search content, a user category that is a user's category, a replacement dictionary that indicates a correspondence relationship between a replacement target word, a category, and replacement information that is information indicating replacement, a vectorization model, a plurality of document data, and a plurality of vector data obtained by replacing and vectorizing the plurality of document data using the replacement dictionary; a determination unit that determines whether or not the search content includes a word to be replaced by using the replacement dictionary; a replacement unit that, when the search content includes a word to be replaced, replaces the word to be replaced included in the search content with the replacement information by using the user category and the replacement dictionary; a vector calculation unit that uses the vectorization model to vectorize a replacement search content, which is the search content including the replacement information, and calculates a similarity between the vectorized replacement search content and each of the plurality of vector data; an output unit that arranges the plurality of document data corresponding to the plurality of vector data in order of decreasing similarity and outputs the plurality of document data as a search result; An information processing device having the above configuration.

2. the output unit outputs, as a search result, document data corresponding to vector data whose similarity is equal to or greater than a predetermined threshold value, from among the plurality of document data corresponding to the plurality of vector data. The information processing device according to claim 1 .

3. An information processing device, a search content, a user category which is a user's category, a replacement dictionary which indicates a correspondence relationship between a replacement target word, a category, and replacement information which is information indicating replacement, a vectorization model, a plurality of document data, and a plurality of vector data obtained by replacing and vectorizing the plurality of document data using the replacement dictionary, determining whether or not the search content includes a replacement target word, and if the search content includes a replacement target word, replacing the replacement target word included in the search content with the replacement information using the user category and the replacement dictionary; vectorizing the replacement search content, the search content including the replacement information, using the vectorization model; Calculating a similarity between the vectorized replacement search content and each of the plurality of vector data; Arranging the plurality of document data corresponding to the plurality of vector data in order of the degree of similarity; outputting the plurality of document data as a search result; How to search.

4. In the information processing device, a search content, a user category which is a user's category, a replacement dictionary which indicates a correspondence relationship between a replacement target word, a category, and replacement information which is information indicating replacement, a vectorization model, a plurality of document data, and a plurality of vector data obtained by replacing and vectorizing the plurality of document data using the replacement dictionary, determining whether or not the search content includes a replacement target word, and if the search content includes a replacement target word, replacing the replacement target word included in the search content with the replacement information using the user category and the replacement dictionary; vectorizing the replacement search content, the search content including the replacement information, using the vectorization model; Calculating a similarity between the vectorized replacement search content and each of the plurality of vector data; Arranging the plurality of document data corresponding to the plurality of vector data in order of the degree of similarity; outputting the plurality of document data as a search result; The search program that performs the process.

Images