Question analysis method and question analysis device

Abstract

This question analysis method comprises: inputting data about respective questions in a plurality of questionnaires and about options for each of the questions, one or more sets of response data and participant information for the data of each of the questions, and information about placement locations of each of the questions in the plurality of questionnaires; acquiring an assessment value for determining whether a pair of questions obtained by combining the questions are identical, on the basis of the data about the questions and the options for the questions, the one or more sets of response data and participant information, and the information about placement locations of the questions in the plurality of questionnaires that have been inputted; determining that a pair of questions for which the acquired assessment value is at or above a threshold value are identical; and executing record linkage of the pair of questions determined to be identical.　

Description

Question sentence analysis method and question sentence analysis device

[0001] The present invention relates to a question analysis method and a question analysis device.

[0002] Surveys such as health checkups and medical examinations conducted by medical institutions may include specimen tests such as blood tests and urine tests, physical measurements such as height and weight, and medical questionnaires. A medical questionnaire is a questionnaire that asks patients questions about their current physical and mental condition, medical history, and whether they have any allergies before undergoing a medical examination or checkup. The content of the medical questionnaire varies depending on the purpose of the interview, and questions may be added or the wording of the questions may be improved during the course of the survey.

[0003] When analyzing medical questionnaires, there is a strong need for an integrated overview of multiple regions, implementation periods, and subjects.Even if the structure of the medical questionnaire or the wording of the questions is changed, there is a need for a question identification method and record linkage technology that can compile response data obtained from questions with the same intent and enable integrated analysis.

[0004] Record linkage is a technique for identifying the same individual or entity in data from multiple data sources. Record linkage is useful when information related to the same individual or entity is scattered across different data sources.

[0005] Conventional question identification methods include a method of calculating linguistic similarity using a sentence vector that mechanically determines that questions with exact matches in sentences (described later) are the same question, and a method of calculating linguistic similarity using the BART score, a natural language processing technology. In both cases, questions with high linguistic similarity are determined to be the same question.

[0006] However, in a method of calculating linguistic similarity using sentence vectors, if the wording of a question is changed, the question is determined to be a different question even if it has the same intention.In a method of calculating linguistic similarity using the BART score, which is a natural language processing technology, the question texts included in the medical questionnaire have a characteristic of being in a uniform format, so that the similarity between the question texts is calculated to be high and the question texts are determined to be the same even though they have different intentions.

[0007] Adly, Noha. “Efficient Record Linkage using a Double Embedding Scheme”, Conference: Proceedings of The 2009 International Conference on Data Mining, DMIN , (2009).Lewis, Mike et al. “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.” Annual Meeting of the Association for Computational Linguistics (2019).

[0008] Because the above-mentioned problems occur with question identification, it was necessary to manually determine whether the intent of the questions was the same and then perform record linkage. This required a lot of work to identify questions before analysis, which made it difficult to perform record linkage efficiently. This issue is not limited to record linkage using questionnaires, but also arises with questionnaires that ask survey subjects about certain matters or surveys that ask people for their opinions and thoughts.

[0009] In response to the above-mentioned problems, an object of the present invention is to provide a technique that enables efficient identification of questions and performs record linkage.

[0010] One aspect of the present invention is a question analysis method that inputs data on each question and option for each question in a plurality of questionnaires, one or more pieces of answer data and participant information for each question, and information on the position of each question in the plurality of questionnaires; obtains an evaluation value for determining whether a pair of questions combining each question is identical or not based on the input data on each question and option for each question, the one or more pieces of answer data and participant information, and the information on the position of each question; determines that a pair of questions for which the obtained evaluation value is equal to or greater than a threshold is identical; and performs record linkage on the pair of questions determined to be identical.

[0011] One aspect of the present invention is a question analysis device that includes: a question sentence / option input unit that inputs data on each question sentence and option for each question sentence in a plurality of questionnaires; a response data / participant information input unit that inputs one or more pieces of response data and participant information for each question sentence; a description position information input unit that inputs information on the description position of each question sentence in the plurality of questionnaires; an evaluation value acquisition unit that acquires an evaluation value for determining whether a pair of questions combining each question sentence is identical or not, based on the input data on each question sentence and option for each question sentence, the one or more pieces of response data and participant information, and the information on the description position of each question sentence; a determination unit that determines, as identical, a pair of questions for which the evaluation value acquired by the evaluation value acquisition unit is equal to or greater than a threshold; and a record linkage implementation unit that performs record linkage on the pair of questions determined to be identical by the determination unit.

[0012] The present invention makes it possible to efficiently identify questions and perform record linkage.

[0013] 1 is a diagram illustrating an example of the configuration of a question sentence identification device according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a record linkage device according to the first embodiment. FIG. 3 is a diagram illustrating an example of information on a question sentence, options, and the position of each question sentence according to the first embodiment. FIG. 4 is a diagram illustrating an example of answer data and participant information according to the first embodiment. FIG. 5 is a flowchart illustrating the flow of question sentence identification processing performed by the question sentence identification device according to the first embodiment. FIG. 6 is a diagram illustrating an example of linguistic similarity according to the first embodiment. FIG. 7 is a diagram illustrating an example of statistics of each question and participant data according to the first embodiment. FIG. 8 is a diagram illustrating an example of position vectors according to the first embodiment. FIG. 9 is a diagram illustrating an example output of a composite function according to the first embodiment. FIG. 10 is a flowchart illustrating the flow of record linkage processing performed by the record linkage device according to the first embodiment. FIG. 11 is a diagram illustrating processing when the linguistic similarity value of options is equal to or greater than a threshold according to the first embodiment. FIG. 12 is a diagram illustrating processing when the linguistic similarity value of options is less than a threshold according to the first embodiment. FIG. 13 is a diagram illustrating an example of the configuration of a question sentence identification device according to the second embodiment. FIG. 14 is a diagram illustrating an example of teacher labels according to the second embodiment. FIG. 15 is a diagram illustrating an example of data input during learning of the question sentence identification device according to the second embodiment. FIG. 16 is a diagram illustrating an example of data input during inference of the question sentence identification device according to the second embodiment. FIG. 17 is a flowchart illustrating the flow of learning processing performed by the question sentence identification device according to the second embodiment. 10 is a flowchart showing the flow of an inference process performed by a question sentence identification device according to a second embodiment.

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] The following describes prior art related to the present invention. First, prior art for calculating linguistic similarity will be described, followed by prior art related to record linkage.

[0016] (Prior Research on Linguistic Similarity Calculation Technology) Prior research on technology for calculating linguistic similarity will be described. Technology for determining the linguistic similarity of sentences has been proposed in the field of natural language processing. Methods for calculating linguistic similarity include a method using sentence vectors as described in Non-Patent Document 1, and a method using a pre-trained model such as the BART score as described in Non-Patent Document 2.

[0017] First, we will explain the technology for calculating linguistic similarity using sentence vectors. A sentence vector is a vector (numerical value) of a sentence. TF-IDF (Term Frequency-Inverse Document Frequency) is an example of a method using sentence vectors. TF-IDF is a method of calculating the importance of a word according to how many times it appears in a sentence, and weighting each word. Here, two questions are defined as questionnaire items a and b. Cosine similarity is calculated for sentence vectors a and b converted from each questionnaire item a and b. Cosine similarity is expressed as the following formula (1).

[0018]

[0019] If the cosine similarity between two sentence vectors a and b is close to "1", the similarity is determined to be high, and if the cosine similarity is close to "0", the similarity is determined to be low.

[0020] Next, we will explain a technique for calculating linguistic similarity using a pre-trained model such as the BART score. The BART score is calculated from a trained model called BART. BART consists of two training processes: training by predicting masked words in a sentence and training by next-sentence prediction, which estimates whether two input sentences were consecutive in the original document or randomly selected from separate documents. Masked word prediction predicts the original word while utilizing the context surrounding the masked word, and learns to synthesize word embeddings using a self-attention mechanism that uses the surrounding context. Next-sentence prediction is performed by connecting a fully connected layer for binary classification to the embedding representation in the final layer corresponding to a special token placed at the beginning of the input, and is trained in parallel with word fill-in prediction.

[0021] The BART score has been proposed as an unsupervised method for estimating similarity between sentences that utilizes the encoding capabilities of BART. The BART score calculates the linguistic similarity of entire sentences based on the cosine similarity between the subword vectors of two sentences encoded by BART.

[0022] (Previous research on record linkage) This section describes previous research on record linkage. As mentioned above, record linkage is a method for identifying the same individual or entity in data obtained from multiple data sources. Record linkage is useful when related information for the same individual or entity is distributed across different data sources.

[0023] Let us assume that different data files obtained from the same subject are data files C and D, respectively, and that the record values ​​contain K common items (K is an integer equal to or greater than 1). In this case, data files C and D are each expressed as a set of records, using the following formula (2):

[0024]

[0025] Assume that data files C and D are finite populations. In the statistical model proposed by Newcombe et al. (1959) and Newcombe and Kennedy (1962), the degree of matching t between items is defined as in equation (3) to obtain a record pair (c, d) based on both data files C and D.

[0026]

[0027] The probability of observing a record matching degree t was calculated by defining it as in equation (3). An item that matches perfectly is set to 1, and an item that does not match is set to 0. If there are many "0s" in t(c, d), the matching degree t is low, and it is estimated that there is a high possibility that the two people are different people. If there are many "1s", the matching degree t is high, and it is estimated that there is a high possibility that the two people are the same person pair. １ = (c １ｌ ,…,c ｉＫ ) and d １ = (d １ｌ ,…,d ｉＫ ) and a technique for identifying records by calculating the cosine similarity with １ and １ There is also a technique for applying clustering to vectors such as these to perform record linkage (see, for example, Non-Patent Document 1). In this way, record linkage is often applied to individuals from multiple data sources to combine data. However, in the present invention, the target of record linkage is not individuals but questions.

[0028] As described above, in the method of calculating linguistic similarity using sentence vectors shown in Non-Patent Document 1, if the wording of a question is changed, the question is determined to be a different question even if it has the same intention. In the method of calculating linguistic similarity using the BART score, which is a natural language processing technology, the question texts included in the medical questionnaire have a characteristic of being uniform in format, so that the similarity between the question texts is calculated to be high and the question texts are determined to be the same, even though the questions have different intentions. Thus, when identifying questions using only linguistic similarity using conventional methods, it is difficult to identify questions that take their intention into consideration.

[0029] Therefore, in the present invention, taking into account the fact that questions included in medical questionnaires have a uniform format and that basic medical questionnaires tend to ask about patient attributes such as height, weight, gender, and age and ask detailed questions about health conditions, questions are identified not only based on the linguistic similarity of the questions and options, but also by using position vectors calculated based on information about the position of the questions, one or more pieces of answer data for each question, and statistics obtained from participant information. After identifying the questions, record linkage is performed on the questions to match the answer data for questions with the same intent. By taking into account not only linguistic similarity but also the statistical characteristics of the data and the position of each question, answer data obtained from questions with the same intent can be compiled and analyzed comprehensively, even if the structure of the medical questionnaire or the wording of the questions has changed.

[0030] The present invention will be described in detail below. First, the terminology of information used in specific processing and a method for acquiring that information will be explained. After that, the specific configuration of the device that executes the processing will be explained.

[0031] (Linguistic Similarity Between Question Sentence and Choices) The linguistic similarity between the question sentence and the choice options is calculated using an existing method capable of calculating linguistic similarity, as described above. Here, the case where BART is used will be described as an example, but any method capable of calculating linguistic similarity may be used.

[0032] (Statistical quantities obtained from one or more pieces of response data for each question and participant information) Any statistical quantities that can be calculated based on response data may be used as the statistical quantities obtained from one or more pieces of response data for each question and participant information. Examples include the missing rate for each question, the gender ratio, the regional ratio, and response statistics (such as the mean and variance) obtained from one or more pieces of response data.

[0033] (Introduction of position vectors indicating the position of the medical questionnaire) Questions included in medical questionnaires tend to have a uniform format. A basic medical questionnaire tends to first ask for patient attribute information such as height, weight, sex, and age, followed by detailed health information. Furthermore, when improving the same questionnaire, even if the wording of individual questions changes, the overall structure of the questionnaire often remains unchanged. Thus, the position of a question on the medical questionnaire is an important feature. Therefore, in this invention, a position vector indicating the position of the medical questionnaire is introduced. A position vector is a vector that represents the position of an input from the beginning of an input sequence.

[0034] The position vectorization will be described in detail below. The position vectorization can be performed using the following equation (4).

[0035]

[0036] In equation (4), the position of the question is designated as pos. The total number of questions l is multiplied by the coefficient λ and substituted into the equation. λ indicates the length of the period and is a hyperparameter. If pos is an even number, a formula using a sine function is used, and if it is an odd number, a formula using a cosine function is used. In this way, the position of the question in the questionnaire is quantified and vectorized using a sine function or cosine function.

[0037] (Regarding the Question Pair Extraction Unit of the Question Identification Device) First, a pair of questions to be identified is selected. Then, a vector combining the linguistic similarity, statistics, and position vectors between the pair of questions to be identified is combined using the composite function shown in the following formula (5). Let L be the linguistic similarity of the questions, M be the linguistic similarity of the options, S be the statistics, and I be the position vector. α, β, γ, and δ in formula (5) are weights indicating the degree of coupling, and are hyperparameters.

[0038]

[0039] Hq in formula (5) is a vector created for each question. The cosine similarity of formula (1) is calculated for the vectors a and b created for each question, and if the cosine similarity is equal to or greater than a threshold, it is determined that the questions have the same intention, and if the cosine similarity is less than the threshold, it is determined that the questions do not have the same intention.

[0040] (Regarding the processing section of the question identification device) The n questions (n ​​is an integer equal to or greater than 1) to be identified are combined using the composite function expressed in the following formula (6). Let L be the linguistic similarity of the questions, M be the linguistic similarity of the options, S be the statistic, and I be the position vector. α, β, γ, and δ in formula (6) are weights indicating the degree of combination, and are hyperparameters.

[0041]

[0042] In equation (6), Hq is a vector created for each question. The processing unit processes the vectors created for each question into a matrix that combines them pairwise. Specifically, if there are n questions to be identified, the processing unit combines features in an n × (n-1) pattern. Then, for multiple question pairs, manually identified teacher labels are assigned to the corresponding questions, and the data is processed into data that can be input to the model learning unit.

[0043] (First embodiment) Next, a specific device configuration will be described. Fig. 1 is a diagram showing an example of the configuration of a question sentence identification device 10 according to the first embodiment. The question sentence identification device 10 is a device that, based on the questions of a plurality of input medical questionnaires, performs binary classification to determine whether pairs of questions have the same intention or not, and outputs the result as a probability value.

[0044] The question sentence identification device 10 includes a central processing unit (CPU), memory, auxiliary storage device, and the like, all connected via a bus, and executes a program. By executing the program, the question sentence identification device 10 functions as a device including a question sentence / options input unit 101, a description position information input unit 102, a response data / participant information input unit 103, a linguistic similarity calculation unit 104, a position vector calculation unit 105, a statistics calculation unit 106, a question pair extraction unit 107, a similarity calculation unit 108, and a determination unit 109. Note that all or part of the functions of the question sentence identification device 10 may be implemented using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The program may also be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include portable media such as a flexible disk, a magneto-optical disk, a read-only memory (ROM), and a CD-ROM, and storage devices such as a hard disk built into a computer system. The program may also be transmitted and received via a telecommunications line.

[0045] The question and option input unit 101 receives the question sentences and option sentences in a plurality of medical questionnaires as input. The question and option input unit 101 outputs the question sentences and option sentences in the input plurality of medical questionnaires to the linguistic similarity calculation unit 104.

[0046] The written position information input unit 102 receives information on the written position of each question sentence and outputs the input information on the written position of each question sentence to the position vector calculation unit 105.

[0047] The answer data and participant information input unit 103 receives one or more pieces of answer data and participant information for each question sentence, and outputs the one or more pieces of answer data and participant information for each input question sentence to the statistics calculation unit 106.

[0048] The linguistic similarity calculation unit 104 calculates the linguistic similarity based on each question sentence and each option sentence in a plurality of medical questionnaires.

[0049] The position vector calculation unit 105 calculates a position vector based on information about the position of each question sentence.

[0050] The statistics calculation unit 106 calculates statistics based on one or more pieces of answer data for each question and the participant information.

[0051] The question pair extraction unit 107 extracts a vector that combines a pair of questions to be identified, the linguistic similarity of each question in the pair of questions to be identified (for example, the linguistic similarity of the question sentences in the pair of questions to be identified and the linguistic similarity of the options), statistics, and a position vector.

[0052] The similarity calculation unit 108 calculates the similarity between the vectors of a pair of questions to be identified based on a composite function. The similarity calculation unit 108 is one aspect of the evaluation value acquisition unit.

[0053] The determination unit 109 determines whether or not the pair of questions is identical based on the similarity calculated by the similarity calculation unit 108. For example, the determination unit 109 determines that a pair of questions whose calculated similarity is equal to or greater than a threshold is identical. For example, the determination unit 109 determines that a pair of questions whose calculated similarity is less than a threshold is not identical.

[0054] 2 is a diagram showing an example of the configuration of the record linkage device 20 according to the first embodiment. The record linkage device 20 is a device that receives the determination result from the question sentence identification device 10 as input and performs record linkage based on the input determination result.

[0055] The record linkage device 20 includes a CPU, memory, auxiliary storage device, and the like, all connected via a bus, and executes a program. By executing the program, the record linkage device 20 functions as a device including a determination unit 21 and a record linkage implementation unit 22. Note that all or part of the functions of the question sentence identification device 10 may be realized using hardware such as an ASIC, PLD, or FPGA. The program may also be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may also be transmitted and received via a telecommunications line.

[0056] The determination unit 21 performs a predetermined determination process on the identification results obtained by the question sentence identification device 10. Specifically, the determination unit 21 determines whether the input identification results indicate questions with the same intention. Furthermore, the determination unit 21 determines whether the linguistic similarity of the options is equal to or greater than a threshold.

[0057] The record linkage implementation unit 22 performs record linkage for question pairs included in the identification results when a predetermined condition is satisfied. The predetermined condition is a condition for performing record linkage, such as whether the determination unit 21 determines that the linguistic similarity between the options is equal to or greater than a threshold, or whether manual preprocessing is completed.

[0058] FIG. 3 is a diagram showing an example of a first information table relating to information on questions, options, and the position of each question in the first embodiment. The first information table shown in FIG. 3 is composed of multiple records in which values ​​for location, number of times conducted, sub-question item ID, question, answer format, option, total number of questions, question number, and position of question are associated with each other. The location indicates the location where the health checkup, medical examination, or cohort survey was conducted. The number of times conducted indicates the number of times the health checkup, medical examination, or cohort survey was conducted. The sub-question item ID indicates identification information for a question included in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. The question indicates the text of the question posed to the user.

[0059] The answer format indicates the format of the answer to the question. In Figure 3, the answer format is set to either enter numbers, multiple choice, or enter letters. The options indicate options for writing an answer. The total number of questions indicates the total number of questions in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. The question number indicates information for identifying the number of the associated question in the questionnaire. The question position indicates the position where the associated question is written in the questionnaire. Note that in Figure 3, the information on the total number of questions, question number, and question position indicates information on the position of each question.

[0060] FIG. 4 is a diagram showing an example of a second information table related to response data and participant information in the first embodiment. The second information table shown in FIG. 4 is composed of multiple records in which participant IDs, participant information, and response values ​​are associated with each other. The participant ID represents information identifying a user who underwent a health checkup, medical examination, or cohort survey. The participant information is information about the user who underwent a health checkup, medical examination, or cohort survey. In the example shown in FIG. 4, information on gender, age, number of times attended, and participation region is shown as the participant information, but the participant information is not limited to these. Gender represents the gender of the participant. Age represents the age of the participant. Number of times attended represents the number of health checkups, medical examinations, or cohort surveys attended by the participant. Participation region represents the region where the health checkups, medical examinations, or cohort surveys attended by the participant were conducted. The responses represent the responses of each participant to the questions identified by the questionnaire item ID. Note that blank fields in FIG. 4 represent blank, unknown, or not applicable.

[0061] 5 is a flowchart showing the flow of the question identification process performed by the question identification device 10 in the first embodiment. The question and option input unit 101 receives as input the question sentences and option data for each of a plurality of medical questionnaires (e.g., two medical questionnaires A and B) (step S101). The question and option input unit 101 outputs the question sentences and option data for each of the input medical questionnaires to the linguistic similarity calculation unit 104.

[0062] The answer data and participant information input unit 103 receives one or more pieces of answer data and participant information for each question (step S102). Here, the one or more pieces of answer data and participant information for each question input to the answer data and participant information input unit 103 are one or more pieces of answer data and participant information for each question in the multiple medical questionnaires input to the question and option input unit 101. The answer data and participant information input unit 103 outputs the one or more pieces of answer data and participant information for each input question to the statistics calculation unit 106.

[0063] The written position information input unit 102 receives information on the written position of each question (step S103). The written position information of each question input to the written position information input unit 102 is one or more pieces of answer data and participant information for each question in the multiple medical questionnaires input to the question and option input unit 101. The written position information input unit 102 outputs the input written position information of each question to the position vector calculation unit 105.

[0064] The linguistic similarity calculation unit 104 calculates linguistic similarity based on the sentences of each question and option in the multiple medical questionnaires using data on the sentences of each question and option in the multiple medical questionnaires output from the question sentence and option input unit 101 (step S104). For example, the linguistic similarity calculation unit 104 calculates the linguistic similarity of each question and option using a conventional method (e.g., BART).

[0065] FIG. 6 is a diagram showing an example of linguistic similarity in the first embodiment. The parent question item ID shown in FIG. 6 represents identification information of a question item included in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. Similarly, the child question item ID shown in FIG. 6 represents identification information of a question item included in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. The child question item ID is a comparison target, and this is an example of a comparison using linguistic similarity to determine how similar the question text and options of the question item of the child question item ID are to the question text and options of the question item of the parent question item ID.

[0066] The linguistic similarity between question sentences of a parent question item ID and a child question item ID shown in FIG. 6 represents the degree of linguistic similarity between the question sentence identified by the parent question item ID and the question sentence identified by the child question item ID. The closer the linguistic similarity between question sentences of a parent question item ID and a child question item ID is to "1," the closer the linguistic similarity between the question sentence identified by the parent question item ID and the question sentence identified by the child question item ID is to. The linguistic similarity between options of a parent question item ID and a child question item ID shown in FIG. 6 represents the degree of linguistic similarity between the options identified by the parent question item ID and the option identified by the child question item ID is to. The closer the linguistic similarity between options of a parent question item ID and a child question item ID is to "1," the closer the linguistic similarity between options of a parent question item ID and a child question item ID is to "1," the closer the linguistic similarity between options of a parent question item ID and a child question item ID is to

[0067] Continuing the explanation, returning to Fig. 5, the linguistic similarity calculation unit 104 outputs the calculated linguistic similarity value (for example, the information shown in Fig. 6) to the question pair extraction unit 107.

[0068] The statistics calculation unit 106 calculates statistics using one or more pieces of answer data and participant information for each question output from the answer data / participant information input unit 103 (step S105). Figure 7 is a diagram showing an example of statistics for each question / participant data in the first embodiment. The child question IDs shown in Figure 7 represent identification information for questions included in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. The missing answer rate for parent questions shown in Figure 7 represents the proportion of missing answers for the parent question. Missing answers for parent questions refer to a state in which an answer different from the original answer was obtained, such as an answer that was unreadable, left unanswered, or not included in the options. The gender ratio for answers for parent questions shown in Figure 7 represents the proportion of men and women who answered the parent question.

[0069] The average age of the answers to the parent question item shown in Figure 7 represents the average age of the participants who answered the parent question item. The regional ratio of the parent question item shown in Figure 7 represents the proportion of regions in which the parent question item is used as a question sentence. The statistics calculation unit 106 outputs the calculated statistical values ​​(for example, the information shown in Figure 7) to the question pair extraction unit 107. Returning to Figure 5, the explanation will be continued. The position vector calculation unit 105 calculates a position vector based on the information on the writing position of each question sentence output from the writing position information input unit 102 (step S106).

[0070] Fig. 8 is a diagram showing an example of a position vector in the first embodiment. The sub-question item IDs shown in Fig. 8 represent identification information of questions included in the questionnaire used when the health checkup, medical examination, or cohort survey was conducted. As shown in Fig. 8, the position vector calculation unit 105 calculates the position vector of the question sentence of each question item identified by the sub-question item ID.

[0071] Continuing the explanation, returning to Fig. 5, the position vector calculation unit 105 outputs the calculated position vector value (for example, the information shown in Fig. 8) to the question pair extraction unit 107.

[0072] The question pair extraction unit 107 receives as input the linguistic similarity values ​​between the question sentences and options of each question output from the linguistic similarity calculation unit 104, the statistical value output from the statistical value calculation unit 106, and the position vector value output from the position vector calculation unit 105. The question pair extraction unit 107 extracts a vector combining the various pieces of input information (step S107). Specifically, the question pair extraction unit 107 calculates a vector combining the linguistic similarity, statistical value, and position vector between the question pair to be identified using the composite function expressed in equation (5) above. The question pair extraction unit 107 calculates a vector combining the linguistic similarity, statistical value, and position vector between the question pair to be identified for each question to be identified.

[0073] The question pair extraction unit 107 outputs the vector calculated for each question to be identified to the similarity calculation unit 108. For example, the question pair extraction unit 107 outputs to the similarity calculation unit 108 as shown in FIG. 9 . The similarity calculation unit 108 calculates the similarity of the vector calculated for each question to be identified output from the question pair extraction unit 107 (step S108). Specifically, the similarity calculation unit 108 calculates the cosine similarity of the above formula (1) for the vector calculated for each question to be identified. The determination unit 109 determines whether the similarity calculated by the similarity calculation unit 108 is equal to or greater than a threshold (step S109).

[0074] If the determination unit 109 determines that the similarity calculated by the similarity calculation unit 108 is equal to or greater than the threshold value (step S109—YES), the determination unit 109 stores an identification result indicating that the question pair are questions with the same intention (e.g., identification result “1”) (step S110). At this time, the determination unit 109 stores the identification result in association with the question pair (e.g., a pair of question item IDs of a question sentence) and the linguistic similarity values ​​of the options of the question pair.

[0075] On the other hand, if the determination unit 109 determines that the similarity calculated by the similarity calculation unit 108 is less than the threshold value (step S109-NO), the determination unit 109 stores an identification result indicating that the question pair are questions with different intentions (e.g., an identification result of "0") (step S111). At this time, the determination unit 109 stores the identification result in association with the question pair (e.g., a pair of question item IDs of a question sentence) and the linguistic similarity values ​​of the options of the question pair.

[0076] The question sentence identification device 10 stores the identification results of a plurality of question pairs by repeatedly executing the process shown in Fig. 5. The question sentence identification device 10 may transmit the stored identification results to the record linkage device 20 in response to a request from the record linkage device 20, or may transmit the stored identification results to the record linkage device 20 every time it stores an identification result.

[0077] 10 is a flowchart showing the flow of the record linkage process performed by the record linkage device 20 in the first embodiment. The record linkage device 20 receives the identification results stored in the question sentence identification device 10 as input (step S201). The judgment unit 21 determines whether the input identification results indicate that the questions have the same intention (step S202). If the identification results indicate "1", the judgment unit 21 determines that the input identification results indicate that the questions have the same intention. On the other hand, if the identification results indicate "0", the judgment unit 21 determines that the input identification results do not indicate that the questions have the same intention.

[0078] If the judgment unit 21 determines that the input identification results do not indicate that the questions have the same intention (step S202-NO), the record linkage implementation unit 22 does not perform record linkage on the question pair associated with the input identification results (step S203). After that, the record linkage device 20 ends the process shown in Fig. 10. Note that if there are still identification results stored in the question sentence identification device 10, the record linkage device 20 may execute the process of step S201.

[0079] On the other hand, if the judgment unit 21 determines that the input identification results indicate questions with the same intention (step S202—YES), the judgment unit 21 determines whether the linguistic similarity values ​​of the options associated with the input identification results are equal to or greater than a threshold (step S204). If the judgment unit 21 determines that the linguistic similarity values ​​of the options are equal to or greater than the threshold (step S204—YES), the judgment unit 21 outputs the identification results to the record linkage implementation unit 22. The record linkage implementation unit 22 performs record linkage on the question pair associated with the identification results output from the judgment unit 21 (step S205).

[0080] On the other hand, if the judgment unit 21 determines that the linguistic similarity value of the options is less than the threshold value (step S204—NO), the record linkage implementation unit 22 displays the contents of the options of the question pair associated with the identification result on the screen. This allows a person to check the contents of the options and perform manual preprocessing (step S206). After the manual preprocessing is completed, the record linkage implementation unit 22 performs record linkage on the question pair associated with the identification result (step S207).

[0081] FIG. 11 is a diagram illustrating processing performed when the linguistic similarity value of options is equal to or greater than a threshold value in the first embodiment. (A) in FIG. 11 shows an identification result obtained from the question sentence identification device 10. Here, a question pair with a parent question item ID "E1_01" and a child question item ID "F1_02" will be described as an example. Assume that the determination unit 21 determines that the linguistic similarity value (0.551 in FIG. 11 ) of options in the question pair with the parent question item ID "E1_01" and the child question item ID "F1_02" is equal to or greater than a threshold value. In this case, the record linkage implementation unit 22 performs record linkage between the question item identified by the parent question item ID "E1_01" shown in (B) in FIG. 11 and the question item identified by the child question item ID "F1_02" shown in (C) in FIG. 11 . As a result, the result shown in (D) in FIG. 11 is obtained.

[0082] FIG. 12 is a diagram illustrating processing performed when the linguistic similarity value of the options in the first embodiment is less than a threshold value. (A) in FIG. 12 shows an identification result obtained from the question sentence identification device 10. Here, a question pair with a parent question item ID "E1_01" and a child question item ID "F1_02" will be described as an example. Assume that the determination unit 21 determines that the linguistic similarity value ("0.551" in FIG. 12) of the options in the question pair with the parent question item ID "E1_01" and the child question item ID "F1_02" is less than the threshold value. In this case, the record linkage implementation unit 22 displays the contents of the options in the question pair on the screen, as shown in (B) in FIG. 12. Assume that the options for the question item with the child question item ID "F1_02" are the character strings shown in (B) in FIG. 11 (e.g., "Applicable," "Somewhat Applicable," "Neither," etc.).

[0083] In such cases, the content of the options is manually checked and manually converted into numerical values ​​as shown in Fig. 12(C).Then, the record linkage implementation unit 22 performs record linkage between the question item identified by the parent question item ID "E1_01" shown in Fig. 12(D) and the question item identified by the child question item ID "F1_02" shown in Fig. 12(C).As a result, the result shown in Fig. 12(E) is obtained.

[0084] The question identification device 10 configured as described above inputs data on each question and each option for each question in multiple questionnaires, one or more pieces of answer data and participant information for each question, and information on the location of each question in the multiple questionnaires. The device calculates similarities based on the input data on each question and each option for each question, one or more pieces of answer data and participant information, and information on the location of each question. The device determines pairs of questions for which the calculated similarity is equal to or greater than a threshold as being identical, and performs record linkage on pairs of questions determined to be identical. This enables accurate record identification using the similarity of the questions and the data distribution obtained for each question, eliminating the need for visual confirmation of question overlap or data processing work before performing statistical analysis. Furthermore, the device performs record linkage on pairs of questions determined to be identical. This enables efficient question identification and record linkage.

[0085] By using this type of question identification device 10, it is possible to propose similar question pairs that minimize the influence of expressions specific to medical examinations and questionnaire surveys. In the future, it will be possible to efficiently combine records for data collected at medical institutions, such as cohort studies and health checkups, which will contribute to improving the efficiency of medical data analysis.

[0086] Second Embodiment In a second embodiment, a configuration for identifying a question sentence using a trained model will be described.

[0087] FIG. 13 is a diagram showing an example of the configuration of a question sentence identification device 10a according to the second embodiment. The question sentence identification device 10a is a device that outputs the results of binary classification of whether a pair of questions has the same intention or not, based on the questions of a plurality of input medical questionnaires, as a probability value. The question sentence identification device 10a performs a learning process and an inference process. The learning process is a process for learning a trained model used in the inference process. The trained model used in the inference process includes an algorithm, weight parameters, and a threshold. The threshold included in the trained model represents a threshold for the classification probability.

[0088] The question sentence identification device 10a includes a CPU, memory, auxiliary storage device, and the like, all connected via a bus, and executes a program. By executing the program, the question sentence identification device 10a functions as a device including a question sentence / options input unit 101, a description position information input unit 102, a response data / participant information input unit 103, a linguistic similarity calculation unit 104, a position vector calculation unit 105, a statistics calculation unit 106, a processing unit 110, a learning unit 111, a memory 112, an inference unit 113, and a determination unit 114. Note that all or part of the functions of the question sentence identification device 10a may be implemented using hardware such as an ASIC, a PLD, or an FPGA. The program may also be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include portable media such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and storage devices such as a hard disk built into a computer system. The program may also be transmitted and received via a telecommunications line.

[0089] The question sentence identification device 10a differs in configuration from the question sentence identification device 10 in that it includes a processing unit 110, a memory 112, an inference unit 113, and a judgment unit 114 instead of the statistics calculation unit 106, the question pair extraction unit 107, the similarity calculation unit 108, and the judgment unit 109. The rest of the configuration of the question sentence identification device 10a is the same as that of the question sentence identification device 10. The following description will focus on the differences from the question sentence identification device 10.

[0090] The processing unit 110 extracts a vector that combines a pair of questions to be identified, the linguistic similarity of each question in the pair of questions to be identified (for example, the linguistic similarity of the question sentences in the pair of questions to be identified and the linguistic similarity of the options), statistics, and a position vector. Specifically, the processing unit 110 calculates a vector for each question to be identified using the composite function expressed in the above formula (6). The processing unit 110 processes the vectors calculated for each question to be identified into a matrix that is pairwise combined.

[0091] The learning unit 111 receives as input learning data (for example, the data shown in FIG. 15 ) that combines the matrix processed by the processing unit 110 and teacher labels manually identified for multiple question sentence pairs. The learning unit 111 generates a trained model by performing learning using various information obtained based on the input learning data.

[0092] The memory 112 stores the trained model trained by the training unit 111 .

[0093] The inference unit 113 acquires a probability value indicating the identification result of a question sentence pair by using the matrix processed by the processing unit 110 and the trained model stored in the memory 112. The inference unit 113 outputs a probability value indicating the identification result of a question sentence pair by inputting the matrix processed by the processing unit 110 into the trained model. The inference unit 113 is one aspect of the evaluation value acquisition unit.

[0094] The determination unit 114 determines whether or not the question pair is identified based on the probability value output by the inference unit 113. For example, the determination unit 114 determines that a question pair whose output probability value is equal to or greater than a threshold is identified. For example, the determination unit 114 determines that a question pair whose output probability value is less than a threshold is not identified.

[0095] FIG. 14 is a diagram showing an example of a teacher label in the second embodiment. The teacher label represents correct answer data used during the learning process. As shown in FIG. 14, a teacher label, which is correct answer data, is associated with each combination of a parent question item ID and a child question item ID. A teacher label of "1" indicates that the question pair, consisting of the question item identified by the parent question item ID and the question item identified by the child question item ID, has the same intention. A teacher label of "0" indicates that the question pair, consisting of the question item identified by the parent question item ID and the question item identified by the child question item ID, does not have the same intention.

[0096] Fig. 15 is a diagram showing an example of data input during the learning process of the question sentence identification device in the second embodiment. The data shown in Fig. 15 is so-called learning data. As shown in Fig. 15, for each combination of a parent question item ID and a child question item ID, a linguistic similarity, a position vector of the question sentence of the parent question item ID, statistics of the parent question item ID, a position vector of the question sentence of the child question item ID, statistics of the child question item ID, and a teacher label are associated with each other.

[0097] The linguistic similarity represents the linguistic similarity between the question sentences of the question item identified by the parent question item ID and the question item identified by the child question item ID, and the linguistic similarity between the options. The position vector of the question sentence of the parent question item ID represents the position vector of the question sentence of the question item identified by the parent question item ID. The statistics of the parent question item ID represent the statistics of the question item identified by the parent question item ID. The position vector of the question sentence of the child question item ID represents the position vector of the question sentence of the question item identified by the child question item ID. The statistics of the child question item ID represent the statistics of the question item identified by the child question item ID. The teacher label represents the correct answer data used during the learning process.

[0098] Fig. 16 is a diagram showing an example of data input during inference processing of the question sentence identification device in the second embodiment. The data shown in Fig. 16 is the so-called data to be processed. As shown in Fig. 16, for each combination of parent question item ID and child question item ID, there is associated linguistic similarity, a position vector of the question sentence for the parent question item ID, statistics for the parent question item ID, and statistics for the position vector of the question sentence for the child question item ID and the child question item ID. Each data item shown in Fig. 16 is the same as the item shown in Fig. 15 except that there is no teacher label.

[0099] 17 is a flowchart showing the flow of the learning process performed by the question sentence identification device in the second embodiment. The question sentence and option input unit 101 receives as input the sentences and option data of each question sentence in a plurality of medical questionnaires (step S301). The question sentence and option input unit 101 outputs the sentences and option data of each question sentence in the plurality of input medical questionnaires to the linguistic similarity calculation unit 104. The answer data and participant information input unit 103 receives as input one or more pieces of answer data and participant information for each question sentence (step S302). The answer data and participant information input unit 103 outputs one or more pieces of answer data and participant information for each input question sentence to the statistics calculation unit 106.

[0100] The written position information input unit 102 receives information on the written position of each question sentence (step S303). The written position information input unit 102 outputs the input information on the written position of each question sentence to the position vector calculation unit 105. The linguistic similarity calculation unit 104 calculates linguistic similarity from the sentences of each question sentence and option in the multiple medical questionnaires using data on the sentences of each question sentence and option in the multiple medical questionnaires output from the question sentence / option input unit 101 (step S304). For example, the linguistic similarity calculation unit 104 calculates the linguistic similarity between the question sentence and option using a conventional method (e.g., BART). The linguistic similarity calculation unit 104 outputs the calculated linguistic similarity value (e.g., information shown in FIG. 6 ) to the processing unit 110.

[0101] The statistical quantity calculation unit 106 calculates statistical quantities using one or more pieces of answer data and participant information for each question sentence output from the answer data and participant information input unit 103 (step S305). The statistical quantity calculation unit 106 outputs the calculated statistical quantity values ​​(e.g., the information shown in FIG. 7 ) to the processing unit 110. The position vector calculation unit 105 calculates position vectors based on the information on the writing position of each question sentence output from the writing position information input unit 102 (step S306). The position vector calculation unit 105 outputs the calculated position vector values ​​(e.g., the information shown in FIG. 8 ) to the processing unit 110.

[0102] The processing unit 110 receives as input the linguistic similarity values ​​between the pairs of questions to be identified and the question sentences and options of each question output from the linguistic similarity calculation unit 104, the statistical values ​​output from the statistics calculation unit 106, and the position vector values ​​output from the position vector calculation unit 105. Using the various input information, the processing unit 110 calculates a vector for each question to be identified based on the above formula (6). The processing unit 110 processes the vectors calculated for each question to be identified into a matrix that combines them pairwise (step S307). The processing unit 110 outputs the matrix obtained by processing to the learning unit 111.

[0103] The learning unit 111 learns a classification model using the matrix output from the processing unit 110 and the teacher labels. The learning unit 111 determines whether a learning termination condition is satisfied (step S309). The learning termination condition is a condition for terminating learning by the learning unit 111, and may be, for example, that a predetermined number of learning processes have been performed, or that the error has become less than a threshold value. If the learning unit 111 determines that the learning termination condition is not satisfied (step S309-NO), it executes the processing of step S308 until the learning termination condition is satisfied.

[0104] If the learning unit 111 determines that the learning termination condition is met (step S309-YES), it stores the algorithm used for learning, and the weight parameters and thresholds obtained by learning in the memory 112 as a learned model (step S310).

[0105] FIG. 18 is a flowchart showing the flow of inference processing performed by the question sentence identification device 10a in the second embodiment. In FIG. 18, the same processes as those in FIG. 17 are denoted by the same reference numerals as those in FIG. 17, and their description will be omitted. After the processing of step S307, the processing unit 110 outputs the matrix obtained by processing to the inference unit 113. The inference unit 113 performs inference processing based on the trained model stored in the memory 112 and the matrix output from the processing unit 110 (step S401). Specifically, the inference unit 113 obtains a probability value by inputting the matrix output from the processing unit 110 into the trained model. The inference unit 113 outputs the obtained probability value to the determination unit 114.

[0106] The determination unit 114 determines whether the probability value output from the inference unit 113 is equal to or greater than a threshold value (step S402). If the determination unit 114 determines that the probability value output from the inference unit 113 is equal to or greater than the threshold value (step S402—YES), the determination unit 114 stores an identification result indicating that the question pair are questions with the same intention (e.g., identification result “1”) (step S403). At this time, the determination unit 114 stores the identification result in association with the question pair (e.g., a pair of question item IDs of a question sentence) and the linguistic similarity values ​​of the options in the question pair.

[0107] On the other hand, if the determination unit 114 determines that the probability value output from the inference unit 113 is less than the threshold value (step S402—NO), the determination unit 114 stores the identification result indicating that the question pair are questions with different intentions (e.g., the identification result “0”) (step S404). At this time, the determination unit 114 stores the identification result in association with the question pair (e.g., a pair of question item IDs of a question sentence) and the linguistic similarity values ​​of the options of the question pair.

[0108] In the second embodiment, the configuration and processing of the record linkage device 20 that performs record linkage are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0109] The question identification device 10a configured as described above can achieve the same effects as those of the first embodiment. The question identification device 10a, for example, uses a trained model to obtain a probability value indicating whether a question pair is identical, and determines whether the question pair is identical based on the obtained probability value. The record linkage device 20 then performs record linkage using the identification results stored in the question identification device 10a.

[0110] (Variation of the Second Embodiment) In the above embodiment, the question sentence identification device 10a is configured to perform both the learning process and the inference process. The learning process and the inference process may be performed by separate devices. For example, the learning process may be performed by a first device, and the inference process may be performed by a second device. In this configuration, the first device outputs a trained model obtained by training to the second device. The second device performs the inference process using the trained model obtained from the first device. In the above configuration, the first device includes a question sentence / options input unit 101, a description position information input unit 102, a response data / participant information input unit 103, a linguistic similarity calculation unit 104, a position vector calculation unit 105, a statistics calculation unit 106, a processing unit 110, a learning unit 111, and a communication unit (not shown). The communication unit included in the first device transmits the trained model obtained by the learning unit 111 to the second device. The second device includes a question sentence / options input unit 101, a description position information input unit 102, a response data / participant information input unit 103, a linguistic similarity calculation unit 104, a position vector calculation unit 105, a statistics calculation unit 106, a processing unit 110, a memory 112, an inference unit 113, a determination unit 114, and a communication unit (not shown). The communication unit included in the second device receives the trained model transmitted from the first device. The communication unit included in the second device stores the received trained model in the memory 112.

[0111] (Modification 1 common to the first and second embodiments) In the above embodiments, a medical questionnaire has been described as an example, but the subject of question identification in the question identification device 10, 10a is not limited to a medical questionnaire. For example, any questionnaire containing one or more questions for a user may be used.

[0112] (Variation 2 common to the first and second embodiments) In each of the above embodiments, the configuration has been described in which question sentence identification and record linkage are performed by separate devices. Question sentence identification and record linkage may also be performed by the same device. In this configuration, in the first embodiment, the question sentence identification device 10 is configured as a question sentence analysis device having the configuration of the record linkage device 20, and in the second embodiment, the question sentence identification device 10a is configured as a question sentence analysis device having the configuration of the record linkage device 20.

[0113] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0114] The present invention can be applied to a technique for identifying a question sentence.

[0115] DESCRIPTION OF SYMBOLS 10, 10a...Question sentence identification device, 20...Record linkage device, 21...Judgment unit, 22...Record linkage implementation unit, 101...Question sentence / options input unit, 102...Description position information input unit, 103...Response data / participant information input unit, 104...Linguistic similarity calculation unit, 105...Position vector calculation unit, 106...Statistical quantity calculation unit, 107...Question pair extraction unit, 108...Similarity calculation unit, 109...Judgment unit, 110...Processing unit, 111...Learning unit, 112...Memory, 113...Inference unit, 114...Judgment unit

Claims

1. A method of analyzing questions, comprising: inputting data on each question and option for each question in a plurality of questionnaires, one or more pieces of answer data for each question, participant information, and information on the position of each question in the plurality of questionnaires; obtaining an evaluation value for determining whether a pair of questions combining each question is identical or not based on the input data on each question and option for each question, the one or more pieces of answer data, participant information, and information on the position of each question; determining as an identical pair of questions those for which the obtained evaluation value is equal to or greater than a threshold; and performing record linkage on the pairs of questions determined to be identical.

2. The question analysis method according to claim 1, wherein the evaluation value is a similarity between vectors obtained by combining the linguistic similarity between the pair of questions to be identified and each question, statistics obtained based on one or more pieces of answer data and participant information for each question, and a position vector obtained based on information on the position of each question.

3. The question sentence analysis method according to claim 1, wherein the evaluation value is a probability value obtained by inputting into a trained model a matrix that combines the linguistic similarity between the question pair to be identified and each question, one or more pieces of answer data and participant information for each question sentence, and a position vector obtained based on information on the position of each question sentence.

4. A question analysis method according to any one of claims 1 to 3, further comprising the steps of: if the linguistic similarity value of the options in the question pair that is determined to be identical is equal to or greater than a threshold, executing record linkage for the question pair; and if the linguistic similarity value of the options is less than the threshold, performing preprocessing to modify the contents of the options before executing record linkage.

5. The question analysis method according to any one of claims 1 to 3, wherein the questionnaire is a medical questionnaire used in any one of a health checkup, a medical examination, or a cohort study.

6. A question analysis device comprising: a question sentence / option input unit for inputting data on each question sentence and option for each question sentence in a plurality of questionnaires; an answer data / participant information input unit for inputting one or more answer data and participant information for each question sentence data; a position information input unit for inputting information on the position of each question sentence in the plurality of questionnaires; an evaluation value acquisition unit for acquiring an evaluation value for determining whether a pair of questions combining each question sentence is identical or not based on the input question sentence and option data for each question sentence, the one or more answer data and participant information, and the information on the position of each question sentence; a determination unit for determining, as identical, a question sentence pair for which the evaluation value acquired by the evaluation value acquisition unit is equal to or greater than a threshold value; and a record linkage implementation unit for executing record linkage on the question sentence pairs determined to be identical by the determination unit.

Images