Method for Evaluating the Credibility of Epidemiological Investigation Q&A Based on Spatiotemporal Consistency and Context Consistency

Abstract

The present invention discloses a method for evaluating the credibility of epidemic investigation Q&A based on spatio-temporal consistency and context consistency. The characteristics of this method include: evaluating the credibility of epidemic investigation Q&A based on spatio-temporal consistency and evaluating the credibility of epidemic investigation Q&A based on context consistency. The evaluation of the credibility of epidemic investigation Q&A based on spatio-temporal consistency specifically includes: time and location matching and calculation of the credibility of spatio-temporal consistency; the evaluation of the credibility of epidemic investigation Q&A based on context consistency, on the basis of a large language model, uses instruction fine-tuning technology to judge the context consistency of the dialogue content. Compared with the prior art, the present invention evaluates the credibility of the collected epidemic investigation dialogues from the perspectives of the spatio-temporal consistency between the dialogue and mobile phone signaling and the context consistency within the dialogue, so as to timely intervene in the epidemic investigation Q&A in the case of low credibility, enabling the epidemic investigation object to provide as much true and reliable information as possible, and providing high-quality and credible data support for later tracing and source tracing.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for evaluating the credibility of epidemiological investigation questions and answers based on the spatio-temporal consistency of conversations and mobile phone signaling and the internal context consistency of conversations. Background Art

[0002] The purpose of epidemiological investigation questions and answers is to automatically investigate the movement trajectories, people encountered, and events that occurred of key populations during a certain period in the past through question-and-answer technology. A key issue in the epidemiological investigation question-and-answer system is how to evaluate whether the collected epidemiological investigation information is credible, and timely intervene in the epidemiological investigation questions and answers in the case of low credibility, so that the respondents can provide true and reliable information as much as possible, providing high-quality and credible data support for later tracing and source tracking.

[0003] Currently, there is no evaluation method for the credibility of epidemiological investigation questions and answers. According to the experience of epidemiological investigation experts, there are mainly two types of untrustworthy problems that may occur in epidemiological investigation questions and answers: one is "inconsistency between the conversation and the spatio-temporal information of mobile phone signaling", and the other is "inconsistency within the internal context of the conversation". Therefore, how to design a method for evaluating the credibility of epidemiological investigation questions and answers for the possible untrustworthy problems in epidemiological investigation questions and answers is an urgent research topic. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for evaluating the credibility of epidemiological investigation questions and answers based on spatio-temporal consistency and context consistency in view of the deficiencies of the prior art. In the evaluation of the credibility of the spatio-temporal consistency between the conversation and the mobile phone signaling, the longitude and latitude of the address given by the respondent and the address recorded by the mobile phone signaling are obtained by calling the Gaode Map API, and the distance between the two points is accurately calculated and judged, solving the problem that geographical matching based on word segmentation technology is prone to misjudgment. In the evaluation of the credibility of the internal context consistency of the conversation, the instruction fine-tuning technology is adopted to fully stimulate the capabilities of the large language model, thereby improving the accuracy of the evaluation based on the internal context consistency of the conversation. The present invention can evaluate the credibility of the collected epidemiological investigation conversations from the perspectives of the spatio-temporal consistency between the conversation and the mobile phone signaling and the internal context consistency of the conversation, so as to timely intervene in the epidemiological investigation questions and answers in the case of low credibility, enabling the respondents to provide true and reliable information as much as possible, providing high-quality and credible data support for later tracing and source tracking. The method is simple, highly practical, and has good application prospects.

[0005] The specific technical solution for achieving the purpose of the present invention is: a method for evaluating the credibility of epidemiological investigation questions and answers based on spatio-temporal consistency and context consistency, characterized in that the method includes: evaluating the credibility of epidemiological investigation questions and answers based on spatio-temporal consistency and evaluating the credibility of epidemiological investigation questions and answers based on context consistency. The evaluation of the credibility of epidemiological investigation questions and answers based on spatio-temporal consistency specifically includes:

[0006] 1) Definition of Spatiotemporal Consistency between Conversation and Mobile Signaling

[0007] Mobile signaling records the activity locations and residence durations of the surveyed objects at different times of a day. If, in the epidemiological investigation conversation, the location stated by the surveyed person during a certain time period is inconsistent with or far away from the location recorded in the mobile signaling, then the surveyed person has the problem of "spatiotemporal inconsistency between conversation and mobile signaling".

[0008] 2) Evaluation of the Credibility of Epidemiological Investigation Q&A Based on Spatiotemporal Consistency

[0009] To evaluate the spatiotemporal consistency between the answers of the epidemiological investigation objects and the mobile signaling, the present invention extracts the (time, location) information of the activities of the epidemiological investigation objects from the conversation, and then compares and matches this information with the (time, location) provided in the mobile signaling to judge the credibility of the conversation content. The specific operations are as follows:

[0010] 1-1: For time matching, the present invention sets the credible time error range as . If, in the conversation of the epidemiological investigation object, the object has been to a certain place at a certain time point , and the mobile signaling record shows that the epidemiological investigation object was indeed at , t +ε] at this location during this time, then this information is considered credible.

[0011] 1-2: For location matching, the present invention sets the credible distance error range as . The calculation method of the distance is as follows: First, call the Gaode Map API to obtain the longitude and latitude of the location described by the case and the location provided in the mobile signaling respectively, and then calculate the distance between the two points through the longitude and latitude. If the distance between the two locations is less than , it is considered that the two locations are the same.

[0012] Generally speaking, through the above time and location matching methods, if the (time, location) information mentioned in the Q&A pair is consistent with the (time, location) data in the mobile signaling of this case, then this Q&A is considered credible. Specifically, the accuracy rate of the credibility evaluation based on the spatiotemporal consistency between the conversation and the mobile signaling is calculated by the following formula (a):

[0013] (a).

[0014] Among them, represents the number of Q&A pairs with correct credibility prediction according to the above method, represents the total number of Q&A pairs.

[0015] The credibility evaluation of the epidemic investigation questions and answers based on context consistency specifically includes:

[0016] 2-1: Definition of intra-dialogue context consistency:

[0017] If the respondent to the epidemic investigation gives inconsistent answers to the same or similar questions raised by the investigator, it is considered that there is a problem of "intra-dialogue context inconsistency", which will affect the credibility of the respondent's answers.

[0018] 2-2: Credibility evaluation of epidemic investigation questions and answers based on context consistency

[0019] When dealing with the problem of whether the intra-dialogue context is consistent, based on a large language model, the present invention adopts the instruction fine-tuning technology and further fine-tunes it in combination with some labeled data to determine whether there is a context inconsistency problem in the dialogue. Then, the accuracy of the prediction result is used as the evaluation criterion for this credibility. Specifically, for the input dialogue, if the credibility label predicted by the model is the same as the actual label, then this prediction is considered correct. Specifically, the accuracy of the credibility evaluation based on the intra-dialogue context consistency is calculated by the following formula (b):

[0020] (b).

[0021] Among them, represents the number of question-and-answer pairs with correct credibility prediction according to the context semantics, represents the total number of question-and-answer pairs.

[0022] The present invention has the following beneficial technical effects and remarkable technical progress compared with the prior art:

[0023] 1) In the credibility evaluation of the spatio-temporal consistency between the dialogue and the mobile phone signaling, by calling the Gaode Map API to obtain the longitude and latitude of the address given by the respondent to the epidemic investigation and the address recorded by the mobile phone signaling, accurately calculate the distance between the two points and make a judgment, which solves the problem that the geographical matching based on the word segmentation technology is prone to misjudgment;

[0024] 2) In the credibility evaluation of the intra-dialogue context consistency, based on a large language model, the present invention adopts the instruction fine-tuning technology to fully stimulate the capabilities of the large language model, thereby improving the accuracy of the evaluation based on the intra-dialogue context consistency. Description of the Drawings

[0025] Figure 1 is the flow chart of the credibility evaluation of the epidemic investigation questions and answers based on spatio-temporal consistency;

[0026] Figure 2 is the flow chart of the credibility evaluation of the epidemic investigation questions and answers based on context consistency. Detailed Embodiments

[0027] The mobile signaling data record of the person under epidemiological investigation is as follows: At 07:59:33 on August 7, 2022, the person appeared near the 5th floor of a certain shopping mall at No. xxx, xxx Road, Hongkou District, Shanghai, and the stay duration was 22.75 minutes. And some fragments of the epidemiological investigation Q&A are as follows: Epidemiological investigator: Excuse me, where were you around 8 o'clock on August 7, 2022?;

[0028] Person under epidemiological investigation: I was at Huijing International Plaza at that time; Epidemiological investigator: Which Huijing International Plaza? What is the specific address?; Person under epidemiological investigation: A certain shopping mall at No. xxx, xxx Road, Xuhui District, Shanghai; Some fragments of the epidemiological investigation Q&A 2: Epidemiological investigator: Excuse me, where did you go after dinner at 21:00 on August 7, 2022?; Person under epidemiological investigation: I went home directly after dinner.... Epidemiological investigator: Did you go home directly after dinner in the evening?; Person under epidemiological investigation: No, I went to the supermarket on the way.... Take this as an example to conduct a credibility evaluation of the epidemiological investigation Q&A based on spatio-temporal consistency and context consistency.

[0029] Example 1

[0030] Refer to Figure 1 , the credibility evaluation of the epidemiological investigation Q&A based on spatio-temporal consistency is as follows:

[0031] Set the credible time error range to 30 minutes, and the credible distance error range is set to 500 meters. According to the mobile signaling record of the person under epidemiological investigation, the person under epidemiological investigation was at 07:59:33

[0032] on August 7, 2022, appeared near the 5th floor of a certain shopping mall at No. xx, xx Road, Hongkou District, Shanghai, and the stay duration was 22.75 minutes.

[0033] According to the conversation content in some fragments of the epidemiological investigation Q&A 1, first analyze the time consistency. When the credible time error range is set to 30 minutes, the credible time range is [7:29:33, 8:29:33], and the time described by the person under epidemiological investigation around 8 o'clock is within this interval. Then analyze the spatial consistency. First, obtain the longitude and latitude of No. xxx, xxx Road, Xuhui District, Shanghai by calling the Gaode Map API. The returned result is (121.423496, 31.195998), and the longitude and latitude of No. xxx, xxx Road, Hongkou District, Shanghai. The returned result is (121.477568, 31.271559), where the former is the address given by the person under epidemiological investigation, and the latter is the address in the mobile signaling record. Then, the distance between the two points can be calculated through the longitude and latitude as 9.29 kilometers, which is much greater than 500 meters. There is a contradiction between the answer of the person under epidemiological investigation and the address in the mobile signaling record. Therefore, the answer content is not credible.

[0034] Generally speaking, through the above time and location matching method, if the (time, location) information mentioned in the Q&A pair matches the (time, location) data in the mobile signaling of the epidemiological investigation object, then this Q&A is considered credible. After judging the credibility of the spatio-temporal consistency between all epidemiological investigation conversations and mobile signaling, the accuracy rate of the credibility evaluation based on the spatio-temporal consistency between conversations and mobile signaling can be calculated according to the following formula (a):

[0035] (a).

[0036] Among them, represents the number of Q&A pairs with correct credibility prediction according to the above method, represents the total number of Q&A pairs.

[0037] Refer to Figure 2 for the credibility evaluation of epidemiological investigation Q&A based on context consistency

[0038] If the epidemiological investigation object answers inconsistently before and after the same or similar questions raised by the epidemiological investigator, it is considered that there is a problem of "inconsistent internal context of the conversation", which will affect the credibility of the answers of the epidemiological investigation object.

[0039] In some fragments of epidemiological investigation Q&A 2, when the epidemiological investigator asked "Where did you go after dinner at 21:00 on August 7, 2022?" in the first round, the epidemiological investigation object answered "I went home directly after dinner", but in the subsequent conversation, when the epidemiological investigator asked again "Did you go home directly after dinner in the evening?", the epidemiological investigation object answered "No, I went to the supermarket halfway", which formed an inconsistent internal context of the conversation.

[0040] When dealing with the problem of whether the internal context of the conversation is consistent, the epidemiological investigation conversation is input into the fine-tuned large language model. In this case, the fine-tuned large language model will determine that there is an inconsistent internal context problem in this conversation.

[0041] If the credibility label predicted by the model is the same as the actual label, then this prediction is considered correct. After judging the context consistency of all epidemiological investigation conversations, the accuracy rate of the credibility evaluation based on the internal context consistency of the conversation is calculated according to the following formula (b):

[0042] (b).

[0043] Among them, represents the number of Q&A pairs with correct credibility prediction according to the context semantics, represents the total number of Q&A pairs.

[0044] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for evaluating the credibility of epidemic investigation questions and answers based on spatio-temporal consistency and context consistency, characterized in that, The method includes: the credibility evaluation of epidemiological investigation Q&A based on spatio-temporal consistency and the credibility evaluation of epidemiological investigation Q&A based on context consistency. The credibility evaluation of epidemiological investigation Q&A based on spatio-temporal consistency extracts the time and location information of the activities of the epidemiological investigation object from the conversation and compares and matches them with the time and location provided in the mobile phone signaling to judge the credibility of the conversation content. The specific operations are as follows: 1-1: Time matching If the object under epidemiological investigation was at a certain point in time ever been to a certain place, and the mobile phone signaling records show that the object under epidemiological investigation was indeed at , during that time at that place, where is a credible time error, then the information is considered credible; 1-2: Location matching Call the Gaode Map API to obtain the longitude and latitude of the locations described by the epidemiological investigation object and the locations provided in the mobile phone signaling, calculate the distance between two points using the longitude and latitude, and if the distance between the two locations is less than the credible distance error , then consider this information to be credible; 1-3: Through the above matching of time and location, if the time and location information mentioned in the Q&A pair is consistent with the data in the mobile phone signaling of the epidemiological investigation object, then this Q&A is considered credible. The accuracy rate of the credibility evaluation based on the spatio-temporal consistency of the conversation and the mobile phone signaling is calculated by the following formula (a): （a）； Among them, is the number of question-and-answer pairs with correct credibility prediction; is the total number of question-and-answer pairs; The credibility evaluation of epidemiological investigation Q&A based on context consistency uses the instruction fine-tuning technology to judge the context consistency of the conversation content on the basis of a large language model. The specific operations are as follows: For the input conversation content, if the credibility label predicted by the large language model is the same as the actual label, then this prediction is considered correct. The accuracy rate of the credibility evaluation based on the context consistency within the conversation is calculated by the following formula (b): （b）； Among them, represents the number of Q&A pairs with correct credibility prediction according to the context semantics, represents the total number of Q&A pairs.

Images