An assisted examination method based on a large language and vision assistant for digestive endoscopy

Abstract

The application discloses an auxiliary examination method of a large language and visual assistant based on a digestive endoscope, which comprises the following steps: acquiring original data of an endoscopic examination case, and performing data cleaning and preprocessing; generating endoscopic visual instruction data based on GPT assistance according to the processed original data, to obtain an endoscopic multi-modal instruction tracking data set; fine-tuning and training a constructed multi-modal large language model based on a progressive transfer learning method, to realize knowledge transfer of the multi-modal large language model and an existing pre-training model to the endoscopic examination data, while avoiding catastrophic forgetting; finally, obtaining an endoscopic multi-modal large language model, and performing evaluation based on GPT-4 and manual operation. The application can exhibit excellent advantages in endoscopic examination, relieve the problem of data scarcity, accurately capture the intention of diagnosis, accurately explain medical terms, eliminate the ambiguity of diagnosis relying on lesion image reasoning, and create a precedent for the application of multi-modal large models in the field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of endoscopy, in particular to an auxiliary examination method of a large language and vision assistant based on a digestive endoscope. BACKGROUND

[0002] Endoscopy plays a crucial role in the diagnosis and treatment of gastrointestinal diseases, helping to discover abnormalities, classify lesions, and determine treatment methods. During the process of digestive tract endoscopy, doctors may encounter practical obstacles such as fatigue, stress, or limited experience, leading to errors in examination results. To address these limitations, artificial intelligence (AI) assisted endoscopy technology has emerged, which can provide higher accuracy, consistency, and speed, making the endoscopy process more efficient and of higher quality.

[0003] Existing AI assisted examination methods rely on different visual models to independently complete various tasks, resulting in fixed interfaces, limited interactivity, and adaptability to user instructions. Recently, large language models such as GPT, Gemini, and LLaMA have reached or approached human level in various language tasks, which has driven the development of general-purpose large language models in the biomedical field. They use pre-trained large prediction models for initialization and fine-tune them according to customized biomedical instruction datasets. The resulting large language models have shown great potential in various biomedical-related fields, such as assisting doctors in diagnosis, understanding patient needs, and providing reasonable treatment recommendations.

[0004] The progress of large language models has prompted researchers to incorporate them as components or core elements of visual-language large models or multi-modal large language models for visual tasks, such as LLaVA, GPT-4V, etc. These large models have shown amazing performance in complex reasoning of natural pictures and multi-round conversations. However, in related fields such as biomedicine, due to the professional and inconsistent nature of images, they cannot be directly and effectively utilized.

[0005] On the other hand, in related fields such as biomedicine, a large amount of data exists in the form of images, especially endoscopic images, tissue sections, and various medical images, which carry rich biological information and are crucial for disease diagnosis, treatment, and research. By combining the semantic understanding ability of language models and the visual information of picture modalities, large language models can more accurately infer the characteristics of lesions in medical images and provide accurate diagnosis and treatment recommendations.

[0006] Due to the lack of large-scale systematic training data, the development of multi-modal large models in the biomedical vertical field is limited. Although some major progress has been made, their applicability in the field of endoscopy is still a challenge. This is mainly because the endoscopy image data relied on by the visual part is limited by the acquisition under specific examination conditions, and its data distribution is obviously deviated from natural images. On the other hand, the language model part faces the problem of lack of professional knowledge specific to the endoscopy field, which is difficult to extract directly from existing public data. SUMMARY

[0007] The purpose of the present application overcomes the deficiencies existing in the prior art, in order to achieve the above purposes, a large language and vision assistant based on digestive endoscopy for auxiliary examination method is adopted to solve the problems raised in the background art.

[0008] A large language and vision assistant based on digestive endoscopy for auxiliary examination method, comprising the following steps:

[0009] Step S1, obtaining the original data of the endoscopy case and performing data cleaning and preprocessing;

[0010] Step S2, based on the processed original data, generating data based on GPT auxiliary endoscopy visual instructions to obtain a multi-modal instruction tracking endoscopy dataset;

[0011] Step S3, constructing a multi-modal large language model, the framework of the model is composed of a visual encoder, a large language model, and a trainable projection matrix; based on the progressive transfer learning method, the multi-modal large language model and the existing pre-training model are adapted to the endoscopy field;

[0012] Step S4, based on the obtained multi-modal large language model, using GPT-4 to score the coherence and accuracy of the model's answers, obtaining the final model, and performing endoscopy auxiliary examination.

[0013] 2. The large language and vision assistant based on digestive endoscopy for auxiliary examination method according to claim 1, wherein the specific steps in step S1 include:

[0014] Obtaining the endoscopy case of the hospital, collecting the original data from the real hospital endoscopy diagnosis scene;

[0015] Cleaning the original data with privacy and ethical issues, and cleaning the examination conclusion, giving each endoscopy picture the corresponding examination conclusion.

[0016] 3. The method of claim 1, wherein the specific steps in step S2 comprise:

[0017] Step S21, creating a set of questions about the content of the image to enhance the ability of the large language model to describe the content of the endoscopic image;

[0018] Step S22, for each endoscopic image, randomly selecting one question as a language instruction, and the answer is the image description and short conclusion after cleaning in step S21, to construct a single round dialogue dataset of image and its corresponding description text;

[0019] Step S23, using the single round question and answer about the endoscopic image generated in step S22, using the pure language large model GPT-3.5-turbo, generating a multi-round instruction following dataset, while injecting medical common sense into the dataset;

[0020] Step S24, generating different multi-round dialogues for endoscopic data with lesions to reduce model bias caused by unbalanced datasets.

[0021] 4. The method of claim 3, wherein the specific architecture of the multi-modal large language model constructed in step S3 comprises:

[0022] The multi-modal large language model is composed of a visual encoder, a large language model, and a projection matrix;

[0023] The visual encoder uses a pre-trained model of a pre-trained contrastive text-image pair to process the input image X v , to generate the corresponding visual feature Z v ;

[0024] Wherein, the visual features before and after the last Transformer layer in the visual encoder are used to construct a projection matrix W using a simple trainable multi-layer perceptron to convert the visual feature Z v into an embedded token H q aligned with the language model, which aligns the image features to the word embedding space, denoted as H v = W·Z v ;

[0025] The probability of generating the target answer X a is calculated as p(X a |X v , X instruct ); At this time, X instruct is a combined term, when t = 1, The instruction is defined as When t>1, then is defined as or

[0026] 5. The auxiliary examination method of a large language and visual assistant based on a digestive endoscope according to claim 1, wherein the specific steps of the progressive migration learning in step S3 comprise:

[0027] Step S31, based on the progressive migration learning method, in the first stage, a powerful biomedical baseline is established by combining LoRA with public biomedical data sets from LLaVA-Med, which contains medical common sense and some professional knowledge in certain biomedical fields, while understanding and generating coherent multi-modal dialogues;

[0028] Step S32, the constructed endoscopic instruction tracking data is fine-tuned in the vertical professional field of the endoscope by using the same LoRA method for the model trained in the first stage;

[0029] Step S33, the LoRA weight and part of the untrained weight are fused to obtain the final model weight.

[0030] 6. The auxiliary examination method of a large language and visual assistant based on a digestive endoscope according to claim 1, wherein the specific steps in step S4 comprise:

[0031] Step S41, a number of images are randomly selected from the validation set, and detailed dialogues for the lesions are generated using the proposed data generation pipeline;

[0032] The constructed endoscopic multi-model large prediction model predicts answers according to questions and visual input images; GPT-4 makes reference prediction according to questions and titles, serving as an upper limit of the teacher model;

[0033] After obtaining the responses of the two models, the question, visual information and the responses generated by the two models are provided to GPT-4, which is required to give an overall score;

[0034] Step S42, manual scoring is performed according to professional knowledge and experience, and high-quality and accurate evaluation of the orderliness, accuracy and relevance of the model's question and answer is required to obtain the overall score;

[0035] Step S43, the scores of the two evaluation methods are averaged to obtain the final evaluation score.

[0036] Compared with the prior art, the present application has the following technical effects:

[0037] By adopting the technical scheme, the original case data is acquired and preprocessed, so that the accuracy and availability of the data are ensured. Subsequently, a multi-modal instruction tracking data set is generated with the assistance of GPT, and a large language model is fine-tuned, so that the complex endoscopic visual instruction is accurately understood and generated. Through progressive transfer learning, the advanced model is effectively integrated into the endoscopy field, and the adaptability and practicality of the model are enhanced. Finally, strict evaluation and manual checking based on GPT-4 ensure the accuracy and reliability of the model, and provide strong technical support for endoscopic auxiliary examination. This scheme not only improves the examination efficiency, but also brings significant beneficial effects to medical diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0038] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings:

[0039] Figure 1 A step schematic diagram of an auxiliary examination method of the disclosed embodiment of the present application is shown in the figure;

[0040] Figure 2 A flowchart of the disclosed embodiment of the present application is shown in the figure;

[0041] Figure 3 A model framework diagram of the disclosed embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] Please refer to Figure 1 and Figure 2 In the embodiments of the present application, an auxiliary examination method based on a large language and visual assistant of a digestive endoscope includes the following steps:

[0044] Step S1, acquiring original data of endoscopy cases and performing data cleaning and preprocessing, the specific steps including:

[0045] Acquiring endoscopy cases of a hospital, collecting original data from real hospital endoscopy diagnosis scenes;

[0046] Cleaning the original data with privacy and ethical issues, and cleaning the examination conclusions, and assigning corresponding examination conclusions to each endoscopy picture.

[0047] In this embodiment, in order to solve the problem of serious lack of multi-modal endoscope data, a new data set is developed, which is composed of 3423 cases, each case including patient gender, age, examination date, colonoscopy image and examination conclusion and other information.

[0048] Endoscopy cases are obtained from hospitals, and the original data is collected from real hospital endoscopy diagnosis scenes, and is cleaned for privacy and ethics issues to ensure that the data used does not contain any patient personal information, only lesion pictures and reports without personal information, and the examination conclusion is manually cleaned, and each endoscopy picture is given the corresponding examination findings.

[0049] Step S2, based on the processed original data, generate data based on GPT assisted endoscopy visual instructions to obtain a multi-modal instruction tracking endoscopy data set, and fine-tune the constructed multi-modal large language model, the specific steps include:

[0050] Step S21, create a set of questions about the content of the image, which is used to enhance the ability of the large language model to describe the content of the endoscopy image.

[0051] A set of questions is created using GPT-3.5-turbo to enhance the ability of the large language model to describe the content of the endoscopy image, such as: "briefly describe the image", "provide a brief description of the given image", "provide a brief explanation of the presented picture", "present a brief summary of the clear and clear image photo" and the like.

[0052] Step S22, for each endoscopy image, randomly select a question as a language instruction, and the answer is the cleaned image description and brief conclusion in step S21, and a single round of dialogue data set of image and its corresponding description text is constructed;

[0053] For each image X v , a question is randomly selected as a language instruction, and the answer is the cleaned image description and brief conclusion in step S21, finally, the embodiment of the application constructs 14,030 images and their corresponding description texts, although the data only provides a single task instruction, i.e. image caption, but it contains diversified and representative biomedical concept samples from the original data.

[0054] Step S23, using the single round of question and answer about the endoscopy image generated in step S22, using the pure language large model GPT-3.5-turbo, generating a multi-round instruction following data set, and injecting medical common sense into the data set;

[0055] In this embodiment, in order to keep the model following the instructions consistent, the embodiments of the present application present and manage various instruction following data through the multi-round dialogue provided about biomedical images. Using the single-round question and answer about endoscopy images generated in S3, a multi-round instruction following data set is generated using the pure language model GPT, and medical common sense is injected into the data set. Specifically, based on image description, instructions are designed in the prompt, and GPT-3.5-turbo is required to generate multi-round questions and answers in the style of a helper question and answer. In order to achieve the best effect, the embodiments of the present application manually plan a small number of samples in the design of the instruction prompt to demonstrate how to generate high-quality dialogue according to the provided title and context to create multi-round question and answer where T is the total number of rounds, q and a are the questions and answers, respectively.

[0056] Step S24, generate different multi-round dialogues for endoscopy data with lesions to alleviate model bias caused by unbalanced data sets.

[0057] It should be particularly noted that, considering the significant imbalance between normal and abnormal endoscopy images, the embodiments of the present application generate different multi-round dialogues for abnormal data. This helps to alleviate the model bias caused by unbalanced data sets.

[0058] Finally, by adjusting the concept and implementing instruction tracking technology, 17,504 language-image instruction tracking samples and 64,640 dialogue samples are successfully compiled.

[0059] In this embodiment, the specific steps of constructing the multi-modal large language model in step S3 include:

[0060] The multi-modal large language model is composed of a visual encoder, a large language model, and a projection matrix;

[0061] The visual encoder of the pre-trained model of the pre-trained contrast text-image pair is used to process the input image X v , to generate the corresponding visual feature Z v ;

[0062] In this embodiment, the embodiments of the present application aim to fully utilize the capabilities of pre-trained large language models and visual models, and fine-tune training in GPT-assisted endoscopy visual instruction data.

[0063] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0064] As Figure 3As shown, the embodiment of the application shows a multimodal large language model framework, which is composed of a visual encoder and a large language model and a trainable projection matrix, and only the projection matrix and part of the parameters of the large language model are updated in model training, and all parameters of the visual encoder are completely frozen.

[0065] The visual encoder adopts the visual encoder ViT-L / 14 of the pre-trained CLIP, which is used to process the input image X v , and generate the corresponding visual feature Z v =g(X V ).

[0066] Specifically, CLIP (Contrastive Language-Image Pre-training) is a pre-training model based on contrastive text-image pairs, which maps image and text vector representations to a joint multimodal space, can learn the matching relationship of text-image pairs, and the purpose of the visual encoder using this model is to preliminarily align the generated visual feature Z v with the language feature in the high-dimensional joint multimodal space.

[0067] The selection of the large language model base directly affects the effect of the visual large language model, and in the embodiment of the application, LLAMA2 is used as the language model, LLAMA2 is a high-quality language large model opened source by Meta company, the context length is expanded from 2048 of LLAMA to 4096, which can understand and generate longer text, including 7B, 13B and 70B three models, and performs outstandingly on various benchmark sets. The most important thing is that this model can be used for research and commercial use. The LLAMA2 model performs better than the open source dialogue model on most benchmarks, and according to the usefulness and safety of human evaluation, it is a suitable alternative to closed source models.

[0068] Among them, in order to make these visual features match the language model, the visual features before and after the last Transformer layer in the visual encoder, a simple trainable multilayer perceptron is used to construct a projection matrix W, which converts the visual feature Z v into the embedded token H q aligned with the language model, denoted as H v =W·Z v .

[0069] The probability calculation of generating the target answer X a is denoted as p(X a |X v , X instruct ); At this time, X instructis a combined term, when t = 1, The instruction is defined as When t > 1, then is defined as Or

[0070] Step S3, a multi-modal large language model is constructed, the framework of the model is composed of a visual encoder, a large language model, and a trainable projection matrix; based on a progressive transfer learning method, the multi-modal large language model and the existing pre-training model are adapted to the endoscopy process, and the specific steps include:

[0071] In the embodiment, in order to better learn the medical common sense in the large-scale medical data set and the professional knowledge in the specific endoscopy field, the embodiment of the application proposes a novel progressive transfer learning method for adapting the above-mentioned model architecture and the existing pre-training model to the endoscopy field. The method includes two stages, both based on LoRA technology, and uses two different data respectively. LoRA is a technology developed by Microsoft researchers to solve the fine-tuning of large language models. The full name is Low-Rank Adaptation of Large Language Models, that is, low-order adaptation of large language models. It is an efficient parameter adaptation method for adapting to specific tasks by introducing a small number of trainable parameters while keeping most of the pre-training model parameters unchanged. The design goal of LoRA is to allow large pre-training models (such as GPT-3, BERT, etc.) to achieve efficient fine-tuning and customization in downstream tasks with lower resource consumption. At the same time, it should be noted that in order to ensure optimal performance, during the training phase, the visual encoder of the model is not trained in any way.

[0072] Step S31, based on the progressive transfer learning method, in the first stage, a powerful biomedical baseline containing medical common sense and some professional knowledge in specific biomedical fields is established by using LoRA combined with public biomedical data sets from LLaVA-Med, while understanding and generating coherent multi-modal dialogues;

[0073] Step S32, the endoscopy instruction tracking data is constructed, and the model trained in the first stage is fine-tuned in the vertical professional field of endoscopy by using the same LoRA method;

[0074] Step S33, the LoRA weight and part of the untrained weight are fused to obtain the final model weight.

[0075] Specifically, in the first stage, a strong biomedical baseline is established using LoRA combined with the public biomedical dataset from LLaVA-Med, which contains medical common sense and some professional knowledge in certain specific biomedical fields, while being able to understand and generate coherent multi-modal dialogues. In the second stage, the embodiments of the present application further use the endoscopic instruction tracking data we constructed to fine-tune the model trained in the first stage in the vertical professional field of endoscopy using the same LoRA method, aiming to improve the performance of the model by integrating data and domain expertise specific to the endoscopic task, and finally fuse the LoRA weights and part of the untrained weights to obtain the LLaVA-Endo model weights.

[0076] Step S4, according to the obtained multi-modal large language model, using GPT-4 to artificially score the coherence and accuracy of the model's answers, to obtain the final model, and to assist in endoscopic examination, the specific steps include:

[0077] Step S41, randomly select several images from the validation set, and use the proposed data generation pipeline to generate detailed dialogues for the lesions;

[0078] The constructed endoscopic multi-model large prediction model predicts answers according to questions and visual input images; GPT-4 makes reference predictions according to questions and titles as the upper limit of the teacher model;

[0079] After obtaining the responses of the two models, the question, visual information and the responses generated by the two models are provided to GPT-4, which is required to give an overall score;

[0080] Step S42, according to professional knowledge and experience, artificially score the coherence, accuracy and relevance of the model's questions and answers, to obtain the overall score;

[0081] Step S43, combine the scores of the two evaluation methods to obtain the final evaluation score.

[0082] Specifically, to evaluate the performance of the proposed model, two evaluation methods are adopted in the embodiments of the present application: GPT-4-based evaluation and manual scoring. Specifically, 20 images are randomly selected from the validation set, and the proposed data generation pipeline is used to generate detailed dialogues for the lesions. LLaVA-Endo predicts answers according to the questions and visual input images. GPT-4 makes reference predictions according to the questions and titles as the upper limit of the teacher model. After obtaining the responses of the two models, the questions, visual information and responses generated by the two models are provided to GPT-4. GPT-4 is required to evaluate the usefulness, relevance, accuracy and detail of the model responses and give an overall score of 1 to 10, where a higher score indicates better overall performance. In addition, GPT-4 is also required to provide a comprehensive explanation for the evaluation so that we can better understand the model. On the other hand, three researchers are invited to manually score according to their professional knowledge and experience, and are required to make high-quality and accurate evaluations of the organization, accuracy and relevance of the model's question and answer, and also give an overall score of 1 to 10. Finally, the scores obtained by combining the two evaluation methods are averaged to obtain the final evaluation score.

[0083] In the embodiments, the proposed model is compared with the above-mentioned excellent model, and the test results are shown in Table 1:

[0084] Table 1 compares the endoscopic test set with the SoTA model. The best result is shown in bold, and the second best result is shown underlined.

[0085]

[0086]

[0087] As can be seen from Table 1, the method proposed in the embodiments of the present application has advantages in the field of endoscopy, can effectively capture human questioning intentions, accurately explain medical proper nouns, and reason according to lesion pictures, effectively solving the inherent ambiguity in endoscopic medical problems.

[0088] Advantages of the embodiments:

[0089] We compare our proposed model with several state-of-the-art multimodal large language models, GPT-4V and Gemini, which are closed-source multimodal large language models developed by OpenAI and Google DeepMind, respectively, and show the most advanced performance in many benchmark tests. LLaVA is an open-source multimodal large language model released by researchers at the University of Wisconsin-Madison, Microsoft Research, and Columbia University, which publishes multiple parameter versions and can be used to analyze and describe images and scenes, etc. LLaVA-Med uses the same architecture and base model as LLaVA and is trained on large-scale biomedical datasets in the public network, enabling the model to understand and process biomedical images. mPLUG-Owl is a multimodal large language model based on modular implementation, which performs outstandingly in multimodal instruction understanding, multi-turn dialogue, knowledge reasoning, and other aspects. MiniGPTv2 establishes a unified input of visual and language multimodality to complete a variety of visual-language tasks, including image description, visual question answering, and visual basic tasks.

[0090] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, the scope of the present application is defined by the appended claims and their equivalents, and all should be included within the scope of protection of the present application.

Claims

1. A large language and visual assistant assisted examination method based on digestive endoscopy, comprising the following steps: Step S1, obtaining the original data of the endoscopic examination case, and performing data cleaning and preprocessing; Step S2, generating data based on GPT assisted endoscopic visual instructions according to the processed original data, obtaining a multi-modal instruction tracking endoscopic data set, the specific steps of which include: Step S21, creating a set of questions about the content of the image, which is used to enhance the ability of the large language model to describe the content of the endoscopic image; Step S22, for each endoscopic image, randomly select a question as a language instruction, and the answer is the cleaned image description and brief conclusion in step S21, to construct a single round dialogue data set of image and its corresponding description text; Step S23, using the single round question and answer about the endoscopic examination image generated in step S22, using the pure language large model GPT-3.5-turbo, generating a multi-round instruction following data set, and injecting medical common sense into the data set; Step S24, generating different multi-round dialogues for endoscopic data with lesions, which is used to reduce the model bias caused by unbalanced data set; Step S3, constructing a multi-modal large language model, the framework of which is composed of a visual encoder, a large language model, and a trainable projection matrix; based on the progressive transfer learning method, the multi-modal large language model and the existing pre-trained model are adapted to the endoscopic examination field; The specific steps of the progressive transfer learning include: Step S31, based on the progressive transfer learning method, in the first stage, using LoRA combined with the public biomedical data set from LLaVA-Med to establish a powerful biomedical baseline, which contains medical common sense and some professional knowledge in certain specific biomedical fields, while understanding and generating coherent multi-modal dialogue; Step S32, constructing endoscopic instruction tracking data, and using the same LoRA method to fine-tune the model trained in the first stage in the vertical professional field of endoscopy; Step S33, fusing the LoRA weight and part of the untrained weight to obtain the final model weight; Step S4, according to the obtained multi-modal large language model, using GPT-4 to score the coherence and accuracy of the model's answers, obtaining the final model, and performing endoscopic assisted examination, the specific steps of which include: Step S41, randomly selecting several images from the validation set, and using the proposed data generation pipeline to generate detailed dialogue for the lesions; The constructed endoscopic multi-model large prediction model predicts the answer according to the question and visual input image; GPT-4 predicts the answer according to the question and title, which is the upper limit of the teacher model; After obtaining the responses of the two models, the question, visual information and responses generated by the two models are provided to GPT-4, which is required to give an overall score; Step S42, according to professional knowledge and experience, manually score the coherence, accuracy and relevance of the model's answers, and obtain the overall score; Step S43, average the scores of the two evaluation methods to obtain the final evaluation score.

2. The method of claim 1, wherein the method is a method of assisted examination of a large language and vision assistant based on a digestive endoscope. The specific steps in the step S1 include: Obtaining the endoscopy cases of the hospital, collecting original data from the real hospital endoscopy diagnosis scene; Cleaning the original data with privacy and ethical issues, and cleaning the examination conclusion, and giving each endoscopy picture a corresponding examination conclusion.

3. The auxiliary examination method of the large language and visual assistant based on the digestive endoscope according to claim 1, and the specific architecture of the multi-modal large language model constructed in the step S3 comprises: The multi-modal large language model is composed of a visual encoder, a large language model, and a projection matrix. a visual encoder of a pre-trained model of pre-trained contrastive text-image pairs is used to process the input image , generating corresponding visual features ; where the visual features before and after the last Transformer layer in the visual encoder are used to construct projection matrices using a simple trainable multi-layer perceptron The visual features are converted to aligned token embeddings for the language model Aligning the image features to the word embedding space, denoted as ; The probability calculation of the target answer is denoted as ; at this time, is a combination item, when , is defined as ; when , is defined as or .

Images