Gastroscopy-assisted detection method and system based on artificial intelligence

Abstract

The present application discloses a gastroscope-assisted detection method and system based on artificial intelligence, and relates to the technical field of gastroscope detection. The method includes: obtaining a real-time image after the gastroscope enters the patient's gastric cavity, and constructing a stomach model of the patient; obtaining gastric lesion points based on real-time image evaluation, extracting gastric lesion points observed without obstruction at the path reference point and using them as observation lesion points; counting the number of observation lesion points corresponding to the path reference point, and screening to obtain the path point; forming multiple path reference lines based on the path point, and selecting the candidate path line without missed detection; analyzing the degree of careful observation of the gastric lesion point by the candidate path line, and screening to obtain the primary path line; obtaining the patient's facial image, judging the patient's gastric mucosal sensitivity, and evaluating the operation stimulation of the primary path line; selecting the best gastroscope examination path line, and sending it to the user end to prompt the operating doctor. The present application improves the patient's comfort during gastroscopy.

Description

Technical Field

[0001] The present application relates to the technical field of gastroscopy detection, and in particular to a gastroscopy-assisted detection method and system based on artificial intelligence. Background Art

[0002] Gastroscopy is a medical examination method. With the help of a thin, soft tube inserted into the stomach, the doctor can directly observe the real situation of the examination parts such as the esophagus, stomach and duodenum. As an advanced medical examination method, the continuous development and improvement of gastroscopy has also provided strong support for medical research and progress. Through the continuous innovation and application of gastroscopy technology, a more scientific basis is provided for the prevention and treatment of diseases. In existing gastroscopy examinations, gastroscopy operations are generally performed based on the doctor's experience, but the doctor's experience is limited after all. During the operation, due to problems such as the patient's emotional tension, the patient may feel uncomfortable during the gastroscopy. Summary of the invention

[0003] The purpose of the present invention is to provide an artificial intelligence-based gastroscopy-assisted detection method and system to solve the problems raised in the above-mentioned background technology.

[0004] In the first aspect, the gastroscopy-assisted detection method based on artificial intelligence provided by the present application adopts the following technical solutions:

[0005] Obtaining real-time images after the gastroscope enters the patient's stomach cavity, and constructing the patient's stomach model based on the real-time images;

[0006] Gastric lesion points are obtained according to real-time image evaluation, multiple points are arbitrarily selected in the gastric model as path reference points, and gastric lesion points observed without obstruction at the path reference points are extracted and used as observed lesion points;

[0007] The number of observed lesion points corresponding to the path reference points is counted and recorded as the observation number, and the path reference points that do not reach the preset observation number threshold are screened and removed as path points;

[0008] A plurality of path reference lines are formed according to the path points, and a path reference line without missed detection is selected as a candidate path line;

[0009] Analyze the degree of careful observation of the gastric lesion point by the candidate path lines, and select the primary path lines from the candidate path lines according to the degree of careful observation;

[0010] Acquire a facial image of the patient, determine the patient's gastric mucosal sensitivity based on the facial image, and evaluate the operation stimulation of the primary path line based on the real-time image;

[0011] The operation stimulation threshold is set according to the sensitivity of the gastric mucosa, and the primary path line whose operation stimulation does not reach the operation stimulation threshold is selected as the intermediate path line;

[0012] The operation convenience of the intermediate path line is obtained, and the intermediate path line with the highest operation convenience is selected as the examination path line, and sent to the user end to prompt the operating doctor.

[0013] Preferably, the step of obtaining a real-time image after the gastroscope enters the patient's stomach cavity and constructing the patient's stomach model according to the real-time image is specifically:

[0014] After the gastroscope enters the patient's stomach cavity, an image is taken as a primary image, and the primary image is used to determine whether there is any obstruction in the direction of the gastroscope's advance;

[0015] If the gastroscope's forward direction is blocked, the gastroscope is rotated to find an unblocked direction to update the gastroscope's forward direction;

[0016] If there is no obstruction in the advancing direction of the gastroscope, the image blur of the primary image is extracted, and the proportion of the artifact area in the primary image is counted;

[0017] Setting an image blur threshold and an artifact area ratio threshold to determine whether the image blur and the artifact area ratio of the primary image reach the image blur threshold and the artifact area ratio threshold respectively;

[0018] If they are respectively reached, it is judged whether the patient is nervous to obtain the patient's nervousness result, and it is confirmed whether the gastroscope is advanced according to the patient's nervousness result;

[0019] If it is not reached, the operating physician is prompted to continue to operate the gastroscope in the direction of its advancement and take stomach images. When the gastric antrum is reached, all stomach images are taken as real-time images and a stomach model is constructed.

[0020] Preferably, if the above conditions are respectively met, the step of judging whether the patient is nervous to obtain a patient nervousness result, and confirming whether the gastroscope should be advanced according to the patient nervousness result, is specifically as follows:

[0021] If they are respectively achieved, the patient's heart rate monitoring data is obtained, and whether the patient is nervous is determined based on the patient's heart rate monitoring data, and the patient's nervousness degree is obtained;

[0022] If the patient is nervous, the forward space is extracted based on the primary image, and the friction probability between the gastroscope and the stomach is obtained based on the forward space analysis;

[0023] A friction probability threshold is set according to the patient's nervousness. If the friction probability is greater than the friction probability threshold, the operating doctor is notified in time to stop the gastroscope from moving forward.

[0024] If the patient is not nervous, notify the operating doctor to check the instruments and operating techniques.

[0025] Preferably, if the patient is nervous, the step of extracting the forward space according to the primary image, and obtaining the friction probability between the gastroscope and the stomach according to the forward space analysis is specifically:

[0026] The volume of the forward space is obtained, and the probability of the gastroscope contacting the stomach is determined based on the volume of the gastroscope and recorded as the primary probability;

[0027] Obtain the experience of the operating doctor, and extract the average probability of the gastroscope contacting the stomach in the doctor's historical gastroscopy operations as the intermediate probability based on the operating doctor's experience;

[0028] Obtain the patient's body position change image, and calculate the probability that the patient correctly executes the body position instruction according to the body position change image and the doctor's instruction and record it as the high-level probability;

[0029] The probability of friction between the gastroscope and the stomach is obtained according to the primary probability, the intermediate probability and the advanced probability.

[0030] Preferably, the step of analyzing the degree of careful observation of the gastric lesion point by the candidate path line and selecting the primary path line from the candidate path lines according to the degree of careful observation is specifically as follows:

[0031] estimating an image of the stomach lesion point obtained after observing the stomach lesion point according to the path line to be selected and recording it as the estimated image;

[0032] Extracting the observation angle of the gastric lesion point in the predicted image, and forming a gastric lesion point model according to the observation angle;

[0033] The lesion type is judged according to the gastric lesion point model to obtain the lesion judgment result;

[0034] According to the lesion judgment result, it is judged whether the observation of the gastric lesion point by the selected path line is careful;

[0035] The number of gastric lesion points that are not carefully observed in the candidate path line is counted and recorded as the fuzzy number;

[0036] A candidate path line with a fuzzy quantity of 0 is selected from the candidate path lines as a primary path line.

[0037] Preferably, the step of judging whether the observation of the stomach lesion point by the selected path line is careful according to the lesion judgment result is specifically:

[0038] According to the gastric lesion point model, it is determined whether the image of any angle of the gastric lesion point is not missed;

[0039] If there is no omission, the estimated average distance value between the path line to be selected and the gastric lesion point is estimated;

[0040] Obtain the historical average distance value of gastric lesion points observed by gastroscopy, and calculate the ratio of the expected average distance value to the historical average distance value to obtain the observation accuracy;

[0041] Set an observation accuracy threshold. If the observation accuracy reaches the observation accuracy threshold, the observation is judged to be careful, otherwise it is judged to be not careful.

[0042] If there are omissions, the number of lesion types in the lesion judgment results is counted, and whether the observation of the gastric lesion points is thorough is judged based on the number of types.

[0043] Preferably, if there is any omission, the step of counting the number of lesion types in the lesion judgment result and judging whether the observation of the gastric lesion point is careful according to the number of types is specifically as follows:

[0044] Determine whether the number of types is 1. If the number of types is 1, extract the lesion type in the lesion judgment result as the reference lesion type;

[0045] Obtain standard features of reference lesion types, extract features of gastric lesion point model and record them as image features;

[0046] Calculate the coincidence rate and similarity between the image features and the standard features, set the feature weight ratios of the coincidence rate and the similarity respectively, calculate the observation accuracy based on the coincidence rate, the similarity and the corresponding feature weight ratio, and judge whether the observation is careful based on the observation accuracy;

[0047] If the number of types is not 1, all lesion types in the lesion judgment results are extracted and recorded as possible lesion types;

[0048] Determine whether the possible lesion type needs to be distinguished with the help of other information. If so, observe carefully;

[0049] If no other information is needed to make the distinction, then the observation is not careful.

[0050] Preferably, the steps of acquiring a facial image of the patient, determining the gastric mucosal sensitivity of the patient according to the facial image of the patient, and evaluating the operation stimulation degree of the primary path line according to the real-time image are specifically as follows:

[0051] Acquire a facial image of the patient, and determine the degree of pain of the patient based on the facial image of the patient;

[0052] Extracting a real-time gastric mucosa image from the real-time image, obtaining a standard gastric mucosa image, and comparing the similarity between the real-time gastric mucosa image and the standard gastric mucosa image;

[0053] The frequency of the patient's discomfort state is extracted and recorded as the discomfort frequency, and the patient's gastric mucosal sensitivity is obtained by combining the patient's pain level and approximation;

[0054] The operational stimulation degree of the primary path line is obtained based on the real-time image evaluation.

[0055] Preferably, the step of obtaining the operation stimulation degree of the primary path line according to the real-time image evaluation is specifically:

[0056] Extracting a gastric peristaltic wave image from a real-time image, and obtaining gastric peristaltic intensity according to the gastric peristaltic wave image;

[0057] The total distance of the primary path line was obtained and recorded as the route distance, and the number of expected rotations of the gastroscope in the primary path line was counted;

[0058] The stimulation weight ratios of gastric peristalsis intensity, route distance and expected number of rotations are set respectively, and the operation stimulation degree is calculated according to the gastric peristalsis intensity, route distance, expected number of rotations and the corresponding stimulation weight ratios.

[0059] In the second aspect, the gastroscopy-assisted detection system based on artificial intelligence provided by the present application adopts the following technical solutions:

[0060] The AI-based gastroscopy-assisted detection system includes:

[0061] The stomach model module obtains the real-time image after the gastroscope enters the patient's stomach cavity and constructs the patient's stomach model based on the real-time image;

[0062] The lesion observation module obtains gastric lesion points based on real-time image evaluation, selects multiple points in the gastric model as path reference points at random, extracts gastric lesion points that are observed without obstruction at the path reference points and uses them as observation lesion points;

[0063] The path reference module counts the number of observed lesion points corresponding to the path reference points and records them as the number of observations, and removes the path reference points that do not reach the preset observation number threshold as path points;

[0064] The candidate path module forms multiple path reference lines according to the path points, and selects the path reference lines without missed detection as the candidate path lines;

[0065] The primary path module analyzes the degree of observation of the gastric lesion points by the candidate path lines, and selects the primary path lines from the candidate path lines according to the degree of observation;

[0066] The operation stimulation module obtains the patient's facial image, determines the patient's gastric mucosal sensitivity based on the patient's facial image, and evaluates the operation stimulation of the primary path line based on the real-time image;

[0067] The intermediate path module sets the operation stimulation threshold according to the sensitivity of the gastric mucosa, and selects the primary path lines whose operation stimulation does not reach the operation stimulation threshold as the intermediate path lines;

[0068] The inspection path module obtains the operation convenience of the intermediate path line, selects the intermediate path line with the highest operation convenience as the inspection path line, and sends it to the user end to prompt the operating doctor.

[0069] In summary, the present application includes at least one of the following beneficial technical effects:

[0070] 1. Build a stomach model based on the real-time image after the gastroscope enters the patient's gastric cavity, evaluate and obtain the gastric lesion points, select path points based on the number of gastric lesion points observed without obstruction at the path reference points, and form multiple path reference lines. Select the path reference lines without missed detection as the candidate path lines, and then select the best inspection path line based on the carefulness of observation of the candidate path lines, the sensitivity of the patient's gastric mucosa, the degree of operation stimulation, and the convenience of operation, and send a prompt to the operating doctor. While meeting the requirements of gastroscopy, it greatly reduces the patient's discomfort and improves the patient's comfort during gastroscopy.

[0071] 2. In the process of taking real-time images before building the stomach model, determine whether the image blur threshold and the proportion of artifact area have reached the threshold. If the threshold is reached, determine whether the patient is nervous. If the patient is nervous, it is necessary to obtain the friction probability between the gastroscope and the stomach based on the space volume of the gastroscope, the doctor's experience, and the patient's cooperation. According to the friction probability, determine whether the gastroscope needs to be temporarily advanced. During the gastroscopy process, the patient's condition is fully considered, the friction between the gastroscope and the stomach is reduced, and the safety of gastroscopy-assisted detection based on artificial intelligence is improved.

[0072] 3. According to whether the shooting angle of the candidate path line to the gastric lesion point has any omissions and the number of lesion judgment results obtained using the gastric lesion point model, judge whether the observation of the candidate path line is careful, and screen out the candidate path lines with inattentive gastric lesions. Then confirm the patient's gastric mucosal sensitivity based on the patient's facial image, gastric mucosal image and the patient's discomfort frequency. The operational stimulation degree of the candidate path line is obtained according to the gastric peristalsis intensity, route distance and expected number of rotations, and then the corresponding operational stimulation degree is set according to the gastric mucosal sensitivity to screen the appropriate candidate path line. It not only takes into account the comprehensiveness of the path line inspection and meets the needs of gastroscopy, but also reduces the patient's discomfort and improves the comprehensiveness of gastroscopy-assisted detection based on artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a schematic diagram of the specific steps of an embodiment of the gastroscopy-assisted detection method based on artificial intelligence of the present invention.

[0074] Figure 2 It is a schematic diagram of module connection of an embodiment of a gastroscopy-assisted detection system based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0075] Below is a combination of the embodiments and Figure 1-Figure 2 The present invention is further described in detail, but the embodiments of the present invention are not limited thereto.

[0076] The present invention discloses a gastroscopy-assisted detection method based on artificial intelligence, which specifically comprises the following steps:

[0077] Step S1, obtaining a real-time image after the gastroscope enters the patient's stomach cavity, and constructing the patient's stomach model according to the real-time image.

[0078] Step S2, obtaining gastric lesion points according to real-time image evaluation, arbitrarily selecting multiple points in the gastric model as path reference points, extracting gastric lesion points observed without obstruction at the path reference points and using them as observed lesion points.

[0079] By comparing the real-time image of the stomach with the normal image of the stomach, the abnormality of the stomach image can be found, so as to judge which parts of the patient's stomach may have lesions, and thus determine the gastric lesion points. At the same time, the position point that the gastroscope can reach in the stomach is selected as the path reference point. Due to the position of the path reference point and the gastric lesion point, not all gastric lesion points can be observed at a certain path reference point. Therefore, the gastric lesion point that can be directly observed without obstruction at the path reference point is selected as the observation lesion point.

[0080] Step S3, counting the number of observed lesion points corresponding to the path reference points and recording them as the observation number, and screening out the path reference points that do not reach the preset observation number threshold as path points.

[0081] The more lesion points that can be observed at a path reference point, the more suitable the path point is for gastroscopy to arrive and observe. On the one hand, more lesion points can be observed at this path point, and on the other hand, the patient's stomach can be observed more comprehensively while reducing the movement of the gastroscope.

[0082] Step S4, forming multiple path reference lines according to the path points, and selecting the path reference lines without missed detection as the path lines to be selected.

[0083] Arranging and combining multiple path points can form multiple different path reference lines. For example, the M method for intragastric observation is an M-shaped path.

[0084] Step S5, analyzing the degree of observation of the gastric lesion point by the candidate path lines, and selecting the primary path line from the candidate path lines according to the degree of observation.

[0085] Step S6, obtaining a facial image of the patient, determining the gastric mucosal sensitivity of the patient based on the facial image of the patient, and evaluating the operation stimulation degree of the primary path line based on the real-time image.

[0086] Step S7, setting an operation stimulation threshold according to the sensitivity of the gastric mucosa, and screening primary path lines whose operation stimulation does not reach the operation stimulation threshold as intermediate path lines.

[0087] The more sensitive the patient's gastric mucosa is, the lower the acceptable operation stimulation is. The corresponding operation stimulation is set as the threshold according to the sensitivity of the gastric mucosa.

[0088] Step S8, obtaining the operation convenience of the intermediate path line, selecting the intermediate path line with the highest operation convenience as the examination path line, and sending it to the user end to prompt the operating doctor.

[0089] The ease of operation is evaluated based on the difficulty of the intermediate route and the accuracy required for the operation, and the most convenient intermediate path line is selected as the examination path line to prompt the operating doctor and provide a reference for the doctor during the gastroscopy process to reduce the patient's discomfort during the gastroscopy process.

[0090] In actual use, after the gastroscope enters the gastric cavity, due to the large space in the gastric cavity, repeated examinations are required in the gastric cavity to reduce the possibility of missed detections. However, due to individual differences, the stomachs of different patients are different, and different patients have different tolerances to gastroscopy. Therefore, the same gastroscopic path examination is not suitable for all patients. The construction of the path in the gastric cavity mainly relies on the doctor's experience, and the main purpose is to ensure that no detection is missed. This causes gastroscopy to cause discomfort to some patients, and even causes stomach damage to patients. Analyzing the images in the gastric cavity through artificial intelligence is conducive to planning an examination path that is more suitable for patients, and minimizing the discomfort of patients during the gastroscopy process while ensuring that no stomach detection is missed.

[0091] The steps of obtaining a real-time image after the gastroscope enters the patient's stomach cavity and constructing the patient's stomach model according to the real-time image are specifically as follows:

[0092] Step S11, after the gastroscope enters the patient's stomach cavity, an image is captured as a primary image, and it is determined based on the primary image whether there is any obstruction in the advancing direction of the gastroscope.

[0093] Step S12: If the forward direction of the gastroscope is blocked, the gastroscope is rotated to find an unblocked direction to update the forward direction of the gastroscope.

[0094] Step S13: if there is no obstruction in the advancing direction of the gastroscope, the image blur of the primary image is extracted, and the proportion of the artifact area in the primary image is counted.

[0095] Step S14, setting an image blur threshold and an artifact area ratio threshold, and determining whether the image blur and the artifact area ratio of the primary image reach the image blur threshold and the artifact area ratio threshold respectively.

[0096] Step S15: if the above conditions are met respectively, it is determined whether the patient is nervous or not to obtain the patient nervousness result, and whether the gastroscope should be advanced is determined according to the patient nervousness result.

[0097] Step S16: if it is not reached, the operating doctor is prompted to continue to operate the gastroscope in the direction of the gastroscope and take stomach images. When the gastric antrum is reached, all stomach images are taken as real-time images and a stomach model is constructed.

[0098] In actual application, the stomach image needs to be clear enough to serve as the basis for diagnosis of the disease. At the same time, whether the stomach image is clear can also reflect the patient's psychological condition. Before constructing the examination path of the gastroscope, it is necessary to obtain a rough image of the stomach and construct a stomach model. When the gastroscope enters the stomach cavity for the first time to take pictures, in order to reduce the discomfort caused by the gastroscope to the patient, it is necessary to pay attention to the patient's mood. When the patient is uncomfortable, the gastroscope image will be blurred and the gastroscope image artifacts will increase. Therefore, when the image blur and the proportion of artifact area reach the threshold, the patient may be nervous. If it is judged that the patient is indeed nervous and it is not suitable to continue the gastroscope process, the operating doctor needs to be prompted to suspend the operation. On the one hand, continuing the operation will increase the patient's discomfort. On the other hand, it is difficult to obtain clear and accurate gastroscope images when the patient is nervous. Appropriately pausing the entry of the gastroscope can relieve the patient's emotions. If the image blur and the proportion of artifact area do not reach the threshold, the patient may be a little nervous, but it has not reached a serious level of nervousness, and the gastroscope can continue to enter.

[0099] If they are respectively achieved, then judging whether the patient is nervous or not to obtain the patient nervousness result, and confirming whether the gastroscope should be advanced according to the patient nervousness result, specifically, the steps are as follows:

[0100] Step S151, if they are respectively reached, the patient's heart rate monitoring data is obtained, and whether the patient is nervous is determined based on the patient's heart rate monitoring data, and the patient's nervousness degree is obtained.

[0101] Step S152: If the patient is nervous, the forward space is extracted based on the primary image, and the friction probability between the gastroscope and the stomach is obtained based on the forward space analysis.

[0102] Step S153, setting a friction probability threshold according to the patient's nervousness. If the friction probability is greater than the friction probability threshold, the operating doctor is promptly notified to stop the gastroscope from moving forward.

[0103] Step S154: If the patient is not nervous, notify the operating doctor to check the equipment and operating techniques.

[0104] In actual use, when patients are nervous and uncomfortable, it does affect the gastroscopy image. However, the blurry gastroscopy image and the increase of gastroscopy image artifacts do not necessarily mean that the patient is nervous. Therefore, when the gastroscopy image is blurred and the gastroscopy image artifacts increase, it is necessary to confirm whether it is caused by the patient's nervousness. The heart rate of the patient will change when the patient is nervous, so it can be confirmed whether the patient is nervous based on the patient's heart rate. If it is confirmed that the patient is nervous, it is necessary to determine whether the gastroscopy operation can be continued in this case. The patient's nervousness can be confirmed based on the patient's heart rate. The faster the patient's heart rate, the more nervous the patient is, and the lower the probability of friction that the patient can accept. Because the friction between the gastroscope and the stomach causes discomfort, it will cause adverse reactions to the patient. When the patient is not so nervous, the acceptable probability of friction will be greater. Therefore, the patient can set the acceptable probability threshold of the gastroscope and the stomach according to the patient's nervousness. If the probability of friction that the gastroscope continues to go deeper has exceeded the threshold of the probability of friction that the patient can bear, it is necessary to prompt the operating doctor to temporarily suspend the entry of the gastroscope. If the patient is not nervous as verified by his heart rate, it means that the blurred endoscopic image and increased endoscopic image artifacts are caused by other reasons, which prompts the doctor to check the instruments and operating techniques in time to reduce the problem of unclear endoscopic images in the future.

[0105] If the patient is nervous, the forward space is extracted according to the primary image, and the friction probability between the gastroscope and the stomach is obtained according to the forward space analysis, specifically, the following steps:

[0106] Step S1521, obtaining the volume of the forward space, and determining the probability of the gastroscope contacting the stomach in combination with the volume of the gastroscope and recording it as the primary probability.

[0107] The historical gastroscopy records are obtained, and the probability of the gastroscope contacting the stomach under different forward space volumes is recorded respectively. The corresponding primary probability is obtained according to the currently obtained forward space volume.

[0108] Step S1522, obtaining the operating doctor's experience, and extracting the average probability of the gastroscope contacting the stomach during the doctor's historical gastroscopy operations as the intermediate probability based on the operating doctor's experience.

[0109] Step S1523, obtaining the patient's body position change image, and calculating the probability that the patient correctly executes the body position instruction based on the body position change image and the doctor's instruction and recording it as the high-level probability.

[0110] Step S1524, obtaining the friction probability between the gastroscope and the stomach according to the primary probability, the intermediate probability and the advanced probability.

[0111] In actual use, the probability of friction between the gastroscope and the stomach is affected by the stomach space and the doctor's operating experience, including the patient's cooperation, which will also affect the friction during the gastroscopy. The smaller the space for the gastroscope to move forward, the greater the probability of contact between the gastroscope and the stomach. The more experienced the doctor is, the easier it is to avoid contact with the stomach during the operation, so the probability of contact between the gastroscope and the stomach is smaller. If the patient's cooperation is lower, the probability of contact between the gastroscope and the stomach is more likely to increase. This is caused by the patient's uncooperative swallowing movements, unstable patient position or frequent movement. Therefore, when considering the probability of contact between the gastroscope and the stomach, it is necessary to comprehensively obtain the probability of friction between the gastroscope and the stomach based on the space for the gastroscope to move forward, the doctor's experience and the patient's cooperation. The greater the primary probability, intermediate probability and advanced probability, the greater the probability of friction between the gastroscope and the stomach. For example, the primary probability is 30%, the intermediate probability is 30%, and the advanced probability is 20%. Then the calculation formula is primary probability + primary probability × (intermediate probability + advanced probability), that is, 30% + 30% × (30% + 20%) = 45%.

[0112] The steps of analyzing the degree of careful observation of the gastric lesion point by the candidate path line and selecting the primary path line from the candidate path lines according to the degree of careful observation are as follows:

[0113] Step S51 , predicting the image of the stomach lesion point obtained after observing the stomach lesion point according to the path line to be selected and recording it as the predicted image.

[0114] Step S52, extracting the observation angle of the gastric lesion point in the predicted image, and forming a gastric lesion point model according to the observation angle.

[0115] Step S53, judging the lesion type according to the stomach lesion point model to obtain a lesion judgment result.

[0116] Step S54, judging whether the path line to be selected observes the stomach lesion point carefully according to the lesion judgment result.

[0117] Step S55, counting the number of gastric lesion points that are not carefully observed in the path line to be selected and recording them as fuzzy numbers.

[0118] Step S56, selecting a candidate path line with a fuzzy quantity of 0 from the candidate path lines as a primary path line.

[0119] In actual application, in order to reduce the missed detection of gastroscopy during gastroscopy, repeated inspections are usually required, which will form an inspection path. Since the stomach morphology of different patients is inconsistent, the inspection path in the stomach also varies from person to person. Choosing a suitable inspection path can reduce the patient's discomfort while satisfying the gastroscopy. In the initial selection process, it is necessary to meet the basic requirements of gastroscopy and clearly obtain the situation of gastric lesions. For example, in the A candidate path line, the captured image of the lesion point A is relatively blurred. Such an inspection path is difficult to meet the needs of gastroscopy, so it needs to be deleted. It is necessary to select a path that can clearly capture the gastric lesion point, which can meet the purpose of gastroscopy and provide a strong basis for disease diagnosis.

[0120] The steps of judging whether the observation of the gastric lesion point by the selected path line is thorough according to the lesion judgment result are as follows:

[0121] Step S541, judging whether images of any angle of the gastric lesion point are not missed according to the gastric lesion point model.

[0122] Step S542: If there is no omission, the estimated average distance value between the path line to be selected and the gastric lesion point is estimated.

[0123] Step S543, obtaining the historical average distance value of the gastric lesion point observed by gastroscopy, and calculating the ratio of the expected average distance value to the historical average distance value to obtain the observation accuracy.

[0124] When a suspected lesion is found, the doctor will bring the front end of the gastroscope closer to the lesion area to more carefully observe the shape, color, border and other characteristics of the lesion. This close observation helps to find tiny lesions or early lesions.

[0125] Step S544, setting an observation accuracy threshold. If the observation accuracy reaches the observation accuracy threshold, it is judged that the observation is careful, otherwise it is judged that the observation is not careful.

[0126] Step S545, if there is any omission, then count the number of lesion types in the lesion judgment result, and judge whether the observation of the gastric lesion point is careful according to the number of types.

[0127] In actual application, how to judge whether the gastric lesion point has been carefully observed needs to be considered from many aspects. If the image of the gastric lesion point is taken from all angles, it means that the shooting angle is relatively comprehensive. At this time, the accuracy of the gastroscopic image observation is judged based on the estimated average distance value between the selected path line and the gastric lesion point. When the distance between the gastroscope and the gastric lesion point is far, it is difficult to clearly capture the subtle changes in the gastric lesion point, and the observation accuracy is low, so it is considered that the gastric lesion point cannot be carefully observed. If there are certain angles of the gastric lesion point that are not photographed, whether the observation is careful needs further confirmation.

[0128] If there are omissions, the number of lesion types in the lesion judgment results is counted, and the steps for judging whether the observation of the gastric lesion point is careful or not are as follows:

[0129] Step S5451, determine whether the number of types is 1. If the number of types is 1, extract the lesion type in the lesion judgment result as the reference lesion type.

[0130] Step S5452, obtain the standard features of the reference lesion type, extract the features of the gastric lesion point model and record them as image features.

[0131] Step S5453, calculate the overlap rate and similarity between the image features and the standard features, set the feature weight ratios of the overlap rate and similarity respectively, calculate the observation accuracy based on the overlap rate, similarity and the corresponding feature weight ratio, and judge whether the observation is careful based on the observation accuracy.

[0132] Step S5454: If the number of types is not 1, all lesion types in the lesion judgment result are extracted and recorded as possible lesion types.

[0133] Step S5455, determine whether the possible lesion type needs to be distinguished with the help of other information. If it needs to be distinguished with the help of other information, then observe carefully.

[0134] Step S5456: If no other information is needed to distinguish, it is determined that the observation is not careful.

[0135] In actual application, if there are angles at which no images are taken of the gastric lesion points, it does not mean that the disease cannot be judged, that is, it cannot be said that the gastroscopy observation is not careful. It is necessary to confirm whether the path inspection meets the gastroscopy requirements based on the actual diagnosis and judgment results. If the gastric lesion point model established based on the gastric images can only match one type of lesion, the accuracy of the lesion judgment observation is analyzed based on the similarity between the features in the lesion type and the standard features, as well as the overlap of the feature locations and quantities, so as to judge whether the observation is careful based on the observation accuracy threshold. If the feature overlap rate and similarity of the shots are high, even if there are no images taken at some angles, it does not affect the judgment of the lesion results, so it is considered that the observation meets the requirements. When the number of matched lesion types is not 1, it is necessary to determine whether other information is needed for distinction. For example, chronic gastritis and early gastric ulcers may both show congestion, edema and erosion of the gastric mucosa under gastroscopy. These manifestations make it difficult to directly distinguish the two on gastroscopy images, especially when the gastric ulcer is still in a very early stage and the ulcer surface has not yet formed or is not obvious. Usually, it is necessary to combine the patient's clinical symptoms, medical history and further biopsy sampling to make a distinction. This shows that the gastroscopy image is carefully observed, because the two lesion types are difficult to distinguish directly from the image, and other information is needed to make a judgment. Therefore, it is considered that the gastroscopy image is clear and accurate, and the observation is careful. If the two lesion types can be directly distinguished from the captured image, but the two lesion types are matched based on the expected image of the gastric lesion point, it means that the image is not accurate and clear, which makes it difficult to judge. Paths that are not carefully observed cannot achieve the purpose of gastroscopy and need to be screened out.

[0136] The steps of obtaining a facial image of a patient, determining the gastric mucosal sensitivity of the patient according to the facial image of the patient, and evaluating the operation stimulation degree of the primary path line according to the real-time image are specifically as follows:

[0137] Step S61, obtaining a facial image of the patient, and determining the degree of pain of the patient according to the facial image of the patient.

[0138] Count the patient's facial pain images, extract the common features of the facial pain images, and set the patient's pain level according to the feature performance. Find the corresponding pain level based on the patient's facial images obtained in real time.

[0139] Step S62, extracting the real-time gastric mucosa image from the real-time image, obtaining the standard gastric mucosa image, and comparing the similarity between the real-time gastric mucosa image and the standard gastric mucosa image.

[0140] Step S63, extracting the frequency of the patient's discomfort state and recording it as discomfort frequency, and combining the patient's pain degree and approximation to obtain the patient's gastric mucosal sensitivity.

[0141] The weight ratios of discomfort frequency, pain degree and approximation are set respectively, and the gastric mucosal sensitivity is calculated according to the discomfort frequency, pain degree, approximation and corresponding weight ratios. Among them, the higher the discomfort frequency, the higher the gastric mucosal sensitivity, the higher the pain degree, the higher the gastric mucosal sensitivity, and the higher the approximation, the lower the gastric mucosal sensitivity.

[0142] Step S64, obtaining the operation stimulation degree of the primary path line according to the real-time image evaluation.

[0143] In actual use, gastroscopy also needs to consider the patient's discomfort. The more sensitive the patient's gastric mucosa is, the stronger the discomfort caused by the stimulation of the gastroscope. Whether the patient feels pain, nausea, vomiting or stomach discomfort during the examination is an important reference for reflecting the sensitivity of the gastric mucosa. If the patient shows strong pain or discomfort, it means that the sensitivity of his gastric mucosa is relatively high. The patient's discomfort includes but is not limited to pain, nausea, vomiting, etc., and the patient's face will show painful emotions after discomfort. At the same time, the results of gastroscopy can directly show the state of the gastric mucosa, including color, texture, texture, etc. If the gastric mucosa shows abnormal manifestations such as congestion, edema, erosion, etc., it means that its sensitivity is relatively high.

[0144] The steps of obtaining the operation stimulation degree of the primary path line according to the real-time image evaluation are specifically as follows:

[0145] Step S641, extracting the gastric peristaltic wave image in the real-time image, and obtaining the gastric peristaltic intensity according to the gastric peristaltic wave image.

[0146] Step S642, obtaining the total distance of the primary path line and recording it as the route distance, and counting the number of times the gastroscope is expected to rotate in the primary path line.

[0147] Step S643, respectively setting the stimulation weight ratios of gastric peristalsis intensity, route distance and expected number of rotations, and calculating the operation stimulation degree according to the gastric peristalsis intensity, route distance, expected number of rotations and the corresponding stimulation weight ratios.

[0148] In actual use, peristaltic waves of the stomach can be observed under the gastroscope. These waves are usually transmitted from the gastric angle to the gastric antrum and pylorus in the form of rings or concentric circles. The shape and continuity of the peristaltic wave can reflect the intensity of peristalsis. For example, when the peristaltic wave is clear, continuous and powerful in the shape of rings or concentric circles, it usually indicates that the gastric peristalsis is strong. When the gastroscope passes through the stomach, it will come into contact with the gastric mucosa. The peristalsis of the stomach will increase the relative movement between the gastroscope and the gastric mucosa. This change in the state of motion will lead to a change in the friction coefficient, thereby increasing the friction between the gastroscope and the stomach and increasing the stimulation of the gastroscope to the patient's stomach. The longer the moving path, the longer the contact time and area between the gastroscope and the inner wall of these parts will increase, thereby increasing the mechanical stimulation to the gastric mucosa. During the rotation of the gastroscope, its front end may rub against the gastric mucosa. The more times it rotates, the more frequent this friction is, thereby increasing the risk of gastric mucosal damage. Therefore, the intensity of gastric peristalsis, the route distance and the expected number of rotations will increase the degree of stimulation of the gastroscope to the patient's stomach.

[0149] The artificial intelligence-based gastroscope-assisted detection system, by applying the artificial intelligence-based gastroscope-assisted detection method as described above, includes:

[0150] The stomach model module obtains the real-time image after the gastroscope enters the patient's stomach cavity and constructs the patient's stomach model based on the real-time image.

[0151] The lesion observation module obtains gastric lesion points based on real-time image evaluation, arbitrarily selects multiple points in the gastric model as path reference points, extracts gastric lesion points that are observed without obstruction at the path reference points and uses them as observation lesion points.

[0152] The path reference module counts the number of observed lesion points corresponding to the path reference points and records them as the number of observations, and screens and removes the path reference points that do not reach the preset observation number threshold as path points.

[0153] The candidate path module forms multiple path reference lines according to the path points, and selects the path reference lines without missed detection as the candidate path lines.

[0154] The primary path module analyzes the degree of observation of the gastric lesion points by the candidate path lines, and selects the primary path lines from the candidate path lines based on the degree of observation.

[0155] The operation stimulation module obtains the patient's facial image, determines the patient's gastric mucosal sensitivity based on the patient's facial image, and evaluates the operation stimulation of the primary path line based on the real-time image.

[0156] The intermediate path module sets the operation stimulation threshold according to the sensitivity of the gastric mucosa, and selects the primary path lines whose operation stimulation does not reach the operation stimulation threshold as the intermediate path lines;

[0157] The inspection path module obtains the operation convenience of the intermediate path line, selects the intermediate path line with the highest operation convenience as the inspection path line, and sends it to the user end to prompt the operating doctor.

[0158] In actual use, when performing gastroscopy for patients, it is necessary to obtain clear and accurate stomach images of the patients. However, the stomach space is large, and the inspection path inside the gastric cavity varies from person to person. It is necessary not only to consider the accuracy, clarity, and comprehensiveness of the gastroscopy image, but also the patient's mood and state. By setting, screening, and confirming the best gastroscopy inspection path, it is helpful to reduce the friction between the gastroscope and the stomach and reduce the patient's discomfort. According to the number of gastric lesion points observed without obstruction by the path reference point, the path points are selected to form multiple path reference lines. The best inspection path line is selected according to the missed detection of the path reference line, the carefulness of observation, the sensitivity of the patient's gastric mucosa, the degree of operation stimulation, and the convenience of operation, and the operating doctor is prompted to effectively reduce the patient's discomfort while meeting the requirements of the gastroscopy.

[0159] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A gastroscopy-assisted detection method based on artificial intelligence, characterized in that: The following steps are involved: Obtaining real-time images after the gastroscope enters the patient's stomach cavity, and constructing the patient's stomach model based on the real-time images; Gastric lesion points are obtained according to real-time image evaluation, multiple points are arbitrarily selected in the gastric model as path reference points, and gastric lesion points observed without obstruction at the path reference points are extracted and used as observed lesion points; The number of observed lesion points corresponding to the path reference points is counted and recorded as the observation number, and the path reference points that do not reach the preset observation number threshold are screened and removed as path points; A plurality of path reference lines are formed according to the path points, and the path reference lines that have not been missed are selected as the path lines to be selected; Analyze the degree of careful observation of the gastric lesion point by the candidate path lines, and select the primary path lines from the candidate path lines according to the degree of careful observation; Acquire a facial image of the patient, determine the patient's gastric mucosal sensitivity based on the facial image, and evaluate the operation stimulation of the primary path line based on the real-time image; The operation stimulation threshold is set according to the sensitivity of the gastric mucosa, and the primary path line whose operation stimulation does not reach the operation stimulation threshold is selected as the intermediate path line; Obtaining the operation convenience of the intermediate path line, selecting the intermediate path line with the highest operation convenience as the examination path line, and sending it to the user end to prompt the operating doctor; The steps of obtaining a facial image of the patient, determining the gastric mucosal sensitivity of the patient according to the facial image of the patient, and evaluating the operation stimulation degree of the primary path line according to the real-time image are specifically as follows: Acquire a facial image of the patient, and determine the degree of pain of the patient based on the facial image of the patient; Extracting a real-time gastric mucosa image from the real-time image, obtaining a standard gastric mucosa image, and comparing the similarity between the real-time gastric mucosa image and the standard gastric mucosa image; The frequency of the patient's discomfort state is extracted and recorded as the discomfort frequency, and the patient's gastric mucosal sensitivity is obtained by combining the patient's pain level and approximation; The operation stimulation degree of the primary path line is obtained according to the real-time image evaluation; The step of obtaining the operation stimulation degree of the primary path line according to the real-time image evaluation is specifically as follows: Extracting a gastric peristaltic wave image from a real-time image, and obtaining gastric peristaltic intensity according to the gastric peristaltic wave image; The total distance of the primary path line was obtained and recorded as the route distance, and the number of expected rotations of the gastroscope in the primary path line was counted; The stimulation weight ratios of gastric peristalsis intensity, route distance and expected number of rotations are set respectively, and the operation stimulation degree is calculated according to the gastric peristalsis intensity, route distance, expected number of rotations and the corresponding stimulation weight ratios.

2. The artificial intelligence-based gastroscopy-assisted detection method according to claim 1, characterized in that: The step of obtaining a real-time image after the gastroscope enters the patient's stomach cavity and constructing the patient's stomach model according to the real-time image is specifically as follows: After the gastroscope enters the patient's stomach cavity, an image is taken as a primary image, and the primary image is used to determine whether there is any obstruction in the direction of the gastroscope's advance; If the gastroscope's forward direction is blocked, the gastroscope is rotated to find an unblocked direction to update the gastroscope's forward direction; If there is no obstruction in the advancing direction of the gastroscope, the image blur of the primary image is extracted, and the proportion of the artifact area in the primary image is counted; Setting an image blur threshold and an artifact area ratio threshold to determine whether the image blur and the artifact area ratio of the primary image reach the image blur threshold and the artifact area ratio threshold respectively; If they are respectively reached, it is judged whether the patient is nervous to obtain the patient's nervousness result, and it is confirmed whether the gastroscope is advanced according to the patient's nervousness result; If it is not reached, the operating physician is prompted to continue to operate the gastroscope in the direction of its advancement and take stomach images. When the gastric antrum is reached, all stomach images are taken as real-time images and a stomach model is constructed.

3. The artificial intelligence-based gastroscopy-assisted detection method according to claim 2, characterized in that: If the above conditions are met respectively, then judging whether the patient is nervous or not to obtain a patient nervousness result, and confirming whether the gastroscope should be advanced according to the patient nervousness result, specifically comprises: If they are respectively achieved, the patient's heart rate monitoring data is obtained, and whether the patient is nervous is determined based on the patient's heart rate monitoring data, and the patient's nervousness degree is obtained; If the patient is nervous, the forward space is extracted based on the primary image, and the friction probability between the gastroscope and the stomach is obtained based on the forward space analysis; A friction probability threshold is set according to the patient's nervousness. If the friction probability is greater than the friction probability threshold, the operating doctor is notified in time to stop the gastroscope from moving forward. If the patient is not nervous, notify the operating doctor to check the instruments and operating techniques.

4. The artificial intelligence-based gastroscopy-assisted detection method according to claim 3, characterized in that: If the patient is nervous, the step of extracting the forward space according to the primary image and obtaining the friction probability between the gastroscope and the stomach according to the forward space analysis is specifically as follows: The volume of the forward space is obtained, and the probability of the gastroscope contacting the stomach is determined based on the volume of the gastroscope and recorded as the primary probability; Obtain the experience of the operating doctor, and extract the average probability of the gastroscope contacting the stomach in the doctor's historical gastroscopy operations as the intermediate probability based on the operating doctor's experience; Obtain the patient's body position change image, and calculate the probability that the patient correctly executes the body position instruction according to the body position change image and the doctor's instruction and record it as the high-level probability; The probability of friction between the gastroscope and the stomach is obtained according to the primary probability, the intermediate probability and the advanced probability.

5. The artificial intelligence-based gastroscopy-assisted detection method according to claim 4, characterized in that: The step of analyzing the degree of careful observation of the gastric lesion point by the candidate path line and selecting the primary path line from the candidate path line according to the degree of careful observation is specifically as follows: estimating an image of the stomach lesion point obtained after observing the stomach lesion point according to the path line to be selected and recording it as the estimated image; Extracting the observation angle of the gastric lesion point in the predicted image, and forming a gastric lesion point model according to the observation angle; The lesion type is judged according to the gastric lesion point model to obtain the lesion judgment result; According to the lesion judgment result, it is judged whether the observation of the gastric lesion point by the selected path line is careful; The number of gastric lesion points that are not carefully observed in the candidate path line is counted and recorded as the fuzzy number; A candidate path line with a fuzzy quantity of 0 is selected from the candidate path lines as a primary path line.

6. The artificial intelligence-based gastroscopy-assisted detection method according to claim 5, characterized in that: The step of judging whether the observation of the stomach lesion point by the selected path line is careful according to the lesion judgment result is specifically: According to the gastric lesion point model, it is determined whether the image of any angle of the gastric lesion point is not missed; If there is no omission, the estimated average distance value between the path line to be selected and the gastric lesion point is estimated; Obtain the historical average distance value of gastric lesion points observed by gastroscopy, and calculate the ratio of the expected average distance value to the historical average distance value to obtain the observation accuracy; Set an observation accuracy threshold. If the observation accuracy reaches the observation accuracy threshold, the observation is judged to be careful, otherwise it is judged to be not careful. If there are omissions, the number of lesion types in the lesion judgment results is counted, and whether the observation of the gastric lesion points is thorough is judged based on the number of types.

7. The artificial intelligence-based gastroscopy-assisted detection method according to claim 6, characterized in that: If there are omissions, the number of lesion types in the lesion judgment result is counted, and the step of judging whether the observation of the gastric lesion point is careful according to the number of types is specifically as follows: Determine whether the number of types is 1. If the number of types is 1, extract the lesion type in the lesion judgment result as the reference lesion type; Obtain standard features of reference lesion types, extract features of gastric lesion point model and record them as image features; Calculate the coincidence rate and similarity between the image features and the standard features, set the feature weight ratios of the coincidence rate and the similarity respectively, calculate the observation accuracy based on the coincidence rate, the similarity and the corresponding feature weight ratio, and judge whether the observation is careful based on the observation accuracy; If the number of types is not 1, all lesion types in the lesion judgment results are extracted and recorded as possible lesion types; Determine whether the possible lesion type needs to be distinguished with the help of other information. If so, observe carefully; If no other information is needed to make the distinction, then the observation is not careful.

8. The gastroscopy-assisted detection system based on artificial intelligence is characterized by: The method for gastroscopy-assisted detection based on artificial intelligence as described in any one of claims 1 to 7 is applied, comprising: The stomach model module obtains the real-time image after the gastroscope enters the patient's stomach cavity and constructs the patient's stomach model based on the real-time image; The lesion observation module obtains gastric lesion points based on real-time image evaluation, selects multiple points in the gastric model as path reference points at random, extracts gastric lesion points that are observed without obstruction at the path reference points and uses them as observation lesion points; The path reference module counts the number of observed lesion points corresponding to the path reference points and records them as the number of observations, and removes the path reference points that do not reach the preset observation number threshold as path points; The candidate path module forms multiple path reference lines according to the path points, and selects the path reference lines without missed detection as the candidate path lines; The primary path module analyzes the degree of observation of the gastric lesion points by the candidate path lines, and selects the primary path lines from the candidate path lines according to the degree of observation; The operation stimulation module obtains the patient's facial image, determines the patient's gastric mucosal sensitivity based on the patient's facial image, and evaluates the operation stimulation of the primary path line based on the real-time image; The intermediate path module sets the operation stimulation threshold according to the sensitivity of the gastric mucosa, and selects the primary path lines whose operation stimulation does not reach the operation stimulation threshold as the intermediate path lines; The inspection path module obtains the operation convenience of the intermediate path line, selects the intermediate path line with the highest operation convenience as the inspection path line, and sends it to the user end to prompt the operating doctor; The steps of obtaining a facial image of the patient, determining the gastric mucosal sensitivity of the patient according to the facial image of the patient, and evaluating the operation stimulation degree of the primary path line according to the real-time image are specifically as follows: Acquire a facial image of the patient, and determine the degree of pain of the patient based on the facial image of the patient; Extracting a real-time gastric mucosa image from the real-time image, obtaining a standard gastric mucosa image, and comparing the similarity between the real-time gastric mucosa image and the standard gastric mucosa image; The frequency of the patient's discomfort state is extracted and recorded as the discomfort frequency, and the patient's gastric mucosal sensitivity is obtained by combining the patient's pain level and approximation; The operation stimulation degree of the primary path line is obtained according to the real-time image evaluation; The step of obtaining the operation stimulation degree of the primary path line according to the real-time image evaluation is specifically as follows: Extracting a gastric peristaltic wave image from a real-time image, and obtaining gastric peristaltic intensity according to the gastric peristaltic wave image; The total distance of the primary path line was obtained and recorded as the route distance, and the number of expected rotations of the gastroscope in the primary path line was counted; The stimulation weight ratios of gastric peristalsis intensity, route distance and expected number of rotations are set respectively, and the operation stimulation degree is calculated according to the gastric peristalsis intensity, route distance, expected number of rotations and the corresponding stimulation weight ratios.

Images