A real-time polyp detection method, system and related equipment for intelligent endoscope

Abstract

The present application relates to a real-time polyp detection method, system and related equipment of an intelligent endoscope, including obtaining historical confirmed images and storing all confirmed images as an initial data set; establishing an initial model for polyp detection, and training and optimizing the initial model for polyp detection according to a standard data set to obtain a polyp detection model; sequentially obtaining a number of images to be detected of the current intestinal tract to be detected acquired by an image acquisition device, and uniformly standardizing the acquired images to obtain a number of images to be detected; sequentially inputting all the images to be detected into the polyp detection model to obtain actual detection results; and evaluating the accuracy of the polyp detection model according to the actual detection results and preset missed detection indicators. The present application helps doctors quickly locate abnormal areas in the intestine through real-time detection and identification by intelligent endoscopes, greatly improving the efficiency and accuracy of polyp detection.

Description

Technical Field

[0001] The present application relates to the technical field of polyp detection, and in particular to a real-time polyp detection method, system and related equipment of an intelligent endoscope. Background Art

[0002] In the modern medical field, accurate diagnosis and timely treatment of digestive system diseases are crucial. Polyps are common lesions in the digestive system, and their early detection is of great significance for preventing the occurrence and development of serious diseases such as cancer.

[0003] Traditional endoscopic examinations mainly rely on the doctor's naked eye observation to detect polyps, which has certain limitations. On the one hand, doctors need to concentrate highly during the examination, and long-term observation can easily lead to fatigue, thereby increasing the risk of missed diagnosis and misdiagnosis. On the other hand, due to the complex internal environment of the human digestive tract, such as the presence of interfering factors such as mucus, bubbles, food residues, and differences in anatomical structures in different parts, some small polyps or polyps with special shapes are difficult to detect in time.

[0004] With the continuous advancement of science and technology, the application of computer technology and image processing technology in the medical field is becoming more and more extensive. In recent years, although some polyp detection methods based on image processing have emerged, most of them analyze the images after the examination to obtain the analysis results, which may cause doctors and patients to spend a lot of time reviewing the images after the examination to determine the accuracy of the analysis results. The overall detection efficiency is low, which prolongs the diagnosis time and may also delay the treatment of patients. Summary of the invention

[0005] In order to improve the current problem of low efficiency of image analysis for polyp detection, the present application provides a real-time polyp detection method, system and related equipment for an intelligent endoscope.

[0006] In the first aspect, the present application provides a real-time polyp detection method using an intelligent endoscope, which adopts the following technical solution:

[0007] A real-time polyp detection method for an intelligent endoscope is based on a real-time polyp detection system for an intelligent endoscope, the system including an image acquisition device, characterized in that the real-time polyp detection method includes:

[0008] Obtain historical confirmed images and store all confirmed images as an initial data set;

[0009] Standardizing the initial data set to obtain a standard data set;

[0010] Establishing an initial model for polyp detection, and training and optimizing the initial model for polyp detection based on a standard data set to obtain a polyp detection model;

[0011] sequentially acquiring a plurality of images of the intestinal tract to be detected acquired by the image acquisition device, and uniformly standardizing the acquired images to obtain a plurality of images to be detected;

[0012] Inputting all the images to be detected into the polyp detection model in sequence to obtain actual detection results;

[0013] Evaluating the accuracy of the polyp detection model according to the actual detection results and the preset missed detection index;

[0014] The polyp detection initial model is trained and optimized according to the standard data set to obtain the polyp detection model, including:

[0015] Dividing the standard data set into a training set and a test set according to a preset ratio, wherein the training set and the test set both include standard images and abnormal images that have been uniformly standardized;

[0016] Training the initial polyp detection model using the training set;

[0017] Testing and optimizing the trained initial polyp detection model using the test set to obtain a polyp detection model;

[0018] The training of the initial polyp detection model using the training set includes:

[0019] Input all the standard images and abnormal images in the training set into the initial model for polyp detection, divide each image into a number of detection areas, and extract the local feature matrix of all the detection areas corresponding to each image;

[0020] The average value of the local feature matrices corresponding to all the standard images is calculated, and the average value is used as the standard feature matrix. The calculation formula of the standard feature matrix is ​​as follows:

[0021]

[0022]

[0023] in, Indicates that the nth standard image in the training set is in the detection area The local feature matrix of represents the standard feature matrix; Indicates the local feature matrix corresponding to the nth standard image Line The elements of the column; N represents the total number of standard images; Represents the local feature matrix corresponding to all standard images Line The average value of the elements of a column;

[0024] An abnormal feature matrix is ​​obtained based on the standard feature matrix and the local feature matrix of the abnormal image to complete the training of the initial model for polyp detection.

[0025] In a specific implementation scheme, the polyp detection initial model includes several detection sub-models for detecting different polyp types; the abnormal feature matrix is ​​obtained based on the standard feature matrix and the local feature matrix of the abnormal image to complete the training of the polyp detection initial model, including:

[0026] Comparing the local feature matrix of each detection area of ​​each abnormal image with the standard feature matrix of the corresponding detection area to obtain a first similarity;

[0027] Obtaining a feature matrix to be merged corresponding to some of the detection areas in each of the abnormal images according to the first similarity, and updating the standard feature matrix according to all the feature matrices to be merged;

[0028] Obtaining a suspected abnormal feature matrix corresponding to a certain detection area in each abnormal image according to the first similarity;

[0029] Each of the suspected abnormal feature matrices is tested, and an abnormal feature matrix is ​​obtained based on all the suspected abnormal feature matrices that pass the test.

[0030] In a specific implementation scheme, obtaining the feature matrix to be merged corresponding to certain detection areas in each abnormal image according to the first similarity, and updating the standard feature matrix according to all the feature matrices to be merged includes:

[0031] When the first similarity is less than a preset similarity threshold, the local feature matrix corresponding to the first similarity is used as the feature matrix to be merged, and the standard feature matrix is ​​updated according to all the feature matrices to be merged. The calculation formula for the update is as follows:

[0032]

[0033]

[0034]

[0035] in, Represents the feature matrix to be merged; represents the standard feature matrix before updating; represents the updated standard feature matrix; Represents the feature matrix Middle Line Elements of a column; Indicates The first of the feature matrices to be merged Line Elements of a column; Indicates the total number of feature matrices to be merged.

[0036] In a specific implementation scheme, each of the suspected abnormal feature matrices is tested, and the abnormal feature matrix is ​​obtained according to all the suspected abnormal feature matrices that pass the test, including:

[0037] After obtaining each suspected abnormal feature matrix, the corresponding detection area is used as the inspection area, and each inspection area is divided twice to obtain several inspection sub-areas;

[0038] Extract the test sub-feature matrix of each test sub-region, and take the average value of all the test sub-feature matrices as the test feature matrix;

[0039] Calculate the phase difference parameters between each inspection feature matrix and the suspected abnormal feature matrix corresponding to the corresponding inspection area;

[0040] If the phase difference parameter is less than a preset phase difference threshold, an abnormal feature matrix is ​​obtained according to all the inspection sub-feature matrices corresponding to the phase difference parameter.

[0041] In a specific embodiment, the calculation formula of the phase difference parameter is as follows:

[0042]

[0043]

[0044]

[0045]

[0046] in, express and The phase difference parameter between ,Right now , indicating the inspection area The corresponding test feature matrix; Display and inspection area The corresponding suspected abnormal feature matrix; Represents the feature matrix The transposed matrix of Represents the feature matrix The transposed matrix of Represents the feature matrix number of rows; Indicates the inspection sub-region The corresponding test sub-feature matrix; U represents the test area The number of inspection sub-regions included in the row; V represents the inspection region The number of test sub-regions included in the column;

[0047] The calculation formula of the abnormal feature matrix is ​​as follows:

[0048] ;

[0049] in, Indicates the inspection area The corresponding abnormal feature matrix; Indicates that the test area corresponding to the suspected abnormal feature matrix is ​​located in Line The feature matrix corresponding to the test sub-region of the column.

[0050] In a specific implementation scheme, the evaluating the accuracy of the polyp detection model according to the actual detection results and the preset missed detection index includes:

[0051] Calculating the missed detection rate of the polyp detection model according to the actual detection results;

[0052] When the missed detection rate is greater than a preset missed detection index, the accuracy of the polyp detection model is considered to be unqualified; when the missed detection rate is less than or equal to the missed detection index, the accuracy of the polyp detection model is considered to be qualified.

[0053] In a second aspect, the present application provides a smart terminal, which adopts the following technical solution:

[0054] An intelligent terminal comprises a memory and a processor, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement a real-time polyp detection method for an intelligent endoscope as described in the first aspect.

[0055] In a third aspect, the present application provides a real-time polyp detection system for an intelligent endoscope, which adopts the following technical solutions:

[0056] A real-time polyp detection system for an intelligent endoscope includes an image acquisition device, a display device, and the intelligent terminal as described in the second aspect:

[0057] The image acquisition device is used to acquire the images to be detected inside the patient's digestive tract in real time, and send the images to be detected to the smart terminal after each image to be detected is acquired;

[0058] The intelligent terminal is in communication connection with the image acquisition device and the display device, and is used to receive the image to be detected sent by the image acquisition device in real time, obtain the detection result, and then send the detection result to the display device;

[0059] The display device is used to present the received test results in real time to assist doctors in detecting the polyp condition of patients.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0061] A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the computer-readable storage medium, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement a real-time polyp detection method for an intelligent endoscope as described in the first aspect.

[0062] In summary, the present application includes at least one of the following beneficial technical effects:

[0063] 1. Improve diagnostic accuracy and efficiency: Through real-time detection and identification by smart endoscopes, doctors can quickly locate abnormal areas in the intestines, reducing the time and effort of searching for potential polyps in a large amount of normal tissue. Boxes of different colors clearly distinguish different types of polyps, allowing doctors to more intuitively understand the nature and extent of the lesions, helping to make more accurate diagnoses;

[0064] 2. Enhance doctors' confidence and decision-making ability: The real-time detection results of the smart endoscope provide doctors with an objective reference, making them more confident in diagnosis and treatment decisions. Doctors can further confirm and analyze the smart detection results based on their own professional knowledge and experience, so as to formulate a more reasonable treatment plan;

[0065] 3. Helps optimize and improve the model: After each real-time diagnosis or image test, the doctor can review the test results stored in the memory. Verify the accuracy of the polyp detection model based on the missed detection indicators, analyze the reasons and work with technical personnel to optimize and improve the model, and continuously improve the accuracy and reliability of the polyp detection model by adjusting parameters, improving algorithms, increasing training data or improving model structure, thereby further improving the performance of real-time endoscopes in intestinal polyp detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a structural schematic diagram of a real-time polyp detection system of an intelligent endoscope according to an embodiment of the present application.

[0067] Figure 2 It is a flow chart of a real-time polyp detection method using an intelligent endoscope according to another embodiment of the present application.

[0068] Figure 3 It is a structural diagram for reflecting the relationship between standard / abnormal images and detection areas in another embodiment of the present application.

[0069] Figure 4 Another embodiment of the present application Figure 3 A in the middle is an enlarged view showing the relationship between the detection area and the detection sub-area.

[0070] Explanation of reference numerals: 100, image acquisition device; 200, display device; 300, intelligent terminal; 310, processor; 320, memory. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0072] The following is a further detailed description of an embodiment of a real-time polyp detection method, system and related equipment of an intelligent endoscope in conjunction with the accompanying drawings of the specification. An embodiment of the present application discloses a real-time polyp detection system of an intelligent endoscope.

[0073] Reference Figure 1 A real-time polyp detection system for an intelligent endoscope includes an image acquisition device 100, a display device 200, and an intelligent terminal 300. The image acquisition device 100 and the display device 200 are both connected to the intelligent terminal 300 in communication. The intelligent terminal 300 of this embodiment includes a processor and a memory 320. The image acquisition device 100 is used to acquire images of the patient's digestive tract in real time, and sends the images to the intelligent terminal 300 after each image is acquired. The intelligent terminal 300 is used to receive the images sent by the image acquisition device 100 in real time for the processor to process, obtain the detection results, and then send the detection results to the display device 200. The display device 200 is used to present the detection results received from the intelligent terminal 300 to assist doctors in detecting the patient's polyp conditions. The memory 320 is used to store at least one instruction, at least one program, code set, or instruction set. The processor 310 performs the following steps of a real-time polyp detection method for an intelligent endoscope when running at least one instruction, at least one program, code set, or instruction set.

[0074] Another embodiment of the present application discloses a real-time polyp detection method of an intelligent endoscope based on the above system; Figure 2 , a real-time polyp detection method of an intelligent endoscope includes:

[0075] S100, obtaining historical confirmed images, and storing all confirmed images as an initial data set;

[0076] Among them, the confirmed images include several standard images of normal tissues in the colorectum and several abnormal images confirmed as polyps. The standard images are endoscopic images of any part of the colorectum without lesions, and the abnormal images are endoscopic images of any polyp in the colorectum and the surrounding tissues of the polyp. The standard images and abnormal images of this embodiment can come from the hospital medical record system, the database of a medical research institution, or a public medical image data set, and all abnormal images are labeled with the type of polyp. The polyp type of this embodiment takes adenomatous polyps / non-adenomatous polyps as an example, wherein adenomatous polyps include tubular adenomas, villous adenomas, and tubulovillous adenomas, and non-adenomatous polyps include hyperplastic polyps, inflammatory polyps, and hamartomatous polyps.

[0077] It should be noted that all standard images and abnormal images in this embodiment are obtained by cropping the images to a uniform size, generally with the minimum adjacent rectangle of a larger polyp as the uniform size, that is, in the abnormal images corresponding to larger polyps, the polyp area is larger; and in the abnormal images corresponding to smaller polyps, the polyp area is smaller and is located in the center of the image, and each abnormal image will more or less contain some normal tissues in the colorectum, that is, tissues adjacent to and connected to the polyps; in addition, the sample quantity of standard images and abnormal images in this embodiment will be updated in real time with the consultation situation of different patients. Generally speaking, the labels of the abnormal images in the initial data set are set by manual detection, and the labels of the newly added abnormal images will first be automatically detected by the intelligent terminal 300, and then verified and determined by humans, and only then will the abnormal images carrying the polyp type labels be added to the initial data set.

[0078] After S100, the following steps are also included:

[0079] S110, standardizing the initial data set to obtain a standard data set;

[0080] Specifically, the unified standardization in S110 includes performing preprocessing steps such as denoising, enhancement, edge detection, image cropping, and detail enhancement on all images in the initial data set, that is, all standard / abnormal images, to improve the clarity of the image and facilitate subsequent image detection; in this embodiment, the edge of the polyp is mainly enhanced by a gradient enhancement algorithm to highlight the edge contour of the polyp in the image, so that the boundary between the polyp and the surrounding normal tissue is clearer; the texture features of the image are enhanced by a filter group. Since different types of polyps may have different texture features, these features can be made more prominent by enhancing the texture. For example, for adenomatous polyps, their surface may have a specific granular texture. The texture enhancement algorithm can make this texture clearer and improve the accuracy of detection; multi-scale analysis methods such as wavelet transform are used for detail enhancement, which helps to better capture the subtle structure and features of the polyp in the future. For example, for small-sized polyps, detail enhancement can make them more obvious in the image and reduce the possibility of missed detection. The standard data set of this embodiment includes a number of standard images and a number of abnormal images that have been image enhanced and have a unified format.

[0081] S200, establishing an initial model for polyp detection, and training and optimizing the initial model for polyp detection according to a standard data set to obtain a polyp detection model;

[0082] The polyp detection model includes several detection sub-models for detecting different types of polyps. These detection sub-models are used to detect different types of polyps, and different detection sub-models have different priorities in the polyp detection model according to the incidence of different types of polyps in the colorectum. Specifically, the relationship between the polyp detection model / polyp detection initial model and the detection sub-model is shown in Table 1 below:

[0083] Table 1

[0084]

[0085] As shown in Table 1 above, this embodiment assumes that Tubular adenomas are composed of densely arranged tubular glands, which are generally small in size, smooth in surface, and mostly benign. They are the most common type of adenomatous polyps, so the detection sub-model corresponding to this type of polyp has a higher priority in the polyp detection model; hyperplastic polyps are usually small, with a diameter of less than a few millimeters, a smooth surface, and a color similar to the surrounding mucosa. They are the most common non-tumor polyps, so the corresponding detection sub-model has a priority second only to the above-mentioned tubular adenomas; villous adenomas have villous or papillary protrusions, loose structures, and rich mucus, and the corresponding detection sub-model has a priority second only to the above-mentioned hyperplastic polyps. Taking the above-mentioned different types of polyps as an example, their corresponding morphology, color, and connection surface with the surrounding tissues are all different, so this embodiment mainly uses the subsequent step S400 to detect polyps.

[0086] It should be noted that the training and optimization process of the initial model for polyp detection in S200 mainly includes two parts: obtaining a standard feature matrix and obtaining an abnormal feature matrix corresponding to each polyp type; wherein the standard feature matrix is ​​a detection standard for determining whether polyps exist in the image to be detected in the intestine in the subsequent S400, and the abnormal feature matrix is ​​a detection standard for determining the specific type of polyps existing in the image to be detected; the smart terminal 300 obtains the standard feature matrix corresponding to the polyp detection model through the following steps S221-S222 (the standard feature matrix is ​​stored in the polyp detection model), and obtains the abnormal feature matrix corresponding to each detection sub-model through the following step S223 (each detection sub-model stores an abnormal feature matrix corresponding to each polyp type).

[0087] Specifically, S200 includes:

[0088] S210, dividing the standard data set into a training set and a test set according to a preset ratio;

[0089] In this embodiment, 70% of the standard images and 70% of the abnormal images in the initial data set are used as a training set, and 30% are used as a test set.

[0090] S220, training the polyp detection model using all standard images and abnormal images in the training set;

[0091] Specifically, S220 includes:

[0092] S221, input all standard images and abnormal images in the training set into the polyp detection model, divide each image into a number of detection areas, and extract the local feature matrix of all detection areas corresponding to each image;

[0093] Among them, refer to Figure 3 , this embodiment adopts The grid divides each image in the training set into Taking 10 as an example, starting from the first detection area in the upper left corner of each image, each divided detection area is named as the position of the detection area ,Right now Indicates that the detection area is located in the corresponding image Line List, From left to right, in increasing order, Increasing from top to bottom, and ; The local feature matrix of this embodiment is Indicates that is one of the training set and the test set, the feature matrix represents Middle Image in the detection area The feature matrix of .

[0094] It should be noted that the element arrangement of all feature matrices in the present embodiment corresponds to the arrangement of pixel grayscale values ​​in the image, and the feature matrix of each detection area represents the distribution pattern of the image grayscale value in the corresponding detection area: the color gradually fades from the middle to the surroundings, the color is evenly distributed, etc. It is a prior art to use feature matrices to represent digital image data, which will not be described in detail herein. In this way, the polyp detection model of the present embodiment can detect polyps in the intestine according to the specific manifestations of different polyp types in the intestine, that is, the distribution characteristics of grayscale values ​​in a certain detection area. For example, in combination with the following steps S200-S400, in a certain image of the intestine to be detected, when the polyp type of a certain detection area is tubular adenoma, since the polyp usually appears dark red due to abundant blood vessels and more blood supply, and the color of the entire polyp is relatively uniform without obvious color difference, that is, compared with the normal tissue image, the image color is darker, and the black and gray blocks are more and more concentrated. The feature matrix to be matched corresponding to the detection area will show the characteristics of higher element values ​​and each element value is relatively average.

[0095] S222, calculating the average value of the local feature matrices corresponding to all standard images, and using the average value as the standard feature matrix;

[0096] Among them, the standard feature matrix of this embodiment is Indicates that the feature matrix represents all standard images in the training set in the detection area The average value of the local feature matrix is ​​used to reflect the gray value distribution characteristics of the standard image. The calculation formula is as follows:

[0097]

[0098]

[0099] in, Indicates that the nth standard image in the training set is in the detection area The local feature matrix of represents the standard feature matrix; Indicates the local feature matrix corresponding to the nth standard image Line The elements of the column; N represents the total number of standard images; Represents the local feature matrix corresponding to all standard images Line The mean of the elements of the column.

[0100] S223, obtaining an abnormal feature matrix corresponding to each polyp type based on the standard feature matrix and the local feature matrices of all abnormal images to complete the training of the corresponding detection sub-model.

[0101] The abnormal feature matrix is ​​used to reflect the gray value distribution characteristics of the abnormal image; the abnormal feature matrices of different polyp types will be stored in the corresponding detection sub-model; specifically, S223 includes:

[0102] S2231, performing a similarity comparison between the local feature matrix of each detection area of ​​each abnormal image and the standard feature matrix of the corresponding detection area to obtain a first similarity;

[0103] Among them, the local feature matrix corresponding to the abnormal image is Indicates that the feature matrix represents the first Anomaly images in the detection area feature matrix; the first similarity of this embodiment is mainly obtained by the cosine similarity calculation formula, and the two feature matrices are obtained by the cosine similarity formula to reflect the first similarity of the similarity between the two prior art, which will not be repeated here.

[0104] S2232, obtaining a feature matrix to be merged corresponding to certain detection areas in each abnormal image according to the first similarity, and updating the standard feature matrix according to all the feature matrices to be merged;

[0105] Specifically, S2232 includes:

[0106] S22321, when the first similarity is less than the similarity threshold, taking the local feature matrix corresponding to the first similarity as the feature matrix to be merged, and updating the standard feature matrix according to all the feature matrices to be merged;

[0107] Among them, the feature matrix to be merged refers to the local feature matrix of all detection areas in the abnormal image where no polyps exist, that is, the detection area corresponding to the local feature matrix with a high similarity to the standard feature matrix can be regarded as a normal tissue area without polyps; therefore, the standard feature matrix in S223 can be updated according to the feature matrix to be merged to continuously improve the accuracy of the standard feature matrix representing the intact standard image, and the calculation formula for completing the update is as follows:

[0108]

[0109]

[0110]

[0111] in, Represents the feature matrix to be merged, that is, the first Anomaly images in the detection area The local feature matrix of represents the standard feature matrix before updating; represents the updated standard feature matrix; Represents the feature matrix Middle Line Elements of a column; Indicates the nth feature matrix to be merged Line Elements of a column; Indicates the total number of feature matrices to be merged.

[0112] S2233, obtaining a suspected abnormal feature matrix corresponding to a certain detection area in all abnormal images of each polyp type according to the first similarity;

[0113] Specifically, S2233 includes:

[0114] S22331, when the first similarity is less than or equal to a preset similarity threshold, taking the local feature matrix corresponding to the first similarity as a suspected abnormal feature matrix;

[0115] Among them, the suspected abnormal feature matrix refers to the local feature matrix of a certain detection area where the polyp is specifically located in the abnormal image, that is, the detection area corresponding to the local feature matrix with lower similarity to the standard feature matrix can be regarded as an abnormal area where polyps exist; the similarity threshold of this embodiment takes 0.6 as an example.

[0116] S2234, testing each suspected abnormal feature matrix, and obtaining an abnormal feature matrix based on all suspected abnormal feature matrices that pass the test;

[0117] Specifically, S2234 includes:

[0118] S22341, after obtaining each suspected abnormal feature matrix, the corresponding detection area is used as the inspection area, and each inspection area is divided twice to obtain a plurality of inspection sub-areas;

[0119] Among them, refer to Figure 4 , this embodiment adopts The grid of this embodiment divides each inspection area into two parts. To facilitate subsequent calculations, Take 2 as an example.

[0120] S22342, extracting the inspection sub-feature matrix of each inspection sub-region, and taking the average value of all inspection sub-feature matrices as the inspection feature matrix;

[0121] Among them, the inspection sub-feature matrix in this embodiment is Indicates that it refers to the area in a certain inspection area. Line The characteristic matrix corresponding to the inspection sub-region of the column; the calculation method of the inspection characteristic matrix is ​​the same as the calculation method of the standard characteristic matrix in S223, which will not be repeated here.

[0122] S22343, calculating the phase difference parameter between each inspection feature matrix and the suspected abnormal feature matrix corresponding to the corresponding inspection area;

[0123] The phase difference parameter is used to reflect the degree of difference between the test feature matrix and the corresponding suspected abnormal feature matrix. The larger the value, the greater the difference between the two. The calculation formula of the phase difference parameter in this embodiment is as follows:

[0124]

[0125]

[0126]

[0127]

[0128] in, express and The phase difference parameter between ,Right now , indicating the inspection area The corresponding test feature matrix; Display and inspection area The corresponding suspected abnormal feature matrix; Represents the feature matrix The transposed matrix of Represents the feature matrix The transposed matrix of Represents the feature matrix The number of rows, in this embodiment, 2 is taken as an example, and the two-dimensional matrix has 2 rows of elements; Indicates the inspection sub-region The corresponding test sub-feature matrix; U represents the test area The number of inspection sub-regions included in the row of , in this embodiment, 2 is taken as an example, that is, S22341 ; V represents the inspection area The number of inspection sub-regions included in the column of , this embodiment takes 2 as an example, that is, S22341 .

[0129] S22344, if the phase difference parameter is less than a preset phase difference threshold, obtaining an abnormal feature matrix according to all inspection sub-feature matrices corresponding to the phase difference parameter;

[0130] The phase difference threshold in this embodiment is 0.3 as an example; the calculation formula of the abnormal feature matrix is ​​as follows:

[0131] ;

[0132] in, Indicates the inspection area The corresponding abnormal feature matrix, in this implementation, is composed of the test sub-regions corresponding to the test sub-regions (two-dimensional), so the abnormal feature matrix takes a four-dimensional array as an example, and the specific number of rows and columns is determined by the grid size of the secondary division in S22341 Decide; Indicates that the test area corresponding to the suspected abnormal feature matrix is ​​located in Line The feature matrix corresponding to the test sub-region of the column.

[0133] S22345, if the phase difference parameter is greater than or equal to the phase difference threshold, the suspected abnormal feature matrix corresponding to the phase difference parameter is regarded as a detection error. At this time, the similarity threshold in S2231 is adjusted and steps S2231-S2234 are repeated until the abnormal feature matrix is ​​obtained.

[0134] Specifically, since the same abnormal image may include several detection sub-areas where no polyps exist, and only includes one abnormal area where the polyps are located, there is only one abnormal feature matrix finally obtained in S223: when the polyp position covers multiple detection areas, this embodiment can continuously adjust the similarity threshold in S22331 so that in S22344, only the abnormal feature matrix of the detection area with the most obvious polyps is retained, and each abnormal feature matrix is ​​stored in the detection sub-model corresponding to the corresponding polyp type.

[0135] S230, testing and optimizing the trained polyp detection model using all standard images and abnormal images in the test set to obtain a polyp detection model that passes the test;

[0136] Specifically, S230 includes:

[0137] S231, inputting all standard images and abnormal images in the test set into the trained polyp detection model;

[0138] Specifically, in step S231, each image in the test set will be sequentially input into each detection sub-model in the polyp detection model for detection. This embodiment takes the steps of detecting and verifying a test image with a tubular adenoma label, that is, an abnormal image in the test set, by a detection sub-model for detecting tubular adenoma in the polyp detection model as an example for explanation. The steps are as follows:

[0139] S232, dividing the image to be tested into a plurality of detection areas, and extracting local feature matrices of all detection areas corresponding to the image to be tested;

[0140] The region segmentation method in step S232 is the same as that in step S221.

[0141] S233, performing a similarity comparison between the local feature matrix extracted from each detection area and the standard feature matrix corresponding to the corresponding detection area, to obtain a second similarity corresponding to each detection area one by one;

[0142] Among them, the second similarity is used to reflect the similarity between the feature matrix of a certain detection area in the image to be tested and the feature matrix of the corresponding detection area in the standard image. When the value is small, it can be preliminarily judged that there is a certain lesion in the detection area corresponding to the value in the image to be tested, and there is a certain possibility of polyps.

[0143] S234, obtaining a detection result based on the second similarity;

[0144] The detection result includes a first judgment result of whether there are polyps in the intestine and a second judgment result reflecting the type of polyps, and each detection result corresponds to an image to be tested; specifically, S234 includes:

[0145] S2341, when the second similarity is greater than or equal to the similarity threshold in S22331, the first judgment result is that there are polyps in the intestine, and the polyp type judgment mechanism is triggered to obtain the second judgment result, and the first judgment result and the second judgment result are used as the detection result;

[0146] Specifically, the polyp type judgment mechanism obtains the second judgment result including:

[0147] S23411, taking the detection area corresponding to the second similarity as an abnormal area, and sequentially inputting the local feature matrix corresponding to the abnormal area into the corresponding detection sub-models until a second judgment result is obtained;

[0148] Among them, sequential input means completing the input of the local feature matrix according to the priority of the detection sub-model, that is, the occurrence probability of each polyp type; specifically, the step of obtaining the polyp type according to the local feature matrix corresponding to the abnormal area for each detection sub-model includes:

[0149] S234111, splitting the abnormal region into abnormal sub-regions for a second time, and obtaining a feature matrix to be matched according to a local feature matrix of each abnormal sub-region;

[0150] The secondary segmentation of the abnormal region in this embodiment is the same as the secondary segmentation of the inspection region in S22341, and the elements of the feature matrix to be matched correspond to the local feature matrix of each abnormal sub-region, that is:

[0151]

[0152] in, Represents the feature matrix to be matched, that is, the abnormal area The corresponding local feature matrix, in this implementation, takes a four-dimensional array as an example, and each local feature matrix is ​​a two-dimensional matrix. The specific number of rows and columns is determined by the grid size of the secondary division in S22341. Decide; Indicates the abnormal area and Line The local feature matrix corresponding to the abnormal sub-region of the column.

[0153] S234112, performing a similarity comparison between the feature matrix to be matched and the abnormal feature matrix in the current detection sub-model to obtain a third similarity corresponding to the abnormal region;

[0154] Among them, the third similarity is used to reflect the similarity between the image to be tested and the abnormal image. When the value is large, it can be further judged that the type of polyp existing in the abnormal area in the image to be tested is the same as the type of polyp corresponding to the current detection sub-model.

[0155] S234113, when the third similarity is less than the similarity threshold in S22331, the second judgment result is that the polyp type is consistent with the polyp type corresponding to the current detection sub-model;

[0156] The second judgment result of this embodiment takes the polyp type as tubular adenoma as an example.

[0157] S234114, when the third similarity is greater than or equal to the similarity threshold, the local feature matrix corresponding to the abnormal area is input into the detection sub-model of the next priority, and steps SS234112-S234113 are repeated until a second judgment result is obtained.

[0158] S2342: When all the second similarities are less than the similarity threshold, the first judgment result is that there are no polyps in the intestine. In this case, the first judgment result is directly used as the detection result.

[0159] S235, obtaining a test result corresponding to each image to be tested according to the first judgment result and the second judgment result;

[0160] The test result is successful when any of the following conditions is met:

[0161] ① The first judgment result is that there are no polyps in the intestine and the image to be tested is a standard image;

[0162] ② The first judgment result is that there are polyps in the intestine and the image to be tested is an abnormal image. At the same time, the second judgment result is consistent with the type of polyps in the abnormal image;

[0163] S236, if the proportion of all the images to be tested whose corresponding test results are successful in the test set reaches the preset accuracy threshold, it is considered to have passed the test; otherwise, adjust the similarity threshold in S22321 and the phase difference threshold in S22344 and return to S220 to retrain the polyp detection model until a polyp detection model that passes the test is obtained;

[0164] In this embodiment, the accuracy threshold is taken as 90% as an example; the polyp detection model includes several detection sub-models that have passed the test.

[0165] It should be noted that, when all images of the test set in S230 have been used for testing, but a polyp detection model that passes the test is still not obtained, the number of samples in the test set can be increased by using technologies such as data augmentation, which is a prior art and will not be described in detail here.

[0166] S300, sequentially acquiring a plurality of images of the intestine to be detected acquired by the image acquisition device 100, and uniformly standardizing the acquired images to obtain a plurality of images to be detected;

[0167] In this embodiment, a process from one end of the intestine to be inspected to the other end is performed to obtain an image to be inspected every 10 cm as an example; the unified standardization in S300 is the same as the unified standardization in S110.

[0168] S400, inputting all the images to be detected into the polyp detection model in sequence to obtain an actual detection result corresponding to all the images to be detected;

[0169] Among them, the actual detection result here can be regarded as the sum of the detection results in S234, including several first judgment results of whether there is a polyp in a certain part of the intestine and several second judgment results reflecting the type of polyps. The difference is that the actual detection result in S400 corresponds to all the images to be detected of the intestine to be detected, that is: only when the first judgment results corresponding to all the images to be detected are that there are no polyps, the actual detection result is that the intestine is normal; when the first judgment result corresponding to any image to be detected is that there are polyps in the intestine, the actual detection result is that there are polyps in the intestine; the actual detection results of this embodiment are as follows: the intestine is normal or the intestine is in the first Detection area Existence Tubular adenoma, in Detection area Existence The Xth position can be automatically determined by the intelligent terminal 300 according to the detection order of the image to be detected, as well as are all natural numbers; specifically, the detection process of the image to be detected in the polyp detection model is consistent with the detection process of the image to be tested in the above steps S231-S234, the only difference is that:

[0170] In the detection test process of the test image in S234, since all abnormal polyp images in the standard data set in S110 have clear polyp type labels, that is, each abnormal polyp image only includes one type of polyp, in this process, as long as one detection sub-model obtains the second judgment result in S234, the detection is stopped;

[0171] In the actual detection process of the detection image in S400, considering that there may be a location in the actual intestine, that is, a same detection area, there are multiple polyps. Therefore, even if the polyp has been detected in a certain detection sub-model, the detection will continue until all the corresponding detection sub-models are traversed, that is, all the polyp types are obtained.

[0172] It should be noted that in the actual detection process of S400, when the first judgment result of the image to be detected is that there are polyps in the intestine, but the second judgment result cannot be obtained after traversing all the detection sub-models, the display device 200 will highlight all abnormal areas and directly use the first judgment result as the actual detection result to facilitate the detection personnel to manually detect all abnormal areas to determine the type of polyps; considering that the occurrence of the above situation usually stems from the appearance of unknown polyp types in the image to be detected, the characteristics of the polyps are not obvious enough, or there are lesions that are not polyps. Therefore, after manually determining that the abnormality is a polyp and excluding other known polyp types, the image to be detected can be used in S100 as an abnormal image corresponding to the new polyp type in the initial data set, and added to the training of the initial polyp detection model in S200 to help the model adapt to more polyp type detection in continuous detection.

[0173] It should be noted that after step S400, the display device 200 will mark the detection results for different types of polyps in different forms, generally taking the corresponding detection area as an example. Through real-time detection and identification of the smart endoscope, the doctor can quickly locate the abnormal area, reducing the time and effort of searching for potential polyps in a large amount of normal tissue, and improving the accuracy and efficiency of diagnosis; the different colored boxes clearly distinguish different types of polyps, allowing doctors to more intuitively understand the nature and scope of the lesions, which helps to make a more accurate diagnosis. For example, a red box indicates a high-risk adenomatous polyp, and a yellow box indicates a hyperplastic polyp with a lower potential risk. Doctors can quickly judge the severity of polyps based on the color and give priority to lesions with higher risks. Secondly, the real-time detection method of endoscopes enhances doctors' confidence and decision-making ability. In actual clinical work, doctors face complex cases and time pressure. The real-time detection results of smart endoscopes provide doctors with an objective reference basis, making them more confident in diagnosis and treatment decisions. In this way, doctors can combine their professional knowledge and experience to further confirm and analyze the results of smart detection, so as to formulate a more reasonable treatment plan. For example, for large or irregularly shaped polyps detected, doctors can consider biopsy or direct resection, while for smaller, low-risk polyps, follow-up observation can be selected.

[0174] In addition, for young doctors and interns, the real-time detection results of the intelligent endoscope can be used as a learning tool: they can learn how to identify different types of polyps and understand their characteristics and risk factors by observing the labels on the display device 200. At the same time, doctors can compare the detection results with the intelligent system, reflect on their own diagnosis process, and improve their own diagnosis level. In teaching hospitals, this real-time detection and identification function can be used for teaching demonstrations to help students better understand the diagnosis and treatment of intestinal polyps.

[0175] Finally, through the real-time detection and identification of the intelligent endoscope, doctors can find and treat intestinal polyps more timely, reducing the risk of missed diagnosis and misdiagnosis. This can not only improve the treatment effect and survival rate of patients, but also reduce the pain and medical expenses of patients. At the same time, patients can also understand their own conditions through the display device 200, enhance their trust and cooperation with doctors, and improve patient satisfaction.

[0176] It should also be added that the real-time polyp detection method of an intelligent endoscope of the present application also includes the following steps:

[0177] S500, evaluating the accuracy of the polyp detection model according to the actual detection results and the preset missed detection index;

[0178] The missed detection index is a parameter used to measure the number of polyps that are not detected during the polyp detection process. The missed detection index of this embodiment takes 3% as an example. Specifically, S500 includes:

[0179] S510, calculating the missed detection rate of the polyp detection model according to the actual detection results;

[0180] Among them, the missed detection rate is used to test the accuracy of the polyp detection model obtained through S200 training in the actual detection of the number of polyps, which means the proportion of the number of polyps missed by the polyp detection model to the total number of polyps among all the actually existing polyps; missed detection rate = number of missed polyps / actual number of polyps × 100%; specifically, after each real-time diagnosis is completed, that is, after S400, the doctor reviews the actual detection results of several images to be detected stored in the memory 320, so as to determine the total number of polyps in each image to be detected / intestinal tract to be detected, and obtains the missed detection rate based on the total number of polyps and the actual detection results obtained by the polyp detection model.

[0181] S520, when the missed detection rate is greater than the preset missed detection index, the accuracy of the polyp detection model is considered to be unqualified; when the missed detection rate is less than or equal to the missed detection index, the accuracy of the polyp detection model is considered to be qualified;

[0182] Among them, generally speaking, the lower the missed detection rate, the higher the accuracy of the model; when the missed detection rate is close to zero, it means that the model can accurately detect almost all polyps, and the accuracy of the model is relatively high at this time; for example, if there are 200 polyps in a test set containing 100 intestinal polyp images, the model detects 195 polyps and misses 5 polyps, then the missed detection rate is 5 / 200×100% = 2.5%. If the missed detection index is 3% at this time, then 2.5%<3%, which indicates that the accuracy of the polyp detection model in detecting intestinal polyps meets the detection requirements.

[0183] In S520, if it is found that the missed detection rate is greater than the missed detection index, the reasons can be further analyzed, considering factors such as unreasonable model parameter settings, incomplete image feature extraction, or insufficient diversity of training data. Through in-depth research on these issues, doctors can work with technical personnel to optimize and improve the polyp detection model, adjust the parameters in S200, improve the algorithm, or add more representative training data in S100 to improve the accuracy and reliability of the model. At the same time, the review process also helps doctors summarize experience and lessons, understand the performance characteristics of different types of polyps in images, and improve their own diagnostic level and ability. In subsequent diagnostic work, doctors can more keenly identify the characteristics of polyps, reduce the occurrence of missed detections and misjudgments, and provide patients with more accurate and timely diagnosis and treatment recommendations.

[0184] Based on the same inventive concept mentioned above, an embodiment of the present application also discloses an intelligent terminal, which includes a memory and a processor, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set can be loaded and executed by the processor to implement a real-time polyp detection method for an intelligent endoscope provided in the above method embodiment.

[0185] Based on the same inventive concept mentioned above, an embodiment of the present application also discloses a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set can be loaded and executed by a processor to implement a real-time polyp detection method for an intelligent endoscope provided in the above method embodiment.

[0186] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0187] Those skilled in the art will appreciate that all or part of the above embodiments may be implemented through hardware, or through a program to instruct related hardware, and the program may be stored in a computer-readable storage medium. The above-mentioned storage medium may include, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0188] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A real-time polyp detection method for an intelligent endoscope, based on a real-time polyp detection system for an intelligent endoscope, the system comprising an image acquisition device (100), characterized in that: The real-time polyp detection method comprises: Obtain historical confirmed images and store all confirmed images as an initial data set; Standardizing the initial data set to obtain a standard data set; Establishing an initial model for polyp detection, and training and optimizing the initial model for polyp detection based on a standard data set to obtain a polyp detection model; sequentially acquiring a plurality of images of the intestinal tract to be detected that are acquired by the image acquisition device (100), and uniformly standardizing the acquired images to obtain a plurality of images to be detected; Inputting all the images to be detected into the polyp detection model in sequence to obtain actual detection results; Evaluating the accuracy of the polyp detection model according to the actual detection results and the preset missed detection index; The polyp detection initial model is trained and optimized according to the standard data set to obtain the polyp detection model, including: Dividing the standard data set into a training set and a test set according to a preset ratio, wherein the training set and the test set both include standard images and abnormal images that have been uniformly standardized; Training the initial polyp detection model using the training set; Testing and optimizing the trained initial polyp detection model using the test set to obtain a polyp detection model; The training of the initial polyp detection model using the training set includes: Input all the standard images and abnormal images in the training set into the initial model for polyp detection, divide each image into a number of detection areas, and extract the local feature matrix of all the detection areas corresponding to each image; The average value of the local feature matrices corresponding to all the standard images is calculated, and the average value is used as the standard feature matrix. The calculation formula of the standard feature matrix is ​​as follows: in, Indicates that the nth standard image in the training set is in the detection area The local feature matrix of represents the standard feature matrix; Indicates the local feature matrix corresponding to the nth standard image Line The elements of the column; N represents the total number of standard images; Represents the local feature matrix corresponding to all standard images Line The average value of the elements of a column; An abnormal feature matrix is ​​obtained based on the standard feature matrix and the local feature matrix of the abnormal image to complete the training of the initial model for polyp detection.

2. The real-time polyp detection method according to claim 1, characterized in that: The polyp detection initial model includes several detection sub-models for detecting different polyp types; the abnormal feature matrix is ​​obtained based on the standard feature matrix and the local feature matrix of the abnormal image to complete the training of the polyp detection initial model, including: Comparing the local feature matrix of each detection area of ​​each abnormal image with the standard feature matrix of the corresponding detection area to obtain a first similarity; Obtaining a feature matrix to be merged corresponding to some of the detection areas in each of the abnormal images according to the first similarity, and updating the standard feature matrix according to all the feature matrices to be merged; Obtaining a suspected abnormal feature matrix corresponding to a certain detection area in each abnormal image according to the first similarity; Each of the suspected abnormal feature matrices is tested, and an abnormal feature matrix is ​​obtained based on all the suspected abnormal feature matrices that pass the test.

3. The real-time polyp detection method according to claim 2, characterized in that: The step of obtaining the feature matrix to be merged corresponding to certain detection areas in each abnormal image according to the first similarity, and updating the standard feature matrix according to all the feature matrices to be merged includes: When the first similarity is less than a preset similarity threshold, the local feature matrix corresponding to the first similarity is used as the feature matrix to be merged, and the standard feature matrix is ​​updated according to all the feature matrices to be merged. The calculation formula for the update is as follows: in, Represents the feature matrix to be merged; represents the standard feature matrix before updating; represents the updated standard feature matrix; Represents the feature matrix Middle Line Elements of a column; Indicates The first Line Elements of a column; Indicates the total number of feature matrices to be merged.

4. The real-time polyp detection method according to claim 3, characterized in that: The step of testing each of the suspected abnormal feature matrices and obtaining an abnormal feature matrix based on all the suspected abnormal feature matrices that pass the test includes: After obtaining each of the suspected abnormal feature matrices, the detection areas corresponding thereto are all used as inspection areas, and each of the inspection areas is divided twice to obtain a number of inspection sub-areas; Extracting a test sub-feature matrix of each test sub-region, and taking an average value of all the test sub-feature matrices as a test feature matrix; Calculating the phase difference parameters between each of the inspection feature matrices and the suspected abnormal feature matrix corresponding to the corresponding inspection area; If the phase difference parameter is less than a preset phase difference threshold, an abnormal feature matrix is ​​obtained according to all the inspection sub-feature matrices corresponding to the phase difference parameter.

5. The real-time polyp detection method according to claim 4, characterized in that: The calculation formula of the phase difference parameter is as follows: in, express and The phase difference parameter between ,Right now , indicating the inspection area The corresponding test feature matrix; Display and inspection area The corresponding suspected abnormal feature matrix; Represents the feature matrix The transposed matrix of Represents the feature matrix The transposed matrix of Represents the feature matrix number of rows; Indicates the inspection sub-region The corresponding test sub-feature matrix; U represents the test area The number of inspection sub-regions included in the row; V represents the inspection region The number of test sub-regions included in the column; The calculation formula of the abnormal feature matrix is ​​as follows: ； in, Indicates the inspection area The corresponding abnormal feature matrix; Indicates that the test area corresponding to the suspected abnormal feature matrix is ​​located in Line The feature matrix corresponding to the test sub-region of the column.

6. The real-time polyp detection method according to claim 1, characterized in that: The evaluating the accuracy of the polyp detection model according to the actual detection results and the preset missed detection index includes: Calculating the missed detection rate of the polyp detection model according to the actual detection results; When the missed detection rate is greater than a preset missed detection index, the accuracy of the polyp detection model is considered to be unqualified; when the missed detection rate is less than or equal to the missed detection index, the accuracy of the polyp detection model is considered to be qualified.

7. An intelligent terminal, characterized in that: It includes a memory and a processor, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the real-time polyp detection method of the intelligent endoscope as described in any one of claims 1 to 6.

8. A real-time polyp detection system for an intelligent endoscope, characterized in that: The invention comprises an image acquisition device (100), a display device (200) and the intelligent terminal (300) as claimed in claim 7: The image acquisition device (100) is used to acquire images to be detected inside the patient's digestive tract in real time, and after each image to be detected is acquired, the image to be detected is sent to the intelligent terminal (300); The intelligent terminal (300) is in communication connection with the image acquisition device (100) and the display device (200), and is used to receive the image to be detected sent by the image acquisition device (100) in real time, obtain a detection result, and then send the detection result to the display device (200); The display device (200) is used to present the received detection results in real time to assist doctors in detecting the polyp condition of patients.

9. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the real-time polyp detection method of the intelligent endoscope as described in any one of claims 1 to 6.

Images