Endoscope image screening method and system
By collecting the status information of the examined object and the movement speed of the endoscopy during the endoscopy, and performing multiple rounds of screening with image features, the problem of too many images after endoscopy is solved, and efficient and accurate image screening and diagnosis are achieved.
Patent Information
- Application Number
- CN202411907271.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-04-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, a large number of images generated after endoscopy require a doctor to check one by one, resulting in low diagnostic efficiency and poor accuracy. As the number of images increases, the doctor's workload and error rate will also increase.
By setting up multiple status monitoring components on the object to be inspected, state information is collected to judge the abnormal time period, and combining the movement speed and image characteristics of the endoscope, multiple rounds of image screening are performed to automatically filter out key images.
It improves the efficiency and accuracy of endoscopic image screening, reduces the workload of manual screening by doctors, and ensures image quality and diagnostic accuracy.
Smart Images
Figure CN119851884A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to an endoscopic image screening method and system. Background Art
[0002] An endoscope is an optical device that enters the human body through a natural orifice or a small incision made during surgery to observe relevant areas. Endoscopes of different lengths and diameters can be used to detect different areas of the human body. According to the different detection areas, endoscopes can be divided into laparoscopes, neuroendoscopes, arthroscopy, esophagoscopes and other types. Endoscopes are widely used in the detection and treatment of various parts of the human body and are an extremely important medical device in modern medicine.
[0003] In the prior art, an endoscopic examination requires taking multiple endoscopic images. Taking capsule endoscope products as an example, the current capsule endoscope products take between 2 and 30 images per second (fps), and the maximum duration is 8 hours. Taking fps as 16, a total of 16×60×60×8=460,800 images are taken during each capsule endoscope examination. If no processing is performed, the doctor must check and analyze each of the nearly 500,000 images to obtain a diagnosis. This process is not only time-consuming and laborious, but the doctor may miss abnormal lesion images and misdiagnose when visually fatigued. In addition, with the development of technology, the number of endoscopic video images will continue to increase, and accordingly, the doctor's diagnostic workload will also increase, so the diagnostic efficiency and diagnostic accuracy will be further reduced.
[0004] Therefore, there is a need to provide an endoscopic image screening method and system for automatically screening endoscopic images and improving the efficiency and accuracy of endoscopic image screening. Summary of the invention
[0005] The present invention provides an endoscopic image screening method, comprising: setting a plurality of status monitoring components on an inspected object, wherein the plurality of status monitoring components are used to collect status information of the inspected object during an endoscopic inspection of the inspected object; judging whether there is an abnormal time period based on the status information of the inspected object, and if so, determining the abnormal time period; performing a first image screening based on the abnormal time period and the time tag of each endoscopic image collected during the endoscopic inspection of the inspected object to obtain a plurality of endoscopic images after the first screening; acquiring a moving speed of the endoscope during the endoscopic inspection of the inspected object; performing a second screening on the plurality of endoscopic images after the first screening according to the moving speed of the endoscope to obtain a plurality of endoscopic images after the second screening; extracting image features of each endoscopic image after the second screening; performing a third screening on the plurality of endoscopic images after the second screening according to the image features of each endoscopic image after the second screening to obtain a plurality of screened endoscopic images.
[0006] Furthermore, a plurality of status monitoring components are provided on the inspected object, including: obtaining basic information of a plurality of historical tested objects and status information during an endoscopic inspection; clustering the plurality of historical tested objects based on the basic information of the plurality of historical tested objects to determine a plurality of historical tested object clusters; for each of the historical tested object clusters, determining a vibration monitoring position corresponding to the historical tested object cluster based on status information of the plurality of historical tested objects included in the historical tested object cluster during an endoscopic inspection; obtaining basic information of the inspected object, and determining similar historical tested object clusters from the plurality of historical tested object clusters based on the basic information of the inspected object; and providing a plurality of status monitoring components on the inspected object based on the vibration monitoring positions corresponding to the similar historical tested object clusters, wherein the status monitoring component includes at least a vibration monitoring device, and the status information of the inspected object includes at least vibration information of the vibration monitoring position collected by the vibration monitoring device.
[0007] Furthermore, based on the status information of the inspected object, it is determined whether there is an abnormal time period, including: for each of the status monitoring components, based on the vibration information of the vibration monitoring position collected by the status monitoring component, determining the first abnormality coefficient of the vibration monitoring position where the status monitoring component is located; calculating a comprehensive abnormality coefficient based on the first abnormality coefficient of the vibration monitoring position where each status monitoring component is located and the abnormality weight corresponding to each vibration monitoring position; based on the comprehensive abnormality coefficient, determining whether there is an abnormal time period, and when the comprehensive abnormality coefficient is greater than the comprehensive abnormality coefficient threshold, determining that there is an abnormal time period.
[0008] Furthermore, based on the state information of the inspected object, an abnormal time period is determined, including: dividing the process of endoscopic inspection of the inspected object into multiple time periods through a segmented model based on the vibration information of the vibration monitoring position collected by each of the state monitoring components; for each time period, based on the vibration information of the vibration monitoring position collected by each of the state monitoring components in the time period, calculating the second abnormality coefficient of each vibration monitoring position where the state monitoring component is located corresponding to the time period and the correlation coefficient of any two vibration monitoring positions where the state monitoring components are located corresponding to the time period; determining the state abnormality parameter corresponding to the time period according to the second abnormality coefficient of each vibration monitoring position where the state monitoring component is located corresponding to the time period, the correlation coefficient of any two vibration monitoring positions where the state monitoring components are located corresponding to the time period and the abnormality weight corresponding to each vibration monitoring position; determining the abnormal time period based on the state abnormality parameter corresponding to each time period.
[0009] Furthermore, the second abnormality coefficient of the vibration monitoring position where each of the condition monitoring components is located corresponding to the time period is calculated, including: performing variational modal decomposition on the vibration information of the vibration monitoring position where the condition monitoring component is located collected during the time period to generate multiple vibration components; for each of the vibration components, extracting component features of the vibration component, and calculating component abnormality values of the vibration component based on the component features of the vibration component; and calculating the second abnormality coefficient of the vibration monitoring position where the condition monitoring component is located corresponding to the time period based on the component abnormality value, center frequency and bandwidth of each of the vibration components.
[0010] Furthermore, during the process of performing endoscopic inspection on the inspected object, the moving speed of the endoscope is obtained, including: setting a magnetic induction array on the inspected object and setting a permanent magnet on the endoscope, wherein the magnetic induction array includes a plurality of magnetic sensors; and determining the moving speed of the endoscope through magnetic field strength information collected by the plurality of magnetic sensors during the process of performing endoscopic inspection on the inspected object.
[0011] Furthermore, according to the moving speed of the endoscope, the multiple endoscopic images after the first screening are screened for a second time to obtain multiple endoscopic images after the second screening, including: according to the moving speed of the endoscope, the multiple endoscopic images after the first screening are divided into multiple endoscopic image groups; for each of the endoscopic image groups, a differential image corresponding to two adjacent endoscopic images is generated, and based on the differential image corresponding to two adjacent endoscopic images, the multiple endoscopic images after the first screening included in the endoscopic image group are screened for a second time to obtain endoscopic images after the second screening.
[0012] Furthermore, extracting image features of each endoscopic image after the second screening includes: extracting image features of each endoscopic image after the second screening by using an image feature extraction model.
[0013] Furthermore, based on the image features of each endoscopic image screened for the second time, a plurality of endoscopic images screened for the second time are screened for a third time to obtain a plurality of screened endoscopic images, including: based on the basic information of the inspected object, generating standard image features of endoscopic images of a plurality of endoscopic detection positions of the inspected object; determining the possibility of a lesion in each endoscopic image screened for the second time based on the image features of each endoscopic image screened for the second time and the standard image features of endoscopic images of a plurality of endoscopic detection positions of the inspected object; based on the possibility of a lesion in each endoscopic image screened for the second time, a plurality of endoscopic images screened for the second time are screened for a third time to obtain the plurality of screened endoscopic images.
[0014] The present invention provides an endoscopic image screening system, which is applied to execute the above-mentioned endoscopic image screening method, including: a state monitoring module, including a plurality of state monitoring components arranged on the inspected object, wherein the plurality of state monitoring components are used to collect state information of the inspected object during the process of performing endoscopic inspection on the inspected object; an abnormality determination module, which is used to determine whether there is an abnormal time period based on the state information of the inspected object, and if so, determine the abnormal time period; an image screening module, which is used to determine the abnormal time period based on the abnormal time period and the time label of each endoscopic image collected during the process of performing endoscopic inspection on the inspected object , perform a first image screening to obtain a plurality of endoscopic images after the first screening; the state monitoring module is also used to obtain the moving speed of the endoscope during the endoscopic inspection of the inspected object; the image screening module is also used to perform a second screening on the plurality of endoscopic images after the first screening according to the moving speed of the endoscope to obtain a plurality of endoscopic images after the second screening; and is also used to extract image features of each endoscopic image after the second screening, and perform a third screening on the plurality of endoscopic images after the second screening according to the image features of each endoscopic image after the second screening to obtain a plurality of screened endoscopic images.
[0015] Compared with the prior art, the endoscopic image screening method and system provided in this specification have at least the following beneficial effects:
[0016] 1. By setting multiple status monitoring components on the inspected object, the status information of the inspected object can be collected comprehensively and in real time. This information includes not only the images directly observed by the endoscope, but also other physiological or physical parameters related to diagnosis, thereby improving the comprehensiveness and accuracy of the diagnosis. By judging the abnormal time period based on the status information, doctors can focus on the endoscopic images in these time periods, reduce the focus on non-critical images, and improve the efficiency and accuracy of diagnosis. Screening is performed according to the time label and abnormal time period of each endoscopic image, and irrelevant images are removed, reducing the amount of data for subsequent processing. Screening according to the moving speed of the endoscope can further eliminate those images that are blurred or cannot clearly show the status of the inspected object due to rapid movement, ensuring that the remaining images are of higher quality and more conducive to diagnosis. Screening based on image features can further eliminate those images with high similarity and low information content, leaving the most representative images for doctors to analyze, thereby improving the utilization value of image data. The entire screening process can be automated, greatly reducing the workload of doctors in manually screening images and improving inspection efficiency. Through multiple rounds of screening, doctors can quickly locate abnormal images, so as to make diagnoses or formulate treatment plans more quickly;
[0017] 2. Through cluster analysis, the historical tested objects are grouped according to basic information, and the vibration monitoring position of each group is determined. In this way, the condition monitoring components, especially the vibration monitoring device, can be customized according to the specific characteristics of the inspected object, so as to more accurately collect the status information related to the diagnosis. After the vibration monitoring position is determined, the vibration information directly related to the health status of the inspected object can be collected more accurately, reducing the interference of other non-critical position information and improving the accuracy of diagnosis;
[0018] 3. Vibration information from different vibration monitoring positions is collected through multiple condition monitoring components to achieve multi-source data fusion. This helps to more comprehensively reflect the health status of the inspected object and improve the accuracy of anomaly detection. The first anomaly coefficient of each condition monitoring component is calculated, and the comprehensive anomaly coefficient is calculated in combination with the anomaly weight. This method takes into account the importance and sensitivity of different monitoring positions, making anomaly judgment more scientific and reasonable.
[0019] The endoscopic inspection process is divided into multiple time periods through a segmented model, and each time period is analyzed independently. This helps to accurately locate the time period when the abnormality occurs and reduce the possibility of misdiagnosis and missed diagnosis. In each time period, the second abnormal coefficient of each state monitoring component and the correlation coefficient between any two state monitoring components are calculated. This further refines the abnormality detection process and improves the accuracy of locating the abnormal time period. The vibration information is subjected to variational modal decomposition to generate multiple vibration components. This method can extract different components in the vibration signal and reveal more features hidden in the vibration information. The component features of the vibration components are extracted, and the component abnormal values are calculated. This helps to identify abnormal components in the vibration signal and provide a more reliable basis for abnormal judgment. After determining the abnormal time period, the endoscopic images can be screened more specifically. This reduces the number of images in the non-abnormal time period and improves the efficiency and accuracy of image screening;
[0020] 4. By setting a magnetic induction array on the object to be inspected and a permanent magnet on the endoscope, the moving speed of the endoscope during the inspection process can be obtained in real time. According to the moving speed of the endoscope, the endoscopic images after the first screening are divided into multiple endoscopic image groups. It helps to classify images with similar motion characteristics for subsequent analysis and processing. Generate a differential image corresponding to two adjacent endoscopic images, and perform a second screening based on the differential image. The differential image can highlight the changes between the images, which helps to identify images with significant changes, thereby improving the accuracy and efficiency of screening. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:
[0022] Figure 1 is a flow chart of an endoscopic image screening method shown in one embodiment of the present application;
[0023] Figure 2 It is a module diagram of an endoscopic image screening system shown in one embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments.
[0025] Figure 1 is a flow chart of an endoscopic image screening method shown in one embodiment of the present application. Figure 1 As shown, an endoscopic image screening method may include the following steps.
[0026] Step 110: setting a plurality of condition monitoring components on the inspected object.
[0027] Among them, multiple status monitoring components are used to collect status information of the inspected object during the process of endoscopic inspection of the inspected object.
[0028] Furthermore, step 110 specifically includes:
[0029] Obtaining basic information of multiple historical test subjects (e.g., gender, age, medical history (e.g., past medical history, family medical history, etc.), physical condition (e.g., body temperature, blood pressure, heart rate, etc.) and status information during the endoscopic examination (e.g., vibration information of multiple locations of the body of the historical test subject);
[0030] Based on the basic information of the multiple historical tested objects, the multiple historical tested objects are clustered to determine the multiple historical tested object clusters. For example, the cosine similarity of the basic information of any two historical tested objects can be calculated, and the multiple historical tested objects are clustered according to the cosine similarity of the basic information of any two historical tested objects by a clustering algorithm (for example, a K-means clustering algorithm, a hierarchical clustering algorithm, etc.) to determine the multiple historical tested object clusters.
[0031] For each historical tested object cluster, based on state information of a plurality of historical tested objects included in the historical tested object cluster during the process of endoscopic examination, determining a vibration monitoring position corresponding to the historical tested object cluster;
[0032] Acquire basic information of the inspected object, and determine a similar historical tested object cluster from multiple historical tested object clusters according to the basic information of the inspected object;
[0033] According to the vibration monitoring positions corresponding to the clusters of tested objects with similar history, multiple status monitoring components are set on the inspected object, wherein the status monitoring components at least include a vibration monitoring device, and the status information of the inspected object at least includes vibration information of the vibration monitoring position collected by the vibration monitoring device.
[0034] It is understandable that patients may experience tremors during endoscopy (such as gastroscopy, colonoscopy, etc.), but not all patients will experience this. The causes of tremors may involve multiple aspects:
[0035] Nervous emotions: Endoscopy itself may bring certain psychological pressure and discomfort to patients, causing them to be overly nervous. Nervous emotions can easily trigger the body's autonomic nervous system reactions, including muscle tension, accelerated heartbeat, etc., which may manifest as body shaking.
[0036] Discomfort: During the endoscopy, the patient may feel some discomfort, such as nausea, vomiting, abdominal pain, etc. These discomforts may also cause the patient's body to tremble.
[0037] Physiological reactions: In some cases, such as when the endoscopy is prolonged or the patient is hungry or fatigued, the body's physiological reactions may be more pronounced, including muscle tremors.
[0038] Drug reaction: If the patient uses certain drugs (such as anesthetics, sedatives, etc.) before or during the examination, these drugs may have a certain effect on the patient's nervous system, thereby causing side effects such as tremors.
[0039] Disease factors: Some patients may have some diseases or conditions, such as neurological diseases, hypoglycemia, hypocalcemia, etc. These diseases or conditions may also cause patients to have abnormal manifestations such as shaking during endoscopic examination.
[0040] For each historical tested subject, the position where the body of the historical tested subject shook during the endoscopic examination can be determined based on the state information of the historical tested subject during the endoscopic examination.
[0041] For each historical tested object cluster, the position where each historical tested object included in the historical tested object cluster vibrates during the endoscopic examination is used as a candidate vibration monitoring position. For each candidate vibration monitoring position, the proportion of the number of historical tested objects included in the historical tested object cluster who vibrate at the candidate vibration monitoring position is calculated, and the candidate vibration monitoring position whose proportion exceeds the proportion threshold is used as the vibration monitoring position corresponding to the historical tested object cluster.
[0042] For example, the historical test object cluster includes historical test object A, historical test object B, and historical test object C, and the positions where the body of historical test object A shakes during the endoscopic examination include position X1, position X2, and position X3, the positions where the body of historical test object B shakes during the endoscopic examination include position X2, position X3, and position X4, and the positions where the body of historical test object C shakes during the endoscopic examination include position X2, position X4, and position X5, then the candidate vibration monitoring positions corresponding to the historical test object cluster include position X1, position X2, position X3, position X4, and position X5, and the corresponding proportions are 1 / 3, 1, 2 / 3, 2 / 3, and 1 / 3, respectively. If the proportion threshold is set to 1 / 3, then the vibration monitoring positions corresponding to the historical test object cluster include position X2, position X3, and position X4.
[0043] The cosine similarity between the basic information of the inspected object and the basic information of the sampled historical tested objects in the historical tested object cluster can be calculated, and the historical tested object cluster to which the historical tested object with the largest cosine similarity belongs is taken as the similar historical tested object cluster.
[0044] Step 120, based on the state information of the inspected object, determine whether there is an abnormal time period, and if so, determine the abnormal time period.
[0045] Furthermore, step 120 specifically includes:
[0046] For each condition monitoring component, based on vibration information of the vibration monitoring position collected by the condition monitoring component, determine a first abnormality coefficient of the vibration monitoring position where the condition monitoring component is located;
[0047] Calculating a comprehensive abnormality coefficient based on a first abnormality coefficient of a vibration monitoring position where each condition monitoring component is located and an abnormality weight corresponding to each vibration monitoring position;
[0048] According to the comprehensive abnormality coefficient, it is judged whether there is an abnormal time period. When the comprehensive abnormality coefficient is greater than the comprehensive abnormality coefficient threshold, it is determined that there is an abnormal time period.
[0049] Specifically, the mean vibration amplitude, standard deviation of vibration amplitude, mean vibration frequency, standard deviation of vibration frequency, mean vibration acceleration, standard deviation of vibration acceleration, mean vibration velocity and standard deviation of vibration velocity can be calculated based on the vibration information of the vibration monitoring position collected by the condition monitoring component at multiple time points during the endoscopic inspection of the inspected object.
[0050] The first anomaly coefficient of the vibration monitoring location where the condition monitoring component is located can be calculated according to the following formula:
[0051]
[0052] D i =a 11 ×(μ (i,amplitude) -μ (0,amplitude) ) 2 +a 12 ×(σ (i,amplitude) -σ (0,amplitude) ) 2 +a 13 ×(μ (i,frequency) -μ (0,frequency) ) 2 +a 14 ×(σ (i,frequency) -σ (0,frequency) ) 2 +a 15 ×(μ (i,acceleration) -μ(0,acceleration) ) 2 +a 16 ×(σ (i,acceleration) -σ (0,acceleration) ) 2 +a 17 ×(μ (i,velocity) -μ (0,velocity) ) 2 +a 18 ×(σ (i,velocity) -σ (0,velocity) ) 2
[0053] Among them, P (i,1) is the first abnormal coefficient of the vibration monitoring position where the i-th condition monitoring component is located, ψ1 is a preset parameter, ψ1 is greater than 0, D i is the difference in the vibration monitoring position of the i-th condition monitoring component, a 11 、a 12 、a 13 、a 14 、a 15 、a 16 、a 17 and a 18 All are preset weights, a 11 、a 12 、a 13 、a 14 、a 15 、a 16 、a 17 and a 18 Both are greater than 0, a 11 +a 12 +a 13 +a 14 +a 15 +a 16 +a 17 +a 18 =1,μ (i,amplitude) is the mean vibration amplitude corresponding to the i-th condition monitoring component, μ (0,amplitude) is the preset vibration amplitude mean, σ (i,amplitude) is the standard deviation of the vibration amplitude corresponding to the i-th condition monitoring component, σ (0,amplitude) is the preset vibration amplitude standard deviation, μ (i,frequency) is the mean vibration frequency corresponding to the i-th condition monitoring component, μ (0,frequency) is the preset vibration frequency mean, σ (i,frequency) is the standard deviation of the vibration frequency corresponding to the i-th condition monitoring component, σ (0,frequency) is the preset vibration frequency standard deviation, μ (i,acceleration) is the mean value of the vibration acceleration corresponding to the i-th condition monitoring component, μ (0,acceleration) is the preset vibration acceleration mean, σ(i,acceleration) is the standard deviation of vibration acceleration corresponding to the i-th condition monitoring component, σ (0,acceleration) is the preset vibration acceleration standard deviation, μ (i,velocity) is the mean vibration velocity corresponding to the i-th condition monitoring component, μ (0,velocity) is the preset vibration velocity mean, σ (i,velocity) is the standard deviation of the vibration velocity corresponding to the i-th condition monitoring component, σ (0,velocity) is the preset vibration velocity standard deviation.
[0054] According to the abnormal weight corresponding to each vibration monitoring position, the first abnormal coefficient of each vibration monitoring position where the state monitoring component is located can be weighted and summed to obtain a comprehensive abnormal coefficient.
[0055] Further, based on the status information of the inspected object, determining the abnormal time period includes:
[0056] The process of performing endoscopic inspection on the inspected object is divided into a plurality of time periods based on the vibration information of the vibration monitoring position collected by each condition monitoring component through a segmented model;
[0057] For each time period, based on the vibration information of the vibration monitoring position where each state monitoring component is located collected in the time period, calculate the second abnormality coefficient of the time period corresponding to the vibration monitoring position where each state monitoring component is located and the correlation coefficient of the time period corresponding to any two vibration monitoring positions where the state monitoring components are located; determine the state abnormality parameter corresponding to the time period according to the second abnormality coefficient of the time period corresponding to the vibration monitoring position where each state monitoring component is located, the correlation coefficient of the time period corresponding to any two vibration monitoring positions where the state monitoring components are located and the abnormality weight corresponding to each vibration monitoring position;
[0058] Based on the state abnormality parameter corresponding to each time period, the abnormal time period is determined. For example, the time period in which the state abnormality parameter is greater than the state abnormality parameter threshold is taken as the abnormal time period.
[0059] Further, calculating the second abnormality coefficient of the vibration monitoring position corresponding to the time period of each condition monitoring component includes:
[0060] Performing variational modal decomposition on the vibration information of the vibration monitoring position where the condition monitoring component is located collected in the time period to generate multiple vibration components;
[0061] For each vibration component, extract component features of the vibration component (e.g., mean value of vibration amplitude, standard deviation of vibration amplitude, mean value of vibration frequency, standard deviation of vibration frequency, mean value of vibration acceleration, standard deviation of vibration acceleration, mean value of vibration velocity, standard deviation of vibration velocity, etc.), and calculate component anomalies of the vibration component based on the component features of the vibration component;
[0062] A second abnormality coefficient of a time period corresponding to a vibration monitoring position where the condition monitoring component is located is calculated according to the component abnormality value, the center frequency and the bandwidth of each vibration component.
[0063] Specifically, the method of calculating the component abnormality value of the vibration component based on the component characteristic of the vibration component is similar to the method of calculating the first abnormality coefficient of the vibration monitoring position where the condition monitoring component is located, and will not be repeated here.
[0064] The state information of each historical tested object included in the cluster of similar historical tested objects during the endoscopic examination can be subjected to variational modal decomposition to generate multiple vibration components, and the target center frequency corresponding to the cluster of similar historical tested objects can be determined according to the multiple vibration components corresponding to each historical tested object included in the cluster of similar historical tested objects through a frequency determination model. It can be understood that the closer to the target center frequency, the more important the information or characteristics of the signal contained in the vibration component. The frequency determination model can be a convolutional neural network model.
[0065] The second abnormality coefficient of the vibration monitoring position corresponding to the time period where the condition monitoring component is located can be calculated according to the following formula:
[0066]
[0067] Among them, P (i,t,2) is the second abnormal coefficient of the vibration monitoring position of the i-th condition monitoring component corresponding to the t-th time period, δ k is the importance parameter of the kth vibration component in the tth time period corresponding to the vibration monitoring position of the i-th condition monitoring component, K is the total number of vibration components in the t-th time period corresponding to the vibration monitoring position of the i-th condition monitoring component, P (i,t,k,2) is the component abnormal value of the kth vibration component corresponding to the tth time period at the vibration monitoring position where the i-th condition monitoring component is located, ψ2 is a preset parameter, ψ2 is greater than 0, f k is the center frequency of the kth vibration component at the vibration monitoring position of the i-th condition monitoring component corresponding to the t-th time period, f aim is the target center frequency corresponding to the cluster of tested objects with similar history, W k is the bandwidth of the kth vibration component in the tth time period corresponding to the vibration monitoring position where the ith condition monitoring component is located.
[0068] For the vibration monitoring positions where any two condition monitoring components are located, the Pearson correlation coefficient can be calculated based on the vibration information collected by the two condition monitoring components as the correlation coefficient of the vibration monitoring positions where the two condition monitoring components are located. A correlation coefficient sequence can be generated based on the correlation coefficient of the corresponding time period of the vibration monitoring positions where any two condition monitoring components are located.
[0069] The abnormal weight corresponding to each vibration monitoring position can be determined based on the state information of each historical test object included in the cluster of similar historical test objects during the endoscopic examination. The vibration monitoring position that is more likely to affect the endoscopic imaging quality has a larger abnormal weight.
[0070] The status anomaly parameter corresponding to the time period can be calculated according to the following formula:
[0071]
[0072] Among them, P t is the state abnormality parameter corresponding to the tth time period, ψ3 is the preset parameter, ψ3 is greater than 0, cos(S t , S0) is the correlation coefficient sequence S corresponding to the tth time period t The cosine similarity between the preset correlation coefficient sequence S0, b i is the abnormal weight corresponding to the vibration monitoring position where the ith condition monitoring component is located, and I is the total number of the ith condition monitoring components.
[0073] Step 130 , based on the abnormal time period and the time tag of each endoscopic image collected during the endoscopic examination of the object to be examined, a first image screening is performed to obtain a plurality of endoscopic images after the first screening.
[0074] Specifically, the endoscopic images with time tags in an abnormal time period may be screened out, and the remaining endoscopic images with time tags not in an abnormal time period may be used as the endoscopic images after the first screening.
[0075] Step 140, during the process of performing endoscopic inspection on the object to be inspected, obtaining the moving speed of the endoscope.
[0076] Specifically include:
[0077] A magnetic induction array is arranged on the object to be inspected, and a permanent magnet is arranged on the endoscope, wherein the magnetic induction array includes a plurality of magnetic sensors;
[0078] The moving speed of the endoscope is determined by using the magnetic field strength information collected by multiple magnetic sensors during the endoscopic examination of the object to be inspected.
[0079] Specifically, the speed determination model can be used to determine the speed of the endoscope at multiple time points during the endoscopic examination of the object to be inspected based on the magnetic field strength information collected by multiple magnetic sensors during the endoscopic examination of the object to be inspected. The speed determination model can be a convolutional neural network model.
[0080] Step 150 performs a second screening on the plurality of endoscopic images after the first screening according to the moving speed of the endoscope to obtain a plurality of endoscopic images after the second screening.
[0081] Specifically include:
[0082] dividing the plurality of endoscopic images after the first screening into a plurality of endoscopic image groups according to the moving speed of the endoscope;
[0083] For each endoscopic image group, a differential image corresponding to two adjacent endoscopic images is generated, and based on the differential image corresponding to the two adjacent endoscopic images, a second screening is performed on multiple endoscopic images after the first screening included in the endoscopic image group to obtain an endoscopic image after the second screening.
[0084] Specifically, the plurality of endoscopic images after the first screening may be divided into a plurality of endoscopic image groups based on the speed of the endoscope at a plurality of time points during the endoscopic examination of the object to be examined by an image grouping model, wherein the image grouping model may be a convolutional neural network model.
[0085] The pixels of the difference image are compared with the threshold to generate a binary image, where pixels greater than the threshold are marked as 1 (indicating significant changes) and pixels less than or equal to the threshold are marked as 0 (indicating no significant changes). If the difference between two adjacent endoscopic images is small (i.e., there are few or no significant change areas in the difference image), one of the images can be discarded to avoid redundancy.
[0086] Step 160 , extracting image features of each endoscopic image after the second screening.
[0087] Specifically include:
[0088] The image features of each endoscopic image after the second screening are extracted through an image feature extraction model, wherein the image feature extraction model can be a convolutional neural network model, and the image features can at least include color features, brightness features, texture features, shape and edge features, etc.
[0089] Step 170 , performing a third screening on the plurality of second-screened endoscopic images according to the image features of each second-screened endoscopic image, to obtain a plurality of screened endoscopic images.
[0090] Specifically include:
[0091] generating standard image features of endoscopic images of a plurality of endoscopic detection positions of the inspected object based on basic information of the inspected object;
[0092] determining the possibility of a lesion in each endoscopic image after the second screening according to image features of each endoscopic image after the second screening and standard image features of endoscopic images of a plurality of endoscopic detection positions of the inspected object;
[0093] Based on the lesion possibility of each endoscopic image after the second screening, a plurality of endoscopic images after the second screening are screened for a third time to obtain a plurality of screened endoscopic images.
[0094] Specifically, based on the basic information of the inspected object, the lesions of the inspected object that may be detected by the endoscope can be predicted, and based on the predicted lesions of the inspected object that may be detected by the endoscope, the standard image features of the endoscopic images of multiple endoscopic detection positions of the inspected object can be generated through a feature generation model, wherein the feature generation model can be a generative adversarial network model.
[0095] The lesion prediction model is used to determine the lesion possibility of each endoscopic image after the second screening based on the image features of each endoscopic image after the second screening and the standard image features of the endoscopic images of multiple endoscopic detection positions of the inspected object, wherein the lesion prediction model can be a convolutional neural network model.
[0096] The endoscopic images after the second screening whose lesion likelihood is less than the lesion likelihood threshold are screened out, and the remaining endoscopic images after the second screening whose lesion likelihood is greater than or equal to the lesion likelihood threshold are used as the screened endoscopic images.
[0097] Figure 2 is a module diagram of an endoscopic image screening system shown in one embodiment of the present application, such as Figure 2 As shown, an endoscope image screening system may include a status monitoring module, an abnormality determination module and an image screening module.
[0098] The state monitoring module may include a plurality of state monitoring components arranged on the inspected object, wherein the plurality of state monitoring components are used to collect state information of the inspected object during the process of performing endoscopic inspection on the inspected object.
[0099] The abnormality determination module can be used to determine whether there is an abnormal time period based on the state information of the inspected object, and if so, determine the abnormal time period.
[0100] The image screening module can be used to perform a first image screening based on the abnormal time period and the time tag of each endoscopic image collected during the endoscopic examination of the inspected object, and obtain multiple endoscopic images after the first screening.
[0101] The state monitoring module is also used to obtain the moving speed of the endoscope during the endoscopic examination of the object to be inspected;
[0102] The image screening module is also used to perform a second screening on multiple endoscopic images after the first screening according to the moving speed of the endoscope, so as to obtain multiple endoscopic images after the second screening; it is also used to extract image features of each endoscopic image after the second screening, and perform a third screening on multiple endoscopic images after the second screening according to the image features of each endoscopic image after the second screening, so as to obtain multiple screened endoscopic images.
[0103] An endoscopic image screening system can be used to execute an endoscopic image screening method. For more description of an endoscopic image screening system, please refer to the relevant description of an endoscopic image screening method, which will not be repeated here.
[0104] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.
Claims
1. An endoscopic image screening method, characterized in that: include: A plurality of state monitoring components are arranged on the inspected object, wherein the plurality of state monitoring components are used to collect state information of the inspected object during the process of performing endoscopic inspection on the inspected object; Based on the state information of the inspected object, determining whether there is an abnormal time period, and if so, determining the abnormal time period; Based on the abnormal time period and the time tag of each endoscopic image collected during the endoscopic examination of the object to be inspected, a first image screening is performed to obtain a plurality of endoscopic images after the first screening; During the process of performing endoscopic inspection on the inspected object, obtaining the moving speed of the endoscope; According to the moving speed of the endoscope, the plurality of endoscopic images after the first screening are screened for a second time to obtain a plurality of endoscopic images after the second screening; extracting image features of each second screened endoscopic image; According to the image features of each endoscopic image after the second screening, a plurality of endoscopic images after the second screening are screened for a third time to obtain a plurality of screened endoscopic images.
2. The method for screening endoscopic images according to claim 1, characterized in that: Multiple condition monitoring components are set up on the inspected object, including: Obtain basic information of multiple historical test subjects and status information during the endoscopic examination process; Based on the basic information of the multiple historical tested objects, clustering the multiple historical tested objects to determine multiple historical tested object clusters; For each of the historical tested object clusters, based on state information of a plurality of historical tested objects included in the historical tested object cluster during the endoscopic examination, determining a vibration monitoring position corresponding to the historical tested object cluster; Acquire basic information of the inspected object, and determine a similar historical tested object cluster from the plurality of historical tested object clusters according to the basic information of the inspected object; According to the vibration monitoring positions corresponding to the cluster of similar historical tested objects, multiple status monitoring components are set on the inspected object, wherein the status monitoring components at least include a vibration monitoring device, and the status information of the inspected object at least includes vibration information of the vibration monitoring position collected by the vibration monitoring device.
3. The method for screening endoscopic images according to claim 1, characterized in that: Based on the state information of the inspected object, determining whether there is an abnormal time period includes: For each of the condition monitoring components, based on the vibration information of the vibration monitoring position collected by the condition monitoring component, a first abnormality coefficient of the vibration monitoring position where the condition monitoring component is located is determined; Calculating a comprehensive abnormality coefficient based on a first abnormality coefficient of a vibration monitoring position where each condition monitoring component is located and an abnormality weight corresponding to each vibration monitoring position; According to the comprehensive abnormality coefficient, it is determined whether there is an abnormal time period. When the comprehensive abnormality coefficient is greater than the comprehensive abnormality coefficient threshold, it is determined that there is an abnormal time period.
4. The method for screening endoscopic images according to claim 3, characterized in that: Determining an abnormal time period based on the state information of the inspected object includes: The process of performing endoscopic inspection on the inspected object is divided into a plurality of time periods based on the vibration information of the vibration monitoring position collected by each of the condition monitoring components by using a segmented model; For each time period, based on the vibration information of the vibration monitoring position where each state monitoring component is located collected during the time period, calculate the second abnormality coefficient of the vibration monitoring position where each state monitoring component is located corresponding to the time period and the correlation coefficient of the vibration monitoring positions where any two state monitoring components are located corresponding to the time period; determine the state abnormality parameter corresponding to the time period according to the second abnormality coefficient of the vibration monitoring position where each state monitoring component is located corresponding to the time period, the correlation coefficient of the vibration monitoring positions where any two state monitoring components are located corresponding to the time period and the abnormality weight corresponding to each vibration monitoring position; Based on the state abnormality parameter corresponding to each of the time periods, an abnormal time period is determined.
5. The method for screening endoscopic images according to claim 4, characterized in that: Calculating the second abnormality coefficient corresponding to the time period at the vibration monitoring position where each of the condition monitoring components is located comprises: Performing variational modal decomposition on the vibration information of the vibration monitoring position where the condition monitoring component is located collected during the time period to generate a plurality of vibration components; For each of the vibration components, extracting a component feature of the vibration component, and calculating a component abnormal value of the vibration component based on the component feature of the vibration component; A second abnormality coefficient corresponding to the time period at the vibration monitoring position where the condition monitoring component is located is calculated according to the component abnormality value, the center frequency and the bandwidth of each of the vibration components.
6. An endoscopic image screening method according to any one of claims 1 to 5, characterized in that: In the process of performing endoscopic inspection on the inspected object, obtaining the moving speed of the endoscope includes: A magnetic induction array is arranged on the object to be inspected, and a permanent magnet is arranged on the endoscope, wherein the magnetic induction array includes a plurality of magnetic sensors; The moving speed of the endoscope is determined by using the magnetic field strength information collected by the multiple magnetic sensors during the endoscopic inspection of the inspected object.
7. An endoscopic image screening method according to any one of claims 1 to 5, characterized in that: According to the moving speed of the endoscope, the plurality of endoscopic images after the first screening are screened for a second time to obtain a plurality of endoscopic images after the second screening, including: dividing the plurality of endoscopic images after the first screening into a plurality of endoscopic image groups according to the moving speed of the endoscope; For each of the endoscopic image groups, a differential image corresponding to two adjacent endoscopic images is generated, and based on the differential image corresponding to the two adjacent endoscopic images, a second screening is performed on the multiple endoscopic images after the first screening included in the endoscopic image group to obtain the endoscopic images after the second screening.
8. An endoscopic image screening method according to any one of claims 1 to 5, characterized in that: Extract image features of each endoscopic image after the second screening, including: The image features of each endoscopic image after the second screening are extracted through the image feature extraction model.
9. An endoscopic image screening method according to any one of claims 2 to 5, characterized in that: According to the image features of each endoscopic image after the second screening, a plurality of endoscopic images after the second screening are screened for a third time to obtain a plurality of screened endoscopic images, including: generating standard image features of endoscopic images of a plurality of endoscopic detection positions of the inspected object based on basic information of the inspected object; Determining the possibility of a lesion in each endoscopic image after the second screening according to the image features of each endoscopic image after the second screening and the standard image features of the endoscopic images of the inspected object at multiple endoscopic detection positions; Based on the lesion possibility of each endoscopic image after the second screening, a plurality of endoscopic images after the second screening are screened for a third time to obtain the plurality of screened endoscopic images.
10. An endoscopic image screening system, characterized in that: A method for performing an endoscopic image screening method according to any one of claims 1 to 9, comprising: A state monitoring module, comprising a plurality of state monitoring components arranged on the inspected object, wherein the plurality of state monitoring components are used to collect state information of the inspected object during an endoscopic inspection of the inspected object; an abnormality determination module, used to determine whether there is an abnormal time period based on the state information of the inspected object, and if so, determine the abnormal time period; An image screening module, configured to perform a first image screening based on the abnormal time period and a time tag of each endoscopic image collected during the endoscopic examination of the inspected object, and obtain a plurality of endoscopic images after the first screening; The state monitoring module is also used to obtain the moving speed of the endoscope during the process of performing endoscopic inspection on the inspected object; The image screening module is also used to perform a second screening on the multiple endoscopic images after the first screening according to the moving speed of the endoscope, so as to obtain multiple endoscopic images after the second screening; and is also used to extract image features of each endoscopic image after the second screening, and perform a third screening on the multiple endoscopic images after the second screening according to the image features of each endoscopic image after the second screening, so as to obtain multiple screened endoscopic images.