Method and apparatus for image processing

Abstract

According to the embodiment of the invention, a method and equipment for image processing are provided. The method includes monitoring respective predictions for images in a first sequence of images. The first image sequence is acquired by the endoscope over time during examination, and the prediction result indicates whether the endoscope is located in the body of the examined object when the corresponding image is acquired. The method further includes determining, from the first sequence of images, a reference image corresponding to entry of the endoscope into the body of the subject under examination based on the monitoring. The method further includes acquiring a second sequence of images acquired by the endoscope after the acquisition time of the reference image. The method further includes determining, based on the second sequence of images, a target site of the examined object examined by the endoscope. According to the scheme, the target part can be automatically identified without depending on external input, so that the accuracy and rapidity of the preorder work of endoscopy are ensured.

Description

Technical Field

[0001] Example embodiments of the present disclosure relate generally to the field of computers, and more particularly to methods and apparatus for image processing. Background Art

[0002] Image processing technology has been widely used in various fields. In the medical field, the demand for processing and analyzing large amounts of medical image data is also growing. Especially in the field of medical endoscope images, a solution is urgently needed to help doctors make diagnoses more accurately and quickly. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for image processing is provided. The method comprises: monitoring corresponding prediction results for images in a first image sequence, the first image sequence being acquired over time by an endoscope during an inspection, and the prediction results indicating whether the endoscope is located inside the body of an inspected object when the corresponding image is acquired; based on the monitoring, determining a reference image corresponding to the endoscope entering the body of the inspected object from the first image sequence; acquiring a second image sequence acquired by the endoscope after the acquisition time of the reference image; and determining a target part of the inspected object inspected by the endoscope based on the second image sequence.

[0004] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing circuit. The at least one processing circuit is configured to: monitor corresponding prediction results for images in a first image sequence, the first image sequence is acquired by an endoscope over time during an inspection, and the prediction results indicate whether the endoscope is located in the body of the inspected object when the corresponding image is acquired; based on the monitoring, determine a reference image corresponding to the endoscope entering the body of the inspected object from the first image sequence; obtain a second image sequence acquired by the endoscope after the acquisition time of the reference image; and determine a target part of the inspected object to be inspected by the endoscope based on the second image sequence.

[0005] In some embodiments of the second aspect, the first image sequence includes a first image, and the corresponding prediction result includes a first prediction result for the first image. The at least one processing circuit is further configured to: for the first image in the first image sequence, based on the trained first classification model, generate a first prediction result as to whether the endoscope is in the body of the inspected object when the first image is acquired.

[0006] In some embodiments of the second aspect, the first image sequence includes a second image, and the corresponding prediction result includes a second prediction result for the second image. The at least one processing circuit is further configured to: for the second image in the first image sequence, extract an image feature of the second image; determine a first similarity between the image feature and an in vivo reference feature, and a second similarity between the image feature and an in vitro reference feature, wherein the in vivo reference feature represents the body of the reference object, and the in vitro reference feature represents the body of the reference object; and generate a second prediction result about whether the endoscope is located in the body of the inspected object when the second image is acquired by comparing the first similarity and the second similarity.

[0007] In some embodiments of the second aspect, at least one processing circuit is further configured to: determine a first number of adjacent images in a first image sequence before determining a reference image based on monitoring; and set a prediction result for an intermediate image in the first number of adjacent images based on the corresponding prediction results of the first number of adjacent images.

[0008] In some embodiments of the second aspect, at least one processing circuit is further configured to: determine, from a first number of adjacent images, an image whose corresponding prediction result indicates that the endoscope is located inside the body of the inspected object; if the number of images determined is greater than a first threshold number related to the first number, set the prediction result for the intermediate image to indicate that the endoscope is inside the body of the inspected object; and if the number of images determined is not greater than the first threshold number, set the prediction result for the intermediate image to indicate that the endoscope is not inside the body of the inspected object.

[0009] In some embodiments of the second aspect, at least one processing circuit is further configured to: determine whether corresponding prediction results of a second number of consecutive images in the first image sequence all indicate that the endoscope is located within the body of the inspected object; and in response to determining that corresponding prediction results of the second number of consecutive images all indicate that the endoscope is located within the body of the inspected object, determine a reference image based on the second number of consecutive images.

[0010] In some embodiments of the second aspect, the acquisition time span corresponding to the second number of consecutive images is related to the image acquisition frequency of the endoscope.

[0011] In some embodiments of the second aspect, at least one processing circuit is further configured to: monitor whether the image quality of individual images acquired by the endoscope after the acquisition time of the reference image exceeds a threshold quality; and in response to monitoring that the image quality of the third image exceeds the threshold quality, use the images acquired sequentially by the endoscope within a period of time after the acquisition time of the third image as a second image sequence.

[0012] In some embodiments of the second aspect, the second image sequence satisfies at least one of the following: an acquisition time span corresponding to the second image sequence is equal to a threshold span, or an accumulation of image distances between adjacent images in the second image sequence is greater than a threshold, wherein the image distance is determined based on optical flow.

[0013] In some embodiments of the second aspect, at least one processing circuit is further configured to: for a third image in the second image sequence, generate a first recognition result as to which part of the plurality of predetermined parts the third image corresponds to based on the trained second classification model.

[0014] In some embodiments of the second aspect, at least one processing circuit is further configured to: extract image features of a fourth image in the second image sequence; determine corresponding similarities between the image features and multiple reference features, where the multiple reference features are respectively used to characterize multiple predetermined parts; and generate a second recognition result as to which part of the multiple predetermined parts the fourth image corresponds to by comparing the corresponding similarities.

[0015] In some embodiments of the second aspect, at least one processing circuit is further configured to: determine that the target part is the first part in response to the number of identification results indicating a first part among multiple predetermined parts in the corresponding identification results exceeding a second threshold number, or determine that the target part is the second part in response to the proportion of identification results indicating a second part among multiple predetermined parts in the corresponding identification results exceeding a threshold proportion.

[0016] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0017] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0018] It should be understood that the contents described in the summary of the present invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0020] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0021] Figure 2 A schematic diagram showing an example architecture of image processing according to some embodiments of the present disclosure;

[0022] Figure 3 A schematic diagram showing an example of stomach or intestine determination according to some embodiments of the present disclosure;

[0023] Figure 4 A flowchart showing a process of image processing according to some embodiments of the present disclosure; and

[0024] Figure 5 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0026] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0027] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0028] The term "circuit" as used herein may refer to a combination of hardware circuits and / or hardware circuits and software. For example, a circuit may be a combination of analog and / or digital hardware circuits and software / firmware. As another example, a circuit may be any part of a hardware processor with software, the hardware processor including (multiple) digital signal processors, software and (multiple) memories, which work together to enable the device to work to perform various functions. In yet another example, a circuit may be a hardware circuit and / or processor, such as a microprocessor or a part of a microprocessor, which requires software / firmware for operation, but the software may not exist when it is not needed for operation. As used herein, the term "circuit" also encompasses the implementation of only a hardware circuit or processor or a part of a hardware circuit or processor and its (or their) accompanying software and / or firmware.

[0029] As briefly mentioned above, there is a growing demand in the medical field for processing and analyzing large amounts of medical image data. Traditionally, doctors need to rely on their own experience and visual judgment to analyze endoscopic images and perform corresponding examination items or make corresponding diagnoses. However, this method has problems such as subjectivity, time-consuming, and possible misjudgment. Therefore, with the development of artificial intelligence (AI), it has been applied to the analysis of endoscopic images. Generally speaking, different AI algorithms or models need to be developed and used for different parts of the object being inspected (e.g., the human body). Therefore, correctly judging the part being inspected is crucial for subsequent image analysis.

[0030] For example, endoscopes are often used in gastrointestinal examinations. In gastrointestinal examinations, the same set of endoscope equipment is usually used to perform gastroscopy and colonoscopy by switching different lenses. The entrance from the outside of the patient's body to the patient's body for gastroscopy is the "mouth", while the entrance from the outside of the patient's body to the patient's body for colonoscopy is the "anus". During a day's examination, depending on the condition of the person being examined, the inspector switches the gastroscope lens and the colonoscopy lens irregularly, and calls the corresponding inspection program after switching the lens. How to accurately and quickly judge the current inspection site is an essential task. In particular, in the case of AI-assisted diagnosis, automatically distinguishing between gastroscopy and colonoscopy is a prerequisite for subsequent AI tasks.

[0031] Traditionally, some solutions focus on identifying parts or organs and cannot be directly applied to the recognition scenarios of the stomach or intestines. Some solutions identify whether it is a certain part or organ for each frame, and cannot complete the recognition task in a fixed time period (such as the first period of time after the endoscope enters the patient's body). In addition, some solutions lack comprehensive judgment of multiple frames, so they are not suitable for the recognition scenarios of the stomach or intestines. Such scenarios require unified judgment of several frames collected by the endoscope entering the patient's body.

[0032] The above describes several image processing problems using the use of endoscopes in human digestive tract examinations as an example. It should be understood that similar image processing problems may also exist in endoscopic examinations of other objects or parts.

[0033] To this end, an embodiment of the present disclosure proposes a scheme for image processing. The scheme includes monitoring corresponding prediction results for images in a first image sequence. Such a first image sequence is collected by an endoscope over time during an inspection. The prediction result indicates whether the endoscope is located inside the body of the inspected object when the corresponding image is collected. Based on the monitoring of the prediction result, a reference image corresponding to the endoscope entering the body of the inspected object is determined from the first image sequence. Further, a second image sequence collected by the endoscope after the collection time of the reference image is acquired. Based on the acquired second image sequence, the target part of the inspected object to be inspected by the endoscope is determined.

[0034] Thus, the corresponding parts or inspection items can be automatically determined through the collected images. In this way, the accuracy and speed of the preliminary work of endoscopic inspection can be guaranteed without relying on additional external input (for example, text, voice or key input, etc.). In particular, for AI-assisted endoscopic inspection, accurate and rapid judgment of the inspected parts is conducive to the accurate and timely application of subsequent AI tasks.

[0035] Example Environment

[0036] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the environment 100, an electronic device 120 can receive an image sequence 110 and output a determination result. The image sequence 110 includes a group of images 112 sorted by acquisition time. The images 112 are, for example, images related to an inspected object acquired by a medical endoscope. The images 112 can be acquired in real time or offline. The determination result can be, for example, a site inspected by an endoscope, also referred to as a target site. For example, in a scenario of a digestive tract examination, the determination result can indicate a colonoscopy or a gastroscopy.

[0037] In the embodiments of the present disclosure, the inspected object may be any suitable type of object, such as a human body, an animal, etc. An endoscope may be used for various suitable inspections, such as a digestive tract inspection, a throat inspection, etc. In the following, some examples or embodiments use the distinction between a colonoscope and a gastroscope in a human digestive tract inspection as an example. However, it should be understood that this is merely exemplary and is not intended to be limiting. In the embodiments of the present disclosure, the types of inspected objects, endoscopes, and parts of inspected objects are not limited.

[0038] In environment 100, electronic device 120 can be any type of device with computing capabilities, including terminal devices. The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. Electronic device 120 can be any type of device with computing capabilities, including server devices. Server devices can include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0039] In particular, in some embodiments, the electronic device 120 may be a computing device that communicates directly with the endoscope. In some embodiments, the electronic device 120 may be implemented in an endoscope, or the various actions described below with reference to the electronic device 120 may be implemented by the endoscope itself.

[0040] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. Figure 1 In the description and subsequent description, the electronic device 110 collects the image sequence 110 and determines the judgment result. However, this is only exemplary and is not intended to limit the scope of the present disclosure. In some embodiments, collecting the image sequence 110 and determining the judgment result can be implemented by different devices.

[0041] Hereinafter, example embodiments of the present disclosure will be described continuously with reference to the accompanying drawings.

[0042] Example Architecture

[0043] Figure 2 A schematic diagram of an example architecture of image processing according to some embodiments of the present disclosure is shown. The architecture 200 can be implemented at the electronic device 120. The architecture 200 generally includes a monitoring module 210, a reference image determination module 220, and a target site determination module 235. Additionally or alternatively, the architecture 200 may also include an endoscope configured to acquire an image sequence. Such an image sequence may include images acquired by the endoscope during the process of entering the body of the subject from outside the body of the subject being examined. Figure 1 Describe the architecture 200.

[0044] Continue to refer Figure 2, the monitoring module 210 receives the first image sequence 205. The first image sequence 205 may be a set of images collected by the endoscope over time during the inspection process. It is understandable that, during the inspection process, the images collected by the endoscope over time can be regarded as a video. Therefore, the images collected by the endoscope are also referred to as frames, that is, the first image sequence 205 may include multiple frames. The monitoring module 210 may monitor multiple frames and make predictions for each of them. Further, the monitoring module 210 may output a prediction result 215 for each frame. Such a prediction result 215 indicates whether the endoscope is located in the body of the inspected object when the corresponding frame is collected by the endoscope. That is, the prediction result 215 is a prediction for a single frame. For ease of description, the indication of whether the endoscope is located in the body of the inspected object will be referred to as in vivo below.

[0045] In some embodiments, such single-frame prediction may be performed using a deep learning approach. For example, the monitoring module 210 may generate a prediction result (also referred to as a first prediction result) regarding whether the endoscope is in the body when each image is acquired, using a trained classification model (also referred to as a first classification model) for each image (e.g., the first image) in the first image sequence 205.

[0046] Exemplarily, in order to train the classification model, in vivo and in vitro data of the same type of inspected object can be collected. For example, when the inspected object is a person and the inspected part is the stomach, the in vivo data includes a series of images of the mouth, throat, esophagus, stomach, etc. For another example, when the inspected object is a person and the inspected part is the intestine, the in vivo data includes a series of images of the buttocks, anus, intestine, etc. Then, the in vivo / in vitro data can be collected and labeled. Furthermore, the labeled data can be used to construct a binary deep learning model. It should be understood that the operations of collecting and labeling data and the operations of training models can be performed on devices other than the electronic device 120, and the present disclosure does not limit this.

[0047] The monitoring module 210 can use such a deep learning model (e.g., a binary classification model) to analyze the first image sequence 205 to predict whether the endoscope is in vivo or in vitro. For example, the monitoring module 210 uses such a deep learning model to obtain an in vivo prediction score. Based on a preset in vivo threshold score, if the prediction score exceeds the threshold score, the monitoring module 210 determines that the prediction result corresponding to the current frame is in vivo, otherwise it is in vitro.

[0048] Alternatively or additionally, in some embodiments, a feature matching method can be used to perform single-frame prediction. For example, for each image in the first image sequence 205 (e.g., the second image), the monitoring module 210 extracts image features of these images. The monitoring module 210 determines the similarity of the extracted image features with the in vivo reference features (also referred to as the first similarity), and the similarity with the in vitro reference features (also referred to as the second similarity). The in vivo reference features represent the in vivo of the reference object, while the in vitro reference features represent the in vitro of the reference object. In other words, the in vivo reference features are representations of the in vivo features of the reference object, while the in vitro reference features are representations of the in vitro features of the reference object. Further, the monitoring module 210 generates a prediction result (also referred to as the second prediction result) of whether the endoscope is in vivo by comparing these two similarities. Such a prediction result indicates whether the endoscope is located in the body of the object being inspected when the corresponding image is collected. For example, if for a certain image, the first similarity is greater than the second similarity, the prediction result can indicate in vivo, otherwise the prediction result can indicate in vitro.

[0049] Exemplarily, in order to perform feature matching, in vivo and in vitro data of the same type of inspected object can be collected, and image features can be extracted from these data. Feature operators include but are not limited to color histograms, texture histograms, deep network features, etc. Further, the electronic device 120 uses a clustering algorithm (such as a Kmeans algorithm) to spatially cluster in vivo features and in vitro features to construct a feature library. It should be understood that the operation of collecting data can be performed on other devices other than the electronic device 120, and the present disclosure does not limit this.

[0050] The monitoring module 210 can use such a feature library to calculate the feature distance ratio of the first image sequence 205 to obtain the in-vivo feature distance ratio. For example, based on a preset in-vivo threshold feature distance ratio, if the feature distance ratio of the current frame is greater than the threshold feature distance ratio, the monitoring module 210 determines that the current frame corresponds to the in-vivo, otherwise it corresponds to the in-vivo.

[0051] Continue to refer Figure 2 , the reference image determination module 220 receives the prediction result 215 and determines the reference image 225. Such a reference image 225 corresponds to the endoscope entering the body of the inspected object. That is, the reference image determination module 220 receives the prediction result 215 of each frame and determines which frame or frames indicate that the endoscope enters the body of the inspected object. It can be seen that the determination of the reference image is the basis for subsequent operations.

[0052] As an example, the first image sequence 205 includes 12 frames, which are frame 1, frame 2, frame 3, ... and frame 12 in chronological order. Assume that the prediction results corresponding to frames 1 to 5 are all in vitro, and the prediction result corresponding to frame 6 indicates in vivo. Ideally, it can be predicted that all frames after frame 6 indicate in vivo, so frame 6 can be determined as a reference image. However, in some cases, the single-frame prediction results described above may be affected by device noise, environmental noise or algorithm accuracy, so there is a possibility of misjudgment.

[0053] In some embodiments, the reference image determination module 220 may determine a predetermined number (also referred to as a first number) of adjacent images in the first image sequence 205 before determining the reference image 225. Such adjacent images include, for example, a predetermined number of consecutive images acquired before the image currently being judged, and a predetermined number of consecutive images acquired thereafter. The reference image determination module 220 may set the prediction results of the intermediate images in these images based on the corresponding prediction results of these images. For example, if the corresponding prediction results of these images all indicate being in vivo, the prediction results of the intermediate images in these images may be set to indicate being in vivo. In this way, the single-frame prediction results may be smoothed to achieve the purpose of denoising, eliminating the influence of algorithm accuracy, and the like, thereby improving the accuracy of reference image determination.

[0054] Continuing with the above example, assume that the first number is 5 frames. When determining whether the 6th frame is a reference image, the adjacent frames collected before the 6th frame are the 4th to 5th frames and the adjacent frames collected thereafter are the 7th to 8th frames. The single-frame prediction results from the 4th to 8th frames may all indicate in vivo, or the prediction results of some of the frames may indicate in vivo. The reference image determination module 220 can combine the prediction results corresponding to the 5 frames to set whether the 6th frame is in vitro or in vitro, thereby smoothing the single-frame prediction results to reduce the risk of misjudgment of the reference image.

[0055] In some embodiments, further, the reference image determination module 220 can determine, from a first number of adjacent images, images whose corresponding prediction results indicate that the endoscope is located inside the body of the inspected object, and determine the number of such images. If the number is greater than a threshold value associated with the first number (also referred to as the first threshold number), the prediction results of the intermediate images in these adjacent images are set to indicate that the endoscope is inside the body of the inspected object. If the number is not greater than the threshold, the prediction results of the intermediate images in these adjacent images are set to indicate that the endoscope is not inside the body of the inspected object. The first threshold number may be less than or equal to the first number. The method of smoothing the prediction results of whether the endoscope is inside / outside the body based on the prediction results of each frame in the continuous frames may also be referred to as smoothing or local smoothing. In this way, the accuracy of the reference image determination can be improved.

[0056] Continuing the above example, assume that the first number is 5 and its corresponding first threshold number is 3. When smoothing the single-frame prediction result of the 6th frame, the adjacent frames collected before the 6th frame are the 4th frame to the 5th frame and the adjacent frames collected after the 6th frame are the 7th frame to the 8th frame. If 3 or more of the prediction results corresponding to the 5 frames indicate that the body is inside the body, the prediction result of the 6th frame is set to be inside the body of the object under examination.

[0057] Further, after determining the prediction results (original or smoothed) of a single frame, the reference image determination module 220 determines a reference image based on these prediction results.

[0058] However, in some cases, if the frame where the in-vivo prediction result appears for the first time is determined as the reference image, it may lead to misjudgment of the reference image. For example, the prediction results of the 3rd, 9th, 10th, and 11th frames from the 1st to the 11th frames indicate that they are in-vivo. Considering that the 3rd frame may be affected by noise, the endoscope actually enters the body of the inspected object from the 9th frame. In this case, there is a certain deviation if the 3rd frame is used as the reference image.

[0059] To this end, in some embodiments, when determining the reference image, a period of time can be set as a buffer. For example, the reference image determination module 220 can determine whether the corresponding prediction results of a predetermined number (also referred to as the second number) of continuous images in the first image sequence 205 all indicate that the endoscope is located inside the body of the inspected object. Furthermore, the reference image determination module 220 can determine the reference image based on the number of continuous images. For example, the reference image determination module 220 can determine the last frame of the continuous frames that meet the second number as the reference image. Thus, the effect of time buffering can be achieved. In this way, the accuracy of reference image determination can be further improved. Assume that the time buffer corresponds to K3 frames. If the prediction results of K3 consecutive frames all indicate outside the body, the last frame of these K3 consecutive frames can be determined as the reference image.

[0060] Continuing with the above example, assume that the second number is 3. When determining the reference image, if the prediction results of the 3rd frame, the 9th frame, the 10th frame, and the 11th frame indicate that they are in the body, then the prediction results of the continuous frames satisfying the second number (i.e., the 9th frame to the 11th frame) all indicate that they are outside the body. The reference image determination module 220 can continue to determine subsequent frames until the prediction results of the continuous frames satisfying the second number (i.e., the 9th frame to the 11th frame) all indicate that they are in the body, and the 9th frame is determined as the reference image.

[0061] The second number may be preset as a fixed value or correspond to a preset acquisition time span, such as 3 frames or the number of frames corresponding to 1 second. In some embodiments, the acquisition time span corresponding to the second number of continuous images is related to the image acquisition frequency of the endoscope. For example, when the acquisition frequency is 60fps, the acquisition time span is 1 second. For another example, when the acquisition frequency is 15fps, the acquisition time span is 4 seconds.

[0062] Continue to refer Figure 2 , the acquisition time of the reference image 225 can be used to determine the start of timing. For example, the acquisition time can be used as the start time of timing, and the image sequence acquired subsequently (also referred to as the second image sequence 230) can be sent to the target part determination module 235. The target part determination module 235 determines the inspection part (also referred to as the target part) of the object to be inspected by the endoscope based on such an image sequence. That is, the timing starts from the acquisition time determined based on the first image sequence 205, and the inspection part is determined based on the second image sequence 230 acquired subsequently. In this way, the influence of noise, algorithm accuracy, etc. can be reduced as much as possible, and an effective image sequence can be acquired as early as possible.

[0063] In some embodiments, the acquisition time of the reference image may not be used as the start of timing. In order to obtain a high-quality second image sequence 230, it is possible to monitor whether the image quality of individual images acquired after the acquisition time of the reference image 225 exceeds a threshold quality. Further, the acquisition time of the image (also referred to as the third image) whose image quality exceeds the threshold quality can be used as the start of timing, and the images sequentially acquired by the endoscope within a period of time after the acquisition time are used as the second image sequence 230. In such an embodiment, the image quality can be determined in any suitable manner, such as but not limited to a supervised deep learning model, grayscale variance product, etc.

[0064] The end point of the second image sequence 230, or the timing of the end of the timing, can be determined in various suitable ways. In some embodiments, the second image sequence 230 satisfies: the corresponding acquisition time span is equal to the threshold span. Such a threshold span can be a preset fixed value, such as 5 seconds. That is, the timing starts from the effective time when the endoscope enters the body and ends after 5 seconds. The second image sequence 230 includes all frames acquired during this period. In this way, it can be ensured that a valid image sequence is acquired in the shortest possible time. In this way, the inspection site can be determined based on a small number of valid images, so as not to affect the execution of subsequent tasks.

[0065] Alternatively or additionally, in some embodiments, the acquisition of the second image sequence 230 may adopt adaptive timing. The second image sequence 230 satisfies: wherein the accumulation of image distances between adjacent images is greater than a threshold. Such image distances are determined based on optical flow. Thus, it can be ensured that a stable image sequence is acquired. In this way, the effectiveness of the acquisition of the second image sequence can be improved.

[0066] For example, after the timing starts, for each frame collected, the optical flow field of a single frame (e.g., a sparse optical flow field) can be extracted, and the distance between the two frames can be calculated. Additionally or alternatively, if the image quality of a certain frame is lower than a threshold, the target part determination module 235 can remove the frame and continue to track the distance of subsequent frames. In this way, images that cannot be tracked due to low image quality can be removed. Further, these distances can be accumulated, for example, from the starting frame collected at the beginning of the timing to each frame collected subsequently, the image distance is accumulated. Such accumulated distance is compared with the threshold distance. If the accumulated distance is greater than the threshold distance, the timing is stopped. The second image sequence 230 includes all frames between the starting frame at the beginning of the timing and the last frame at the end of the timing.

[0067] Additionally or alternatively, the timing of the acquisition time of the second image sequence 230 may be stopped when any one of the conditions that the acquisition time span reaches a threshold span or the cumulative distance between images is greater than a threshold distance is met.

[0068] The above embodiment describes how to determine the acquisition time period of the second image sequence 230. The following further describes how the target part determination module 235 determines the target part after acquiring the second image sequence 230.

[0069] In some embodiments, for the images in the second image sequence 230, the target part determination module 235 can generate corresponding recognition results about the target part, that is, perform single-frame recognition. Such a recognition result indicates a predetermined part among multiple predetermined parts. Thus, the target part determination module 235 can determine the target part based on the corresponding recognition result. For example, the predetermined parts include the esophagus, the stomach, and the intestines. The target part determination module 235 can identify whether each frame in the second image sequence 230 includes features corresponding to these parts, and then determine which part of the object being inspected is currently being inspected based on the recognition results of these frames. In this way, the part of the endoscopic examination can be automatically determined.

[0070] In some embodiments, single-frame recognition of the target part can adopt a deep learning method. For example, for the image in the second image sequence 230 (for example, the third image), the target part determination module 235 can generate a recognition result (also called a first recognition result) based on a trained classification model (also called a second classification model). Such a recognition result can indicate which inspection part (also called a predetermined part) each frame in the second image sequence 230 corresponds to. For example, in order to obtain such a classification model, historical inspection data of these inspection parts can be collected and labeled. Then, the labeled historical inspection data can be used to construct a multi-classification (for example, binary classification) deep learning model. It should be understood that the operation of collecting and labeling data and the operation of training the model can be performed on other devices other than the electronic device 120, and the present disclosure does not limit this.

[0071] The target part determination module 235 can use such a deep learning model to analyze the second image sequence 230 to predict the current examination part. For example, the examination parts include the stomach and the intestine. The target part determination module 235 uses the deep learning model to obtain a prediction score. Based on a preset threshold score of the stomach. If the prediction score exceeds the threshold score corresponding to the stomach, the target part determination module 235 determines that the examination part corresponding to the current frame is the stomach, otherwise it is the intestine.

[0072] In some embodiments, the single-frame recognition of the target part can adopt a feature matching method. For example, for an image (e.g., the fourth image) in the second image sequence 230, the target part determination module 235 can extract its image features. Further, the target part determination module 235 determines the corresponding similarities between the extracted image features and multiple reference features. The multiple reference features are used to characterize multiple inspection parts respectively. The target part determination module 235 generates a recognition result (also referred to as a second recognition result) by comparing the corresponding similarities. Such a recognition result can indicate which inspection part all the frames in the second image sequence 230 correspond to.

[0073] Exemplarily, in order to obtain the reference features used, historical inspection data of these inspection parts can be collected, and image features can be extracted from these historical inspection data. Feature operators include but are not limited to color histograms, texture histograms, etc. Further, the electronic device 120 uses a clustering algorithm (such as a Kmeans algorithm) to perform spatial clustering on the features of different inspection parts to construct a feature library. It should be understood that the operation of collecting data can be performed on other devices other than the electronic device 120, and the present disclosure does not limit this.

[0074] The target part determination module 235 can use such a feature library to calculate the feature distance ratio of the second image sequence 230 to obtain the feature distance ratio. For example, the inspection part includes the stomach and the intestine. Based on the preset threshold feature distance ratio of the stomach, if the feature distance ratio of the current frame is greater than the threshold feature distance ratio, the target part determination module 235 determines that the inspection part corresponding to the current frame is the stomach, otherwise it is the intestine.

[0075] Continue to refer Figure 2 , the target part determination module 235 outputs a determination result about the target part based on the inspection part corresponding to each frame in the second image sequence 230. In some cases, due to the influence of noise or algorithm accuracy, these frames may correspond to different inspection parts respectively. In some embodiments, if the number of recognition results indicating the first part of the plurality of predetermined parts in the corresponding recognition results exceeds the second threshold number, the target part determination module 235 determines that the target part is the first part.

[0076] As an example, the inspection site includes, for example, the stomach and the intestine. Assume that the second image sequence includes a total of 10 frames. The second threshold number is, for example, 5 frames. If the recognition results corresponding to 8 frames in the second image sequence 230 indicate that the target site is the stomach. The number of these frames is greater than the second threshold number, then the target site determination module 235 determines that the target site is the stomach, otherwise it is the intestine.

[0077] In some embodiments, if the proportion of the recognition results indicating the second part of the plurality of predetermined parts in the corresponding recognition results exceeds the threshold proportion, the target part determination module 235 determines that the target part is the second part. Continuing with the above example, the threshold proportion is, for example, 0.7. If the recognition results corresponding to 8 frames out of 10 frames in the second image sequence 230 indicate that they are the stomach. The proportion of the number of these frames to the total number of frames is greater than the threshold proportion, then the target part determination module 235 determines that the target part is the stomach, otherwise it is the intestine.

[0078] The data processing method is described above through various embodiments of the present disclosure. The beneficial effects of these embodiments include that the identification of the endoscopic inspection site can be completed fully automatically without other prompts or physician operations. In addition, such a scheme is not limited to a certain device, and only a segment of image sequence needs to be collected to determine the inspection site. Such an image sequence can be collected online in real time or offline. Furthermore, such a scheme can quickly determine the inspection site when the endoscope enters the body of the inspected object without affecting the execution of subsequent tasks. It can be seen that the scheme can ensure the accuracy and speed of the preliminary work of the endoscopic inspection without relying on other methods (such as text, voice or key input, etc.).

[0079] Example implementation of digestive tract examination

[0080] In order to better understand the embodiments of the present disclosure, the following reference will be made to Figure 3 Describe a specific example.

[0081] Figure 3 A schematic diagram of an example 300 of determining the stomach or intestine according to some embodiments of the present disclosure is shown. Example 300 may be implemented at the electronic device 120. Assuming that the inspection site includes the stomach and the intestine, the corresponding inspection items are, for example, gastroscopy and colonoscopy. Figure 1 Give a description.

[0082] In block 305, the electronic device 120 acquires video frames associated with the object under inspection through the endoscope, such as frames acquired in real time after the endoscope is turned on.

[0083] At block 310 , the electronic device 120 performs a prediction on each frame in the video to determine whether the frame corresponds to the endoscope being located inside or outside the body of the subject being examined.

[0084] In block 315, the electronic device 120 determines whether the current frame corresponds to a situation where the endoscope is located inside the body of the inspected object. If the electronic device 120 determines that the current frame corresponds to the endoscope being located inside the body of the inspected object, the process proceeds to block 320. That is, the current frame is the reference image described above. If the electronic device 120 determines that the current frame corresponds to the endoscope being located outside the body of the inspected object, the process returns to block 310 and the electronic device 120 continues to determine the next frame.

[0085] In block 320, the electronic device 120 starts timing based on the determination that the endoscope is located in the body of the subject to be examined. Such timing is recorded from the acquisition time of the valid frame corresponding to the determination that the endoscope enters the body.

[0086] In block 325, the electronic device 120 acquires a target video frame associated with the inspection object, for example, through an endoscope. Such a target video frame includes a plurality of consecutive frames acquired after the start of timing.

[0087] In block 330 , as the target video frame is captured, that is, during the timing process, the electronic device 120 performs stomach / intestine prediction for each captured frame.

[0088] At block 335 , the electronic device 120 ends the timing and stops collecting the second video frame. The timing may be terminated in response to reaching a predetermined duration or the accumulated distance between adjacent frames exceeding a threshold.

[0089] In box 340, the electronic device 120 performs stomach / intestine prediction on all video frames. For example, the electronic device 120 combines the prediction results of each target frame to determine the prediction result of the stomach / intestine. For another example, the electronic device 120 performs stomach / intestine prediction on each target video frame, and then combines the prediction results of these frames to determine the prediction result of the stomach / intestine. The stomach / intestine prediction for all target video frames is performed, for example, to determine a prediction result based on the number or proportion of frames that meet the conditions in all target frames. Furthermore, the electronic device 120 determines whether the part currently detected by the endoscope is the stomach or the intestine based on the combined prediction results.

[0090] Through the above process, the electronic device 120 can use the endoscope to start taking images from the outside of the body of the inspected object. When the endoscope enters the body of the inspected object from the mouth / anus, the electronic device 120 first completes the prediction of the inside / outside the body, and then starts timing. Then, the electronic device 120 can use the endoscope to continue taking pictures inside the body of the inspected object. After the timing is over, the electronic device 120 completes the determination of the stomach / intestine based on the recognition results of the several images collected. In this way, the identification of the target part can be automatically completed without relying on other methods, thereby ensuring the accuracy and speed of the preliminary work of the endoscopic examination.

[0091] Example Process

[0092] Figure 4 4 is a flowchart of a process 400 for image processing according to some embodiments of the present disclosure. The process 400 may be implemented at the electronic device 120. For ease of discussion, reference is made to Figure 1 4. The process 400 is described below.

[0093] At block 410, the electronic device 120 monitors corresponding prediction results for images in a first sequence of images acquired over time by an endoscope during an examination, and the prediction results indicate whether the endoscope is located within the body of the subject being examined when the corresponding images are acquired.

[0094] At block 420 , the electronic device 120 determines, based on the monitoring, from the first image sequence a reference image corresponding to the endoscope entering the body of the subject being examined.

[0095] At block 430 , the electronic device 120 acquires a second sequence of images acquired by the endoscope after the acquisition time of the reference image.

[0096] In block 440 , the electronic device 120 determines a target site of the object to be examined by the endoscope based on the second image sequence.

[0097] In some embodiments, the first image sequence includes a first image, the corresponding prediction result includes a first prediction result for the first image, and the process 400 also includes: the electronic device 120 generates a first prediction result for the first image in the first image sequence, based on a trained first classification model, as to whether the endoscope is inside the body of the inspected object when the first image is captured.

[0098] In some embodiments, the first image sequence includes a second image, and the corresponding prediction result includes a second prediction result for the second image. The process 400 also includes: the electronic device 120 extracts image features of the second image for the second image in the first image sequence; determines a first similarity between the image features and an in vivo reference feature, and a second similarity between the image features and an in vitro reference feature, the in vivo reference feature representing the body of a reference object, and the in vitro reference feature representing the body of the reference object; and generates a second prediction result as to whether the endoscope is located inside the body of the inspected object when the second image is captured by comparing the first similarity and the second similarity.

[0099] In some embodiments, process 400 also includes: the electronic device 120 determines a first number of adjacent images in the first image sequence before determining a reference image based on monitoring; and based on the corresponding prediction results of the first number of adjacent images, sets the prediction results for the intermediate images in the first number of adjacent images.

[0100] In some embodiments, process 400 also includes: the electronic device 120 determines, from a first number of adjacent images, an image whose corresponding prediction result indicates that the endoscope is located within the body of the inspected object; if the number of determined images is greater than a first threshold number related to the first number, setting the prediction result for the intermediate image to indicate that the endoscope is within the body of the inspected object; and if the number of determined images is not greater than the first threshold number, setting the prediction result for the intermediate image to indicate that the endoscope is not within the body of the inspected object.

[0101] In some embodiments, process 400 also includes: the electronic device 120 determines whether corresponding prediction results of a second number of consecutive images in the first image sequence all indicate that the endoscope is located within the body of the inspected object; and in response to determining that corresponding prediction results of the second number of consecutive images all indicate that the endoscope is located within the body of the inspected object, determines a reference image based on the second number of consecutive images.

[0102] In some embodiments, the acquisition time span corresponding to the second number of consecutive images is related to the image acquisition frequency of the endoscope.

[0103] In some embodiments, process 400 also includes: the electronic device 120 monitors whether the image quality of individual images acquired by the endoscope after the acquisition time of the reference image exceeds a threshold quality; and in response to monitoring that the image quality of the third image exceeds the threshold quality, the images acquired sequentially by the endoscope within a period of time after the acquisition time of the third image are taken as a second image sequence.

[0104] In some embodiments, the second image sequence satisfies at least one of the following: the acquisition time span corresponding to the second image sequence is equal to the threshold span, or the accumulation of image distances between adjacent images in the second image sequence is greater than a threshold, wherein the image distance is determined based on optical flow.

[0105] In some embodiments, the process 400 further includes: the electronic device 120 generating a corresponding recognition result about the target site for the images in the second image sequence, the recognition result indicating one of a plurality of predetermined sites; and determining the target site based on the corresponding recognition result.

[0106] In some embodiments, the process 400 further includes: the electronic device 120 generates, for a third image in the second image sequence, a first recognition result regarding which part of the plurality of predetermined parts the third image corresponds to based on the trained second classification model.

[0107] In some embodiments, process 400 also includes: the electronic device 120 extracts image features of a fourth image in the second image sequence; determines corresponding similarities between the image features and multiple reference features, where the multiple reference features are respectively used to characterize multiple predetermined parts; and generates a second recognition result as to which part of the multiple predetermined parts the fourth image corresponds to by comparing the corresponding similarities.

[0108] In some embodiments, process 400 also includes: the electronic device 120 determines that the target part is the first part in response to the number of recognition results indicating a first part among multiple predetermined parts in the corresponding recognition results exceeding a second threshold number, or determines that the target part is the second part in response to the proportion of recognition results indicating a second part among multiple predetermined parts in the corresponding recognition results exceeding a threshold proportion.

[0109] Example Device

[0110] Figure 5 A block diagram of an electronic device 500 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to implement Figure 1An electronic device 120.

[0111] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0112] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 500.

[0113] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0114] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0115] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0116] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0117] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0118] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0119] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0120] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0121] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. An image processing method, comprising: monitoring corresponding prediction results for images in a first sequence of images acquired over time by an endoscope during an examination, the prediction results indicating whether the endoscope was located within the body of the subject under examination when the corresponding images were acquired; Based on the monitoring, determining a reference image corresponding to the endoscope entering the body of the inspected object from the first image sequence; Acquiring a second image sequence acquired by the endoscope after the acquisition time of the reference image; as well as Based on the second image sequence, a target site of the object to be examined by the endoscope is determined.

2. The method according to claim 1, wherein the first image sequence comprises a first image, the corresponding prediction result comprises a first prediction result for the first image, the method further comprising: Based on the trained first classification model, a first prediction result is generated as to whether the endoscope is inside the body of the examined object when the first image is acquired.

3. The method according to claim 1, wherein the first image sequence includes a second image, the corresponding prediction result includes a second prediction result for the second image, and the method further comprises: extracting image features of the second image; Determining a first similarity between the image feature and an in vivo reference feature and a second similarity between the image feature and an in vitro reference feature, the in vivo reference feature representing an in vivo portion of a reference object and the in vitro reference feature representing an in vitro portion of a reference object; as well as By comparing the first similarity and the second similarity, a second prediction result is generated as to whether the endoscope is located inside the body of the subject when the second image is acquired.

4. The method according to claim 1, further comprising: Before determining the reference image based on the monitoring, determining a first number of adjacent images in the first sequence of images; as well as A prediction result for an intermediate image in the first number of adjacent images is set based on the corresponding prediction results of the first number of adjacent images.

5. The method according to claim 4, wherein setting the prediction result for the intermediate images in the first number of adjacent images comprises: determining, from the first number of adjacent images, images for which corresponding prediction results indicate that the endoscope is located within the body of the inspected object; If the determined number of images is greater than a first threshold number associated with the first number, setting the prediction result for the intermediate image to indicate that the endoscope is within the body of the inspected object; as well as If the determined number of images is not greater than the first threshold number, the prediction result for the intermediate image is set to indicate that the endoscope is not within the body of the detected object.

6. The method according to claim 1, wherein determining, from the first image sequence, a reference image corresponding to the endoscope entering the body of the inspected object comprises: determining whether corresponding prediction results of a second number of consecutive images in the first image sequence all indicate that the endoscope is located within the body of the examined object; as well as In response to determining that corresponding prediction results of the second number of consecutive images all indicate that the endoscope is located within the body of the inspected object, the reference image is determined based on the second number of consecutive images. 7 . The method according to claim 6 , wherein the acquisition time span corresponding to the second number of consecutive images is related to the image acquisition frequency of the endoscope.

8. The method according to claim 1, wherein acquiring a second image sequence acquired by the endoscope after the acquisition time of the reference image comprises: monitoring whether an image quality of an individual image acquired by the endoscope after an acquisition time of the reference image exceeds a threshold quality; as well as In response to monitoring that the image quality of the third image exceeds the threshold quality, images sequentially acquired by the endoscope within a period of time after the acquisition time of the third image are used as the second image sequence.

9. The method according to claim 1, wherein the second image sequence satisfies at least one of the following: The acquisition time span corresponding to the second image sequence is equal to the threshold span, or An accumulation of image distances between adjacent images in the second image sequence is greater than a threshold, wherein the image distances are determined based on optical flow.

10. The method according to claim 1, wherein determining a target site of the object to be examined by the endoscope comprises: generating a corresponding recognition result about the target part for the image in the second image sequence, the recognition result indicating one of a plurality of predetermined parts; as well as The target part is determined based on the corresponding recognition result.

11. The method according to claim 10, wherein generating corresponding recognition results about the target part for the images in the second image sequence comprises: For a third image in the second image sequence, based on the trained second classification model, a first recognition result is generated as to which part of the plurality of predetermined parts the third image corresponds to.

12. The method according to claim 10, wherein generating corresponding recognition results about the target part for the images in the second image sequence comprises: For a fourth image in the second image sequence, extracting image features of the fourth image; determining respective similarities between the image feature and a plurality of reference features, the plurality of reference features being respectively used to characterize the plurality of predetermined parts; and By comparing the corresponding similarities, a second recognition result is generated as to which part of the plurality of predetermined parts the fourth image corresponds to.

13. The method according to claim 1, wherein determining the target site based on the corresponding recognition result comprises one of the following: In response to the number of recognition results indicating a first part of the plurality of predetermined parts in the corresponding recognition results exceeding a second threshold number, determining that the target part is the first part, or In response to a ratio of the corresponding recognition results indicating a second part among the plurality of predetermined parts exceeding a threshold ratio, the target part is determined to be the second part.

14. An electronic device comprising: At least one processing circuit configured to perform the method according to any one of claims 1 to 13.

Images