Method for image processing and electronic apparatus

Abstract

To provide a method for image processing and an apparatus.SOLUTION: A process 400 comprises: Step 410 for monitoring a prediction result indicating whether an endoscope is located in the body of a test subject at a time point when a corresponding image is collected, the prediction result being a result corresponding to an image in a first image sequence collected by the endoscope chronologically during a test; Step 420 for determining a reference image corresponding to entry of the endoscope to the body of the test object from the first image sequence, on the basis of the monitoring; Step 430 for acquiring a second image sequence collected by the endoscope after a collection time of the reference image; and Step 440 for determining a target portion of the test subject to be tested by the endoscope, on the basis of the second image sequence. By the process, identification of the target portion can be completed automatically without depending on external input, therefore correctness and a speed of a preliminary step operation of an endoscope test can be secured.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] TECHNICAL FIELD Exemplary embodiments of the present disclosure relate generally to the field of computers, and more particularly to methods and apparatus for image processing. [Background technology]

[0002] Image processing technology is widely used in various fields. In the medical field, there is an increasing demand for processing and analyzing large amounts of medical image data. In particular, there is a demand for solutions that enable doctors to make more accurate and faster diagnoses regarding medical endoscopic images. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method of image processing is provided, comprising: monitoring a prediction result of a correspondence for an image in a first image sequence collected by an endoscope over time during an examination, the prediction result being indicative of whether the endoscope is located inside the body of the examined subject at the time the corresponding image was collected, determining a reference image from the first image sequence based on the monitoring, the reference image corresponding to the endoscope entering the body of the examined subject, obtaining a second image sequence collected by the endoscope after the reference image collection time, and determining a target site of the examined subject to be examined by the endoscope based on the second image sequence.

[0004] In a second aspect of the present disclosure, an electronic device is provided, the electronic device including at least one processing circuit configured to: monitor a prediction result of a correspondence for an image in a first image sequence acquired by an endoscope over time during an examination, the prediction result being indicative of whether the endoscope is located inside the body of the examined subject at the time the corresponding image was acquired; determine a reference image from the first image sequence corresponding to the endoscope entering the body of the examined subject based on the monitoring; obtain a second image sequence acquired by the endoscope after the acquisition time of the reference image; and determine a target site of the examined subject inspected by the endoscope based on the second image sequence.

[0005] In some embodiments of the second aspect, the first image sequence includes a first image and the corresponding prediction result includes a first prediction result for the first image. The at least one processing circuit is further configured to generate, for the first image in the first image sequence, a first prediction result regarding whether the endoscope is within the body of the inspected subject at the time the first image was acquired based on the first trained classification model.

[0006] In some embodiments of the second aspect, the first image sequence includes a second image, and the corresponding prediction result includes a second prediction result for the second image. The at least one processing circuit is further configured to, for a second image in the first image sequence, extract image features of the second image, determine a first similarity of the image features to an internal reference feature characterizing an interior of the reference object and a second similarity of the image features to an external reference feature characterizing an exterior of the reference object, and generate a second prediction result regarding whether the endoscope is located within the inspected object's body at the time the second image was acquired by comparing the first similarity and the second similarity.

[0007] In some embodiments of the second aspect, the at least one processing circuit is further configured to, before determining the reference image based on the monitoring, determine a first number of adjacent images from the first image sequence, and set a prediction result for an intermediate image among the first number of adjacent images based on a corresponding prediction result of the first number of adjacent images.

[0008] In some embodiments of the second aspect, the at least one processing circuit is further configured to determine from a first number of adjacent images those images whose corresponding prediction results indicate that the endoscope is located within the subject's body, and if the determined number of images is greater than a first threshold number associated with the first number, set the prediction result for the intermediate image as indicating that the endoscope is located within the subject's body, and if the determined number of images is not greater than the first threshold number, set the prediction result for the intermediate image as indicating that the endoscope is not within the subject's body.

[0009] In some embodiments of the second aspect, the at least one processing circuit is further configured to determine whether all predicted results of correspondences of a second number of consecutive images in the first image sequence indicate that the endoscope is located within the body of the subject, and in response to determining that all predicted results of correspondences of the second number of consecutive images indicate that the endoscope is located within the body of the subject, determine a reference image based on the second number of consecutive images.

[0010] In some embodiments of the second aspect, the acquisition time span corresponding to the second number of consecutive images is related to an image acquisition frequency of the endoscope.

[0011] In some embodiments of the second aspect, the at least one processing circuit is further configured to monitor whether the image quality of each image acquired by the endoscope after the acquisition time of the reference image exceeds a threshold image quality, and in response to monitoring that the image quality of the third image exceeds the threshold image quality, configure the images acquired sequentially by the endoscope within a certain time period after the acquisition time of the third image as a second image sequence.

[0012] In some embodiments of the second aspect, the second image sequence satisfies at least one of: a collection time span corresponding to the second image sequence is equal to a threshold span; or a cumulative image distance between adjacent images in the second image sequence is greater than a threshold, and the image distance is determined based on optical flow.

[0013] In some embodiments of the second aspect, the at least one processing circuit is further configured to generate, for a third image in the second image sequence, a first identification result regarding which of the multiple predetermined regions the third image corresponds to based on the trained second classification model.

[0014] In some embodiments of the second aspect, the at least one processing circuit is further configured to perform, for a fourth image in the second image sequence, the steps of extracting image features of the fourth image, determining a similarity of correspondence of the image features to a plurality of reference features used to characterize each of the plurality of predetermined regions, and generating a second identification result regarding which of the plurality of predetermined regions the fourth image corresponds to by comparing the similarity of correspondence.

[0015] In some embodiments of the second aspect, the at least one processing circuit is further configured to determine that the target site is the first site in response to a number of identification results indicative of a first site among the plurality of predetermined sites in the corresponding identification result exceeding a second threshold number, or to determine that the target site is the second site in response to a proportion of identification results indicative of a second site among the plurality of predetermined sites in the corresponding identification result exceeding a threshold proportion.

[0016] In a third aspect of the present disclosure, an electronic device is provided, the device comprising at least one processing unit and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the method of the first aspect.

[0017] In a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the method according to the first aspect.

[0018] It should be understood that the contents described in the summary of the present invention are not intended to limit the main or important features of the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure. Other features of the present disclosure will be easily understood from the following description. [Brief description of the drawings]

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which the same or similar reference numerals indicate the same or similar elements. [Figure 1] 1 illustrates a schematic diagram of an exemplary environment in which embodiments of the present disclosure can be practiced. [Diagram 2] 1 shows a schematic diagram of an example architecture for image processing according to some embodiments of the present disclosure. [Diagram 3] 1 shows a schematic diagram of an example of stomach or intestine determination according to some embodiments of the present disclosure. [Figure 4] 1 shows a flowchart of a process for image processing according to some embodiments of the present disclosure. [Diagram 5] FIG. 1 illustrates a block diagram of an electronic device in which several embodiments of the present disclosure can be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the accompanying drawings show some embodiments of the present disclosure, it should be understood that the present disclosure can be realized in various forms and should not be interpreted as being limited to the embodiments described herein, but rather, these embodiments are provided to allow a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are merely intended to play an exemplary role and are not intended to limit the scope of protection of the present disclosure.

[0021] It should be noted that any section / subsection headings provided herein are not limiting. Various embodiments are described throughout this specification, and any type of embodiment may be included in any section / subsection. In addition, embodiments described in any section / subsection may be combined in any manner with other embodiments described in the same section / subsection and / or different sections / subsections.

[0022] In describing embodiments of the present disclosure, the term "comprising" and similar terms should be understood as an open inclusion, i.e., "including, but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicit and implicit definitions may be included below.

[0023] The term "circuitry" as used herein may refer to hardware circuits and / or a combination of hardware circuits and software. For example, a circuit may be a combination of analog and / or digital hardware circuits and software / firmware. As another example, a circuit may be any portion of a hardware processor with software, which comprises digital signal processor(s), software, and memory(s) that together enable a device to operate to perform various functions. In yet another example, a circuit may be a hardware circuit and / or processor, e.g., a microprocessor or part of a microprocessor, that requires software / firmware to operate, but may be absent when not required for operation. As used herein, the term "circuitry" also includes only a hardware circuit or processor, or a part of a hardware circuit or processor, and its (or their) associated software and / or firmware implementation.

[0024] As briefly mentioned above, there is an increasing demand in the medical field to process and analyze large amounts of medical image data. Traditionally, doctors have to rely on their own experience and visual judgment to analyze endoscopic images, perform corresponding examination items, and make appropriate diagnoses. However, this method has issues such as subjectivity, time-consuming, and possible misjudgments. Therefore, with its development, artificial intelligence (AI) has been applied to the analysis of endoscopic images. In general, different AI algorithms and models need to be developed and used for each part of the object being examined (e.g., the human body). Therefore, correctly determining the part being examined is crucial for subsequent image analysis.

[0025] For example, endoscopes are often used in gastrointestinal examinations. In gastrointestinal examinations, gastroscopy and intestinal endoscopy are usually performed by using the same endoscopic device and switching to different lenses. The lens used for gastroscopy is the "mouth" of the patient's body, and the lens used for intestinal endoscopy is the "anus" of the patient's body. During a day's examination, depending on the condition of the patient, the examiner irregularly switches between the gastroscopy lens and the intestinal endoscopy lens, and the corresponding examination program is called after the lens is switched. It is an essential task to determine how to accurately and quickly determine the part currently being examined. In particular, in the case of AI-based assisted diagnosis, automatic discrimination between gastroscopy and intestinal endoscopy is a preliminary operation for the subsequent AI task.

[0026] Conventionally, some approaches focus on identifying parts or organs, and therefore cannot be directly applied to stomach or intestine identification scenes. Some other approaches identify whether a certain part or organ is present for each frame, and are unable to complete the identification task within a certain time period (e.g., within the first time period when the endoscope enters the patient's body). Still other approaches lack a comprehensive judgment for multiple frames, and therefore are not suitable for stomach or intestine identification scenes that require a comprehensive judgment of a few frames collected by the endoscope entering the patient's body.

[0027] Although certain imaging challenges have been described above using endoscopes in the examination of the human digestive tract, it will be appreciated that similar imaging challenges may exist in the endoscopic examination of other subjects or sites.

[0028] Therefore, an embodiment of the present disclosure provides an approach for image processing, which includes monitoring a prediction result of a correspondence for an image in a first image sequence, the first image sequence being collected by an endoscope over time during an examination. The prediction result indicates whether the endoscope is located within the body of the inspected subject at the time the corresponding image is collected. Based on the monitoring of the prediction result, a reference image is determined from the first image sequence, which corresponds to the endoscope entering the body of the inspected subject. Further, a second image sequence is acquired, the second image sequence being collected by the endoscope after the reference image collection time. Based on the acquired second image sequence, a target region of the inspected subject to be inspected by the endoscope is determined.

[0029] Therefore, it is possible to automatically determine the corresponding area and examination items from the few images collected. In this way, it is possible to ensure the accuracy and speed of the pre-stage operations of endoscopic examination without relying on additional external input (e.g., text, voice, key input, etc.). In particular, in AI-assisted endoscopic examination, accurate and quick determination of the area to be examined is useful for the accurate and timely application of subsequent AI tasks.

[0030] Example Environment FIG. 1 shows a schematic diagram of an exemplary environment 100 in which an embodiment of the present disclosure can be implemented. In the environment 100, an electronic device 120 may receive an image sequence 110 and output a determination result. The image sequence 110 includes a set of images 112 sorted by acquisition time. The images 112 are, for example, images of an object to be examined acquired by a medical endoscope. The images 112 may be acquired in real time or offline. The determination result is, for example, a site inspected by the endoscope, also referred to as a target site. For example, in a digestive tract examination scene, the determination result may indicate an intestinal endoscope or a gastroscopy endoscope.

[0031] In the embodiments of the present disclosure, the object to be examined may be any suitable type of object, such as, for example, a human body, an animal, etc. The endoscope may be used for various suitable examinations, such as, for example, a digestive tract examination, a throat examination, etc. In the following, some examples and embodiments will be described by way of example, in which an intestinal endoscope and a gastroscope are used to examine the digestive tract of a human body. However, it should be understood that this is merely illustrative and is not intended to be limiting. In the embodiments of the present disclosure, the type of the object to be examined, the endoscope, and the part of the object to be examined are not limited.

[0032] In the environment 100, the electronic device 120 may be any type of device having a computing capability, including a terminal device. The terminal device may be any type of mobile, fixed, or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book reader, a gaming device, or any combination thereof, and further including accessories and peripherals of these devices, or any combination thereof. The electronic device 120 may be any type of device having a computing capability, including a server device. The server device may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc.

[0033] In particular, in some embodiments, electronics 120 may be a computing device that communicates directly with an endoscope. In some embodiments, electronics 120 may be implemented such that it is included in the endoscope, or individual operations described below with reference to electronics 120 may be performed by the endoscope itself.

[0034] It should be understood that the structure and functionality of environment 100 is described for illustrative purposes only and is not intended to imply any limitation on the scope of the present disclosure. In FIG. 1 and the following description, electronic device 110 collects image sequence 110 and determines the decision result. However, this is merely illustrative and is not intended to limit the scope of the present disclosure. In some embodiments, the collection of image sequence 110 and the determination of the decision result may be performed by different devices.

[0035] Hereinafter, the exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0036] Exemplary Architecture 2 shows a schematic diagram of an example architecture of image processing according to some embodiments of the present disclosure. The architecture 200 may be implemented in the electronic device 120. The architecture 200 generally comprises a monitoring module 210, a reference image determination module 220, and a target region determination module 235. Additionally or alternatively, the architecture 200 may further comprise an endoscope configured to collect an image sequence. Such an image sequence may include images collected by the endoscope in the process of entering the body of the subject from outside the body. The architecture 200 will now be described with reference to FIG. 1.

[0037] Continuing to refer to FIG. 2, the monitoring module 210 receives a first image sequence 205. The first image sequence 205 may be a set of images collected by the endoscope over time in the inspection process. It should be understood that the images collected by the endoscope over time in the inspection process may be considered as a video. Thus, the images collected by the endoscope are also referred to as frames, i.e., the first image sequence 205 may include a plurality of frames. The monitoring module 210 may monitor the plurality of frames and perform a prediction for each frame therein. Furthermore, the monitoring module 210 may output a prediction result 215 for each frame. Such a prediction result 215 indicates whether the endoscope is located inside the body of the inspected subject at the time the corresponding frame is collected by the endoscope. That is, the prediction result 215 is a prediction for a single frame. Hereinafter, for ease of explanation, indicating whether the endoscope is located inside the body of the inspected subject will simply be referred to as being inside the body.

[0038] In some embodiments, deep learning methods may be employed to perform such single frame predictions. For example, for each image (e.g., a first image) in the first image sequence 205, the monitoring module 210 may use the trained classification model (also referred to as a first classification model) to generate a prediction (also referred to as a first prediction) regarding whether the endoscope was in the body at the time the respective image was acquired.

[0039] Exemplarily, in-body and out-body data of the same type of subject may be collected to train the classification model. For example, if the subject is a human and the part to be examined is a stomach, the in-body data may include a series of images of the oral cavity, throat, esophagus, stomach, etc. Also, for example, if the subject is a human and the part to be examined is an intestine, the in-body data may include a series of images of the buttocks, anus, intestine, etc. Then, the in-body / out-body data may be collected and annotated. Furthermore, the annotated data may be used to build a deep learning model of binary classification. It should be understood that the operations of collecting and annotating data and training the model may be performed by a device other than the electronic device 120, and the present disclosure is not limited thereto.

[0040] The monitoring module 210 may utilize such a deep learning model (e.g., a binary classification model) to analyze the first image sequence 205 and predict whether the endoscope is in-body or out-of-body. For example, the monitoring module 210 utilizes such a deep learning model to obtain an in-body prediction score. Based on a pre-set in-body threshold score, if the prediction score exceeds the threshold score, the monitoring module 210 determines that the prediction result corresponding to the current frame is in-body, otherwise, it is out-of-body.

[0041] Alternatively or additionally, in some embodiments, a feature matching method may be employed to perform single frame prediction. For example, for each image (e.g., the second image) in the first image sequence 205, the monitoring module 210 extracts image features of these images. The monitoring module 210 determines a similarity of the extracted image features to an internal reference feature (also referred to as a first similarity) and to an external reference feature (also referred to as a second similarity). The internal reference feature characterizes the inside of the reference object, and the external reference feature characterizes the outside of the reference object. In other words, the internal reference feature is representative of the feature inside the reference object, and the external reference feature is representative of the feature outside the reference object. Furthermore, the monitoring module 210 generates a prediction result (also referred to as a second prediction result) of whether the endoscope is inside the body or not by comparing the two similarities. Such a prediction result indicates whether the endoscope is located inside the body of the examined object at the time when the corresponding image was acquired. For example, for a given image, if the first similarity is greater than the second similarity, the predicted result may indicate inside the body, and if not, the predicted result may indicate outside the body.

[0042] Exemplarily, for feature matching, internal and external data may be collected for the same type of subject, and image features may be extracted from the data. Feature operators include, but are not limited to, color histograms, texture histograms, deep network features, and the like. Furthermore, the electronic device 120 may use a clustering algorithm (e.g., Kmeans algorithm) to spatially cluster the internal and external features and build a feature library. It should be understood that the operation of collecting data may be performed by a device other than the electronic device 120, and the present disclosure is not limited in this regard.

[0043] The monitoring module 210 may use such feature library to perform a feature distance ratio calculation on the first image sequence 205 to obtain an in-body feature distance ratio. For example, based on a pre-set in-body threshold feature distance ratio, if the feature distance ratio of the current frame is greater than the threshold feature distance ratio, the monitoring module 210 determines that the current frame corresponds to in-body, otherwise, it corresponds to extra-body.

[0044] 2, the reference image determination module 220 receives the prediction result 215 and determines a reference image 225. Such a reference image 225 corresponds to the endoscope entering the body of the subject. That is, the reference image determination module 220 receives the prediction result 215 for each frame and determines which frame or frames indicate that the endoscope has entered the body of the subject. As can be seen, the determination of the reference image is the basis for subsequent operations.

[0045] As an example, the first image sequence 205 includes, for example, 12 frames, ie, frame 1, frame 2, frame 3, ... frame 12, in chronological order. Assume that the prediction results corresponding to frames 1 to 5 all indicate outside the body, and the prediction result corresponding to frame 6 indicates inside the body. Ideally, it can be predicted that frames 6 and after all indicate inside the body, so that frame 6 can be determined as the reference image. However, in some cases, the prediction result of the above single frame may be affected by device noise, environmental noise, or algorithm accuracy, which may result in a false positive.

[0046] In some embodiments, the reference image determination module 220 may determine a predetermined number (also referred to as a first number) of adjacent images from the first image sequence 205 before determining the reference image 225. Such adjacent images may include, for example, a predetermined number of consecutive images acquired before the image currently being determined and a predetermined number of consecutive images acquired after the image currently being determined. The reference image determination module 220 may set a prediction result of an intermediate image among these images based on the corresponding prediction results of these images. For example, if the corresponding prediction results of these images all indicate being inside the body, the prediction result of the intermediate image among these images may be set to indicate being inside the body. In this way, the prediction result of a single frame can be smoothed to achieve the purpose of removing noise, removing the influence of algorithm accuracy, etc., so that the accuracy of the reference image determination can be improved.

[0047] Continuing with the above example, assume that the first number is 5 frames. When determining whether the 6th frame is a reference image, the adjacent frames acquired before the 6th frame are the 4th to 5th frames, and the adjacent frames acquired after the 6th frame are the 7th to 8th frames. The prediction results of the single frames from the 4th to 8th frames may all indicate that they are in-body, or the prediction results of some of these frames may indicate that they are in-body. The reference image determination module 220 can smooth the prediction results of the single frame by synthesizing the prediction results corresponding to these five frames to set whether the 6th frame is in-body or out-of-body, thereby reducing the risk of misjudging the reference image.

[0048] In some embodiments, the reference image determination module 220 may further determine, from a first number of adjacent images, images having corresponding prediction results indicating that the endoscope is located inside the body of the subject, and determine the number of such images. If the number is greater than a threshold value for the first number (also referred to as a first threshold number), the prediction results of intermediate images among these adjacent images are set as indicating that the endoscope is inside the body of the subject. If the number is not greater than the threshold, the prediction results of intermediate images among these adjacent images are set as indicating that the endoscope is not inside the body of the subject. It should be noted that the first threshold number may be equal to or less than the first number. The aspect of smoothing the prediction results indicating whether the endoscope is inside or outside the body based on the prediction results for each frame of the consecutive frames may also be referred to as smoothing or local smoothing. In this way, the accuracy of the determination of the reference image can be improved.

[0049] Continuing with the above example, assume that the first number is 5 and the corresponding first threshold number is 3. When smoothing the single-frame prediction result of the 6th frame, the adjacent frames collected before the 6th frame are the 4th to 5th frames, and the adjacent frames collected after the 6th frame are the 7th to 8th frames. If the prediction results of three or more of the prediction results corresponding to the images of these five frames indicate that the image is inside the body, the prediction result of the 6th frame is set as being inside the body of the subject.

[0050] Furthermore, after determining the prediction results (original or smoothed) of a single frame, the reference image determination module 220 determines a reference image based on these prediction results.

[0051] However, in some cases, if a frame in which an in-vivo prediction result is first indicated is determined as a reference image, there is a possibility that the reference image will be erroneously determined. For example, the prediction results of the third, ninth, tenth, and eleventh frames among the first to eleventh frames indicate that the image is inside the body. Considering that the third frame may be affected by noise, the endoscope actually truly entered the body of the subject from the ninth frame. In this case, if the third frame is determined as a reference image, a certain bias will occur.

[0052] For this reason, in some embodiments, a certain time may be set as a buffer when determining a reference image. For example, the reference image determination module 220 may determine whether or not all corresponding prediction results of a predetermined number (also referred to as a second number) of consecutive images in the first image sequence 205 indicate that the endoscope is located inside the body of the subject. Furthermore, the reference image determination module 220 may determine a reference image based on the number of consecutive images. For example, the reference image determination module 220 may determine the last frame of the consecutive frames that satisfy the second number as the reference image. As a result, the effect of a time buffer can be obtained. In this way, the accuracy of the reference image determination can be further improved. It is assumed that the time buffer corresponds to K3 frames. If all prediction results of the consecutive K3 frames indicate outside the body, the last frame of these consecutive K3 frames may be determined as the reference image.

[0053] Continuing with the above example, assume that the second number is 3. When determining the reference image, if the prediction results of the third, ninth, tenth, and eleventh frames indicate that they are in-body, then the prediction results of the consecutive frames (i.e., frames 9 to 11) that satisfy the second number all indicate that they are outside the body. The reference image determination module 220 may continue to determine subsequent frames until it determines frame 9 as the reference image when the prediction results of the consecutive frames (i.e., frames 9 to 11) that satisfy the second number all indicate that they are in-body.

[0054] The second number may be preset as a fixed value or may be preset as a number of frames corresponding to a preset acquisition time span, such as 3 frames or the number of frames corresponding to 1 second. In some embodiments, the acquisition time span corresponding to the second number of consecutive images is related to the image acquisition frequency of the endoscope. For example, if the acquisition frequency is 60 fps, the acquisition time span is 1 second. As another example, if the acquisition frequency is 15 fps, the acquisition time span is 4 seconds.

[0055] 2, the acquisition time of the reference image 225 may be used to determine the start of timing. For example, this acquisition time may be used as the time to start timing, and an image sequence acquired thereafter (also referred to as a second image sequence 230) may be sent to the target site determination module 235. The target site determination module 235 determines an examination site (also referred to as a target site) of an object to be examined by an endoscope based on such an image sequence. That is, the time is started from the acquisition time determined based on the first image sequence 205, and the examination site is determined based on the second image sequence 230 acquired thereafter. In this way, the effects of noise, algorithm accuracy, etc. can be minimized, and a valid image sequence can be acquired as early as possible.

[0056] In some embodiments, the acquisition time of the reference image may not be used as the start of the timing. To obtain a high quality second image sequence 230, the image quality of each image acquired after the acquisition time of the reference image 225 may be monitored to see if it exceeds a threshold image quality. Furthermore, the acquisition time of an image (also called a third image) whose monitored image quality exceeds the threshold image quality may be used as the start of the timing, and images acquired by the endoscope sequentially within a certain time after the acquisition time may be used as the second image sequence 230. In such embodiments, the image quality may be determined in any suitable manner (e.g., but not limited to, a supervised deep learning model, a grayscale variance product, etc.).

[0057] The end point of the second image sequence 230, i.e., the timing of the end of the timing, may be determined in various suitable ways. In some embodiments, the second image sequence 230 satisfies that the corresponding acquisition time span is equal to the threshold span. Such a threshold span may be a preset fixed value, for example, 5 seconds. That is, the timing starts from the effective time of the endoscope's entry into the body and ends after 5 seconds. The second image sequence 230 includes all frames acquired during this period. This ensures that a valid image sequence is acquired in the shortest possible time. In this way, the examination site can be determined based on a small number of valid images, which does not interfere with subsequent operations.

[0058] Alternatively or additionally, in some embodiments, the acquisition of the second image sequence 230 may use adaptive timing. The second image sequence 230 satisfies that a cumulative image distance between adjacent images is greater than a threshold. Such image distance is determined based on optical flow. This can ensure that a stable image sequence is acquired. In this way, the effectiveness of the acquisition of the second image sequence can be improved.

[0059] For example, after timing is started, for each frame collected, a single-frame optical flow field (e.g., a sparse optical flow field) may be extracted and the distance between the two frames may be calculated respectively. Alternatively or additionally, if the image quality of a frame is below a threshold, the target region determination module 235 may remove the frame and continue tracking the distance for subsequent frames. In this way, images whose image quality is too low to be tracked may be removed. Furthermore, these distances may be accumulated, e.g., image distances may be accumulated from a starting frame collected at the start of timing to each frame collected thereafter. Such accumulated distances are compared to a threshold distance. If the accumulated distance is greater than the threshold distance, timing is stopped. The second image sequence 230 includes all frames between the starting frame when timing starts and the final frame when timing ends.

[0060] Alternatively or additionally, timing of the capture time of the second image sequence 230 may be stopped when either the capture time span reaches a threshold span or the cumulative distance between images is greater than a threshold distance.

[0061] The above embodiment describes how to determine the collection time period of the second image sequence 230. Hereinafter, we will continue to describe how the target region determination module 235 determines the target region after the second image sequence 230 is acquired.

[0062] In some embodiments, for images in the second image sequence 230, the target region determination module 235 may generate a corresponding identification result for the target region, i.e., perform a single frame identification. Such an identification result indicates one of a plurality of predetermined regions. The target region determination module 235 may then determine the target region based on the corresponding identification result. For example, the predetermined region includes the esophagus, the stomach, and the intestine. The target region determination module 235 may identify whether each frame of the second image sequence 230 contains features corresponding to these regions, and further determine which region of the inspected object is currently being inspected based on the identification results of these frames. In this way, the region inspected by the endoscope may be automatically determined.

[0063] In some embodiments, the single frame identification of the target region may be performed using a deep learning method. For example, for an image (e.g., the third image) in the second image sequence 230, the target region determination module 235 may generate an identification result (also referred to as a first identification result) based on a trained classification model (also referred to as a second classification model). Such an identification result may indicate which inspection region (also referred to as a predetermined region) each individual frame in the second image sequence 230 corresponds to. For example, to obtain such a classification model, past inspection data of these inspection regions may be collected and annotated. Then, the annotated past inspection data may be utilized to build a deep learning model of multiple classifications (e.g., binary classification). It should be understood that the operations of collecting and annotating data and training the model may be performed in a device other than the electronic device 120, and the present disclosure is not limited in this respect.

[0064] The target site determination module 235 may use such a deep learning model to analyze the second image sequence 230 and predict the current examination site. For example, the examination site includes the stomach and the intestine. The target site determination module 235 uses the deep learning model to obtain a prediction score. Based on a preset threshold score for the stomach, if the prediction score exceeds the threshold score corresponding to the stomach, the target site determination module 235 determines that the examination site corresponding to the current frame is the stomach, otherwise, it is the intestine.

[0065] In some embodiments, single frame identification of the target region may be performed using feature matching. For example, for an image (e.g., the fourth image) in the second image sequence 230, the target region determination module 235 may extract image features thereof. The target region determination module 235 then determines corresponding similarities between the extracted image features and a number of reference features, which are used to characterize a number of inspection regions, respectively. The target region determination module 235 generates an identification result (also referred to as a second identification result) by comparing the corresponding similarities. Such an identification result may indicate which inspection region each of all frames in the second image sequence 230 corresponds to.

[0066] Exemplarily, to obtain the reference features to be used, past inspection data of these inspection sites may be collected and image features may be extracted from these past inspection data. Feature operators include, but are not limited to, color histograms, texture histograms, and the like. Furthermore, the electronic device 120 may use a clustering algorithm (e.g., Kmeans algorithm) to spatially cluster the features of different inspection sites and build a feature library. It should be understood that the operation of collecting data may be performed by a device other than the electronic device 120, and the present disclosure is not limited in this regard.

[0067] The target region determination module 235 may use such feature library to perform a feature distance ratio calculation on the second image sequence 230 to obtain a feature distance ratio. For example, the examination region includes a stomach and an intestine. Based on a preset threshold feature distance ratio of the stomach, if the feature distance ratio of the current frame is greater than the threshold feature distance ratio, the target region determination module 235 determines that the examination region corresponding to the current frame is the stomach, otherwise, it is the intestine.

[0068] 2, the target portion determination module 235 outputs a determination result regarding the target portion based on the inspection portion corresponding to each frame in the second image sequence 230. In some embodiments, due to the effects of noise or algorithm accuracy, these frames may correspond to different inspection portions. In some embodiments, if the number of identification results indicating the first portion of the multiple predetermined portions in the corresponding identification results exceeds a second threshold number, the target portion determination module 235 determines that the target portion is the first portion.

[0069] As an example, the examination region includes, for example, the stomach and the intestine. Assume that the second image sequence includes a total of 10 frames. The second threshold number is, for example, 5 frames. Assume that the identification results corresponding to 8 frames in the second image sequence 230 indicate the stomach. If the number of these frames is greater than the second threshold number, the target region determination module 235 determines that the target region is the stomach, otherwise it is the intestine.

[0070] In some embodiments, if the proportion of the identification results indicating the second region among the plurality of predetermined regions in the corresponding identification results exceeds a threshold proportion, the target region determination module 235 determines that the target region is the second region. Continuing with the above example, the threshold proportion may be, for example, 0.7. Assume that the identification results corresponding to 8 of the 10 frames of the second image sequence 230 indicate the stomach. If the proportion of the number of these frames to the total number of frames is greater than the threshold proportion, the target region determination module 235 determines that the target region is the stomach, otherwise it is the intestine.

[0071] The data processing method has been described above through various embodiments of the present disclosure. The beneficial effects of these embodiments include that the identification of the endoscope examination site can be completed automatically without the need for other presentations or operations by a doctor. In addition, such an approach is not limited to a specific device, and the examination site can be determined by simply collecting a certain image sequence. Such an image sequence may be collected online in real time or offline. Furthermore, such an approach can quickly determine the examination site when the endoscope enters the body of the subject, without interfering with the performance of subsequent tasks. As can be seen, this approach can ensure the accuracy and speed of the pre-stage operation of the endoscope examination without relying on other means (e.g., text, voice, or key input, etc.).

[0072] Example implementation of a gastrointestinal examination In order to better understand the embodiments of the present disclosure, a specific example will be described below with reference to FIG.

[0073] 3 shows a schematic diagram of an example 300 of stomach or intestine determination according to some embodiments of the present disclosure. The example 300 may be implemented in the electronic device 120. It is assumed that the examination site includes the stomach and intestines, and the corresponding examination items are, for example, gastroscopy and intestinal endoscopy. The following description will be given with reference to FIG. 1.

[0074] In box 305, electronics 120 acquires video frames related to the object under inspection via the endoscope, such as each frame that the endoscope collects in real time after powering on.

[0075] In box 310, electronics 120 predicts for each frame in the video frame to determine whether the frame corresponds to whether the endoscope is located inside or outside the body of the inspected subject.

[0076] In box 315, the electronics 120 determines whether the current frame corresponds to a situation in which the endoscope is located inside the body of the subject. If the electronics 120 determines that the current frame corresponds to the endoscope being located inside the body of the subject, the process proceeds to box 320. That is, the current frame is the reference image described above. If the electronics 120 determines that the current frame corresponds to the endoscope being located outside the body of the subject, the process returns to box 310, where the electronics 120 continues the determination for the next frame.

[0077] In box 320, the electronics 120 begins timing based on a determination that the endoscope is located within the body of the inspected subject, such timing being recorded from the time of collection of a valid frame corresponding to the determination that the endoscope has entered the body.

[0078] In box 325, the electronics 120 acquires target video frames associated with the subject under examination, for example, via an endoscope. Such target video frames may include a number of consecutive frames acquired since a time count was started.

[0079] In box 330, as the target video frames are collected, ie, in a timing process, the electronics 120 performs stomach / intestinal prediction for each of the collected frames.

[0080] In box 335, electronics 120 ends timing and stops collecting the second video frames. Timing may be ended in response to reaching a predetermined duration or the cumulative distance of adjacent frames exceeding a threshold.

[0081] In box 340, the electronic device 120 performs stomach / intestine prediction for all video frames. For example, the electronic device 120 determines the stomach / intestine prediction result by combining the prediction results of each target frame. In another example, the electronic device 120 performs stomach / intestine prediction for each target video frame, and further determines the stomach / intestine prediction result by combining the prediction results of these frames. The stomach / intestine prediction for all target video frames determines the prediction result based on, for example, the number or percentage of frames that satisfy a condition among all target frames. Furthermore, the electronic device 120 determines whether the currently detected part of the endoscope is the stomach or the intestine based on the combined prediction result.

[0082] Through the above-mentioned process, the electronic device 120 can start capturing images from outside the body of the subject using the endoscope. When the endoscope enters the body of the subject through the mouth / anus, the electronic device 120 first completes the prediction of inside / outside the body, and then starts timing. Then, the electronic device 120 can continue capturing images inside the body of the subject using the endoscope. When the timing ends, the electronic device 120 completes the determination of the stomach / intestines based on the identification results of the few collected images. In this way, the identification of the target site is automatically completed without relying on other means, so that the accuracy and speed of the pre-stage operation of the endoscopic examination can be ensured.

[0083] Example Process 4 shows a flow chart of a process 400 of image processing according to some embodiments of the present disclosure. The process 400 may be implemented in the electronic device 120. For ease of discussion, the process 400 will be described with reference to FIG.

[0084] In box 410, electronics 120 monitors predictions of correspondences for images in a first sequence of images acquired by an endoscope over time during an examination, the predictions indicating whether the endoscope was positioned within the subject's body at the time the corresponding images were acquired.

[0085] In box 420, electronics 120 determines, based on the monitoring, a reference image from the first image sequence that corresponds to the endoscope entering the inspected subject's body.

[0086] In box 430, electronics 120 acquires a second sequence of images acquired by the endoscope after the acquisition time of the reference image.

[0087] In box 440, the electronics 120 determines a target region of the inspected object to be inspected by the endoscope based on the second image sequence.

[0088] In some embodiments, the first image sequence includes a first image and the corresponding prediction result includes a first prediction result for the first image, and process 400 further includes electronic device 120 generating, for the first image in the first image sequence, a first prediction result regarding whether or not the endoscope was within the body of the subject at the time the first image was acquired based on the trained first classification model.

[0089] In some embodiments, the first image sequence includes a second image and the corresponding prediction result includes a second prediction result for the second image, and the process 400 further includes the electronic device 120 extracting image features of the second image for the second image in the first image sequence, determining a first similarity of the image features to internal reference features characterizing the inside of the reference object and a second similarity of the image features to external reference features characterizing the outside of the reference object, and generating a second prediction result regarding whether the endoscope is located inside the inspected object's body at the time the second image is acquired by comparing the first similarity and the second similarity.

[0090] In some embodiments, the process 400 further includes, before the electronic device 120 determines the reference image based on the monitoring, determining a first number of adjacent images from the first image sequence, and setting a prediction result for an intermediate image among the first number of adjacent images based on corresponding prediction results of the first number of adjacent images.

[0091] In some embodiments, the process 400 further includes the electronic device 120 determining from a first number of adjacent images an image having a corresponding prediction result indicating that the endoscope is located within the subject's body, and if the determined number of images is greater than a first threshold number associated with the first number, setting the prediction result for the intermediate image as indicating that the endoscope is within the subject's body, and if the determined number of images is not greater than the first threshold number, setting the prediction result for the intermediate image as indicating that the endoscope is not within the subject's body.

[0092] In some embodiments, the process 400 further includes the electronic device 120 determining whether all of the corresponding prediction results of the second number of consecutive images in the first image sequence indicate that the endoscope is located within the subject's body, and in response to determining that all of the corresponding prediction results of the second number of consecutive images indicate that the endoscope is located within the subject's body, determining a reference image based on the second number of consecutive images.

[0093] In some embodiments, the acquisition time span corresponding to the second number of consecutive images is related to the image acquisition frequency of the endoscope.

[0094] In some embodiments, the process 400 further includes the electronic device 120 monitoring whether the image quality of each image captured by the endoscope after the capture time of the reference image exceeds a threshold image quality, and in response to monitoring that the image quality of the third image exceeds the threshold image quality, forming images captured sequentially by the endoscope within a certain time period after the capture time of the third image into a second image sequence.

[0095] In some embodiments, the second image sequence satisfies at least one of: a collection time span corresponding to the second image sequence is equal to a threshold span; or a cumulative image distance between adjacent images in the second image sequence is greater than a threshold, where the image distance is determined based on optical flow.

[0096] In some embodiments, process 400 further includes electronic device 120 generating a corresponding identification result for the target site for an image in the second image sequence and determining the target site based on the corresponding identification result, the identification result indicating one of a plurality of predetermined sites.

[0097] In some embodiments, process 400 further includes, for a third image in the second image sequence, electronic device 120 generating a first identification result regarding which of the multiple predetermined regions the third image corresponds to based on the trained second classification model.

[0098] In some embodiments, process 400 further includes, for a fourth image in the second image sequence, electronic device 120 extracting image features of the fourth image, determining corresponding similarities of the image features to a plurality of reference features used to characterize each of the plurality of predetermined regions, and generating a second identification result regarding which of the plurality of predetermined regions the fourth image corresponds to by comparing the corresponding similarities.

[0099] In some embodiments, process 400 further includes electronic device 120 determining that the target location is the first location in response to a number of identification results indicative of a first location among the plurality of predetermined locations in the corresponding identification result exceeding a second threshold number, or determining that the target location is the second location in response to a proportion of identification results indicative of a second location among the plurality of predetermined locations in the corresponding identification result exceeding a threshold proportion.

[0100] Example Equipment 5 illustrates a block diagram of an electronic device 500 capable of implementing one or more embodiments of the present disclosure. It should be understood that the electronic device 500 illustrated in FIG. 5 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. The electronic device 500 illustrated in FIG. 5 can be used to implement the electronic device 120 of FIG. 1.

[0101] As shown in Fig. 5, the electronic device 500 is in the form of a general-purpose electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be a real processor or a virtual processor, and is capable of performing various processes based on a program stored in the memory 520. In a multiprocessor system, the multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capability of the electronic device 500.

[0102] The electronic device 500 typically includes a number of computer storage media. Such media may be any available media accessible by the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a combination thereof. The storage device 530 may be removable or non-removable media and may include machine-readable media, such as a flash drive, a disk, or any other medium that may be used to store information and / or data (e.g., training data for training) and that may be accessible within the electronic device 500.

[0103] The electronic device 500 may further include other removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, a disk drive for reading from and writing to a removable non-volatile disk (e.g., a "floppy disk") and a disk drive for reading from and writing to a removable non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data medium interfaces. The memory 520 may include a computer program product 525 having one or more program modules configured to perform various methods or operations of various embodiments of the present disclosure.

[0104] The communication unit 540 allows communication with other electronic devices over a communication medium. Furthermore, the functionality of the components of the electronic device 500 may be implemented as a single computing cluster or multiple computing machines that can communicate over a communication connection. Thus, the electronic device 500 can use logical connections to one or more other servers, networked personal computers (PCs), or other network nodes to operate in a networked environment.

[0105] The input device 550 may be one or more input devices, such as, for example, a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as, for example, a monitor, a speaker, a printer, etc. The electronic device 500 may communicate, as necessary, via the communication unit 540 with one or more external devices (not shown), such as a storage device, a display device, one or more devices that allow a user to interact with the electronic device 500, or any device (e.g., a network card, a modem, etc.) that allows the electronic device 500 to communicate with one or more other electronic devices. Such communication may be performed via an input / output (I / O) interface (not shown).

[0106] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium having computer-executable instructions stored thereon is provided, the computer-executable instructions being executed by a processor to perform the above-mentioned method. According to an exemplary embodiment of the present disclosure, a computer program product is also provided, the computer program product being tangibly stored on a non-transitory computer-readable medium and having computer-executable instructions, the computer-executable instructions being executed by a processor to perform the above-mentioned method.

[0107] Aspects of the present disclosure have been described herein with reference to flowcharts and / or block diagrams of methods, apparatus, devices, and computer program products implemented in accordance with the present disclosure. It will be understood that each box in the flowcharts and / or block diagrams, and combinations of boxes in the flowcharts and / or block diagrams, can be implemented by computer readable program instructions.

[0108] These computer readable program instructions can be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to create a machine such that the instructions, when executed by the processing unit of the computer or other programmable data processing apparatus, create an apparatus that performs the functions / operations specified in one or more boxes of the flowcharts and / or block diagrams. Also, by storing these computer readable program instructions, which cause a computer, programmable data processing apparatus, and / or other device to function in a particular manner, on a computer readable storage medium, the computer readable medium having the instructions stored thereon has an article of manufacture including instructions that perform each aspect of the functions / operations specified in one or more boxes of the flowcharts and / or block diagrams.

[0109] The computer readable program instructions, when loaded onto a computer, other programmable data processing apparatus, or other device, may cause a series of operational steps to be executed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowcharts and / or block diagrams.

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate architecture, functionality, and operation that may be implemented in the systems, methods, and computer program products implemented according to the present disclosure. In this regard, each box in the flowcharts or block diagrams may represent a module, program segment, or part of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the boxes may occur in a different order than those marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or may be executed in a reverse order depending on the functions involved. It should also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that executes a given function or operation, or may be implemented in a combination of dedicated hardware and computer instructions.

[0111] Although each implementation of the present disclosure has been described above, the above description is illustrative, not exhaustive, and not limited to each of the disclosed implementations. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of each of the described implementations. The selection of terms used in this specification is intended to best explain the principles, practical applications, or improvements in technology in the marketplace of each implementation, or to allow other ordinary skilled in the art to understand the various embodiments disclosed herein.

Claims

1. monitoring predictions corresponding to images in a first sequence of images acquired by an endoscope over time during an examination, the predictions indicating whether the endoscope was positioned within the body of a subject at the time the corresponding images were acquired; determining a reference image from the first sequence of images based on the monitoring, the reference image corresponding to entry of the endoscope into the inspected subject's body; acquiring a second sequence of images acquired by the endoscope after a time of acquisition of the reference image; determining a target region of the inspected object to be inspected by the endoscope based on the second image sequence; An image processing method comprising:

2. the first sequence of images includes a first image, and the corresponding prediction result includes a first prediction result for the first image; The method further includes generating a first prediction result regarding whether the endoscope is within the body of the inspected subject at the time the first image is acquired based on the trained first classification model. The method of claim 1.

3. the first sequence of images includes a second image, and the corresponding prediction result includes a second prediction result for the second image; The method comprises: extracting image features of the second image; determining a first similarity of the image feature to an internal reference feature characterizing an interior of a reference object, and a second similarity of the image feature to an external reference feature characterizing an exterior of the reference object; generating a second prediction as to whether the endoscope is located within the subject's body at the time the second image is acquired by comparing the first similarity with the second similarity; The method of claim 1 further comprising:

4. prior to determining the reference image based on the monitoring, determining a first number of adjacent images from the first sequence of images; setting a prediction result for an intermediate image among the first number of adjacent images based on the corresponding prediction result of the first number of adjacent images; The method of claim 1 further comprising:

5. Setting a prediction result for an intermediate image among the first number of neighboring images includes: determining an image from the first number of adjacent images where a corresponding prediction indicates that the endoscope is located within the subject's body; setting the prediction result for the intermediate image as an indication that the endoscope is within the inspected subject's body if the determined number of images is greater than a first threshold number related to the first number; if the determined number of images is not greater than the first threshold number, setting the prediction result for the intermediate images as indicating that the endoscope is not within the inspected subject's body; The method of claim 4 , comprising:

6. Determining a reference image from the first image sequence corresponding to entry of the endoscope into the body of the inspected subject includes: determining whether all of the predicted corresponding images of a second number of consecutive images in the first image sequence indicate that the endoscope is located within the body of the inspected subject; determining the reference image based on the second number of consecutive images in response to determining that all of the predicted corresponding images of the second number of consecutive images indicate that the endoscope is located within the body of the subject; and The method of claim 1 , comprising:

7. The method of claim 6 , wherein a capture time span corresponding to the second number of consecutive images is related to an image capture frequency of the endoscope.

8. Acquiring a second sequence of images acquired by the endoscope after a time of acquisition of the reference image includes: monitoring whether the image quality of each image acquired by the endoscope after the acquisition time of the reference image exceeds a threshold image quality; in response to monitoring that the image quality of a third image exceeds the threshold image quality, forming the second image sequence from images sequentially acquired by the endoscope within a time period following acquisition of the third image; The method of claim 1 , comprising:

9. The second image sequence satisfies at least one of the following: a collection time span corresponding to the second image sequence is equal to a threshold span; or a cumulative image distance between adjacent images in the second image sequence is greater than a threshold; The method of claim 1 , wherein the image distance is determined based on optical flow.

10. Determining a target site of the inspected object to be inspected by the endoscope includes: generating a corresponding identification for the target location for an image in the second image sequence, the identification indicating one of a plurality of predetermined locations; determining the target location based on the corresponding identification; The method of claim 1 , comprising:

11. Generating a corresponding identification of the target region for images in the second image sequence includes: generating a first classification result for a third image in the second image sequence as to which of the plurality of predetermined regions the third image corresponds to based on the trained second classification model; The method of claim 10, comprising:

12. Generating a corresponding identification of the target region for images in the second image sequence includes: for a fourth image in the second sequence of images, extracting image features of the fourth image; determining a similarity of correspondence of the image features to a plurality of reference features used to characterize each of the plurality of predetermined regions; generating a second classification result regarding which of the plurality of predetermined regions the fourth image corresponds to by comparing the similarity of the correspondence; The method of claim 10, comprising:

13. Determining the target site based on the corresponding identification result includes: determining that the target location is the first location in response to a number of identification results indicating a first location among the plurality of predetermined locations in the corresponding identification results exceeding a second threshold number; or determining that the target site is the second site in response to a proportion of the identification results indicating a second site among the plurality of predetermined sites in the corresponding identification results exceeding a threshold proportion; The method of claim 1 , comprising one of:

14. Electronic device comprising at least one processing circuit configured to carry out the method according to any one of claims 1 to 13.

Images