Endoscope observation area tracking method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an endoscope observation area tracking method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring first feature information of a reference observation area in a reference endoscope image; the reference endoscope image comprises a target observation object; obtaining a to-be-detected endoscopic image, wherein the to-be-detected endoscopic image comprises a target observation object; determining at least one candidate observation area from the to-be-detected endoscopic image; for each candidate observation area in the at least one candidate observation area, obtaining second feature information of the candidate observation area; the second feature information is consistent with the type of features contained in the first feature information; calculating the similarity between the candidate observation area and the reference observation area based on the second feature information and the first feature information; and based on the similarity corresponding to the at least one candidate observation area, determining a target observation area containing the same target observation object as the reference observation area. And automatic tracking and identification of the observation area in the endoscopic image can be realized.

Description

Technical Field

[0001] The present application relates to the technical field of endoscopes, and more specifically to an endoscope observation area tracking method, an endoscope observation area tracking system, an electronic device and a non-volatile storage medium. Background Art

[0002] Endoscopes are a widely used medical instrument. Endoscopes send imaging elements into the body through the natural cavity of the organism and return images of the internal tissues of the organism. Using these images, the internal parts of the natural cavity of the organism can be inspected.

[0003] When performing an endoscopic examination, it is necessary to determine the object that needs to be observed in the endoscopic image (i.e., the target observation object) so as to analyze the target observation object to understand the internal situation of the organism. In the related art, the target observation object in the image is generally screened by the human eye. Since the biological cavity is tortuous, and different observation angles often need to be changed during the use of the endoscope. During the peristalsis of the biological cavity and / or the change of the endoscope angle, the position of the target observation object in the image will also change. The user needs to screen the target observation object in each frame of the image, which seriously affects the efficiency of the endoscopic examination. Summary of the invention

[0004] The present application is proposed in view of the above problems. The present application provides an endoscope observation area tracking method, an endoscope observation area tracking system, an electronic device and a non-volatile storage medium.

[0005] According to one aspect of the present application, an endoscopic observation area tracking method is provided, comprising: obtaining first feature information of a baseline observation area in a baseline endoscopic image; the baseline endoscopic image contains a target observation object, the baseline observation area is the area where the target observation object is located, and the first feature information includes at least one of the following features: a rotational invariance feature, a color feature, and a directional gradient feature; obtaining an endoscopic image to be tested, the endoscopic image to be tested containing the target observation object; determining at least one candidate observation area from the endoscopic image to be tested; for each candidate observation area in at least one candidate observation area, obtaining second feature information of the candidate observation area; the second feature information is consistent with the type of feature contained in the first feature information; based on the second feature information and the first feature information, calculating the similarity between the candidate observation area and the baseline observation area; based on the similarity corresponding to each of the at least one candidate observation area, determining the target observation area, the target observation area being a candidate observation area containing the same target observation object as the baseline observation area.

[0006] Exemplarily, each of the at least one candidate observation area has the same size as the benchmark observation area, the benchmark observation area includes a plurality of first sub-areas, the first feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of first sub-areas, each candidate observation area includes a plurality of second sub-areas, the plurality of first sub-areas correspond one-to-one to the plurality of second sub-areas, and the corresponding first sub-areas and second sub-areas have the same size, the second feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of second sub-areas, wherein based on the second feature information and the first feature information, calculating the similarity between the candidate observation area and the benchmark observation area includes: calculating the similarity between the candidate observation area and the benchmark observation area based on the feature information corresponding to each second sub-area in the candidate observation area and the benchmark observation area; The method comprises the following steps: determining the target observation area based on the feature information corresponding to the corresponding first sub-area in the candidate observation area, calculating the sub-similarity between each second sub-area in the candidate observation area and the corresponding first sub-area in the benchmark observation area, wherein the similarity includes the sub-similarity corresponding to each second sub-area in the candidate observation area one by one; and determining the target observation area based on the similarity corresponding to at least one candidate observation area, comprising: for each candidate observation area in at least one candidate observation area, determining the target second sub-area whose sub-similarity corresponding to the candidate observation area is greater than the similarity threshold, and determining the candidate observation area as the target observation area containing the same target observation object as the benchmark observation area when the target second sub-area in the candidate observation area meets the target requirement.

[0007] Exemplarily, the target requirement includes: the number of target second sub-regions in the candidate observation region exceeds a preset number threshold, or the proportion of the area of ​​the target second sub-region in the candidate observation region to the area of ​​the candidate observation region exceeds a preset proportion threshold.

[0008] Exemplarily, determining the target observation area based on the similarities corresponding to at least one candidate observation area also includes: when none of the target second sub-areas in at least one candidate observation area meets the target requirements, selecting the candidate observation area with the largest number of target second sub-areas from at least one candidate observation area as the target observation area.

[0009] Exemplarily, obtaining first characteristic information of a baseline observation area in a baseline endoscopic image includes: obtaining a baseline endoscopic image; determining an initial observation area on the baseline endoscopic image, the initial observation area being an area containing a target observation object; extending the boundaries of the initial observation area to obtain a baseline observation area; and extracting features of the baseline observation area to obtain first characteristic information.

[0010] Exemplarily, extending the boundaries of the initial observation area to obtain the reference observation area includes: taking the center of the initial observation area as the center, and extending the boundaries of the initial observation area based on a preset multiple to obtain the reference observation area.

[0011] Exemplarily, determining at least one candidate observation area from the endoscopic image to be tested includes: dividing at least one candidate observation area on the endoscopic image to be tested; wherein the size of each candidate observation area in the at least one candidate observation area is the same as the size of the baseline observation area; when the number of candidate observation areas in the at least one candidate observation area is multiple, the multiple candidate observation areas do not overlap with each other; or, obtaining at least one candidate observation area by sliding a sliding window on the endoscopic image to be tested.

[0012] Exemplarily, the first feature information includes rotational invariance features, color features, and directional gradient features; obtaining the first feature information of a reference observation area in a reference endoscopic image includes: respectively obtaining the rotational invariance features, color features, and directional gradient features in the reference observation area; aggregating the rotational invariance features, color features, and directional gradient features to obtain the first feature information.

[0013] Exemplarily, the method also includes: respectively acquiring the image parameters of the baseline observation area and the image parameters of the target observation area; judging whether the image parameters of the target observation area meet preset requirements, wherein the preset requirements include: the image parameter difference between the image parameters of the target observation area and the image parameters of the baseline observation area is less than or equal to a preset difference threshold; when the image parameters of the target observation area do not meet the preset requirements, adjusting the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements.

[0014] Exemplarily, the image parameters include clarity; the preset requirements include a first preset requirement; the preset difference threshold includes a first difference threshold; determining whether the image parameters of the target observation area meet the preset requirements includes: determining whether the difference between the clarity of the target observation area and the clarity of the benchmark observation area is less than the first difference threshold; if the difference between the clarity of the target observation area and the clarity of the benchmark observation area is less than the first difference threshold, determining that the image parameters of the target observation area meet the first preset requirements.

[0015] Exemplarily, the imaging parameters include focusing parameters of an image acquisition device in an endoscope system. When the image parameters of a target observation area do not meet preset requirements, the imaging parameters of the endoscope system are adjusted so that the image parameters of the target observation area in subsequent endoscopic images to be tested meet the preset requirements, including: when the image parameters of the target observation area do not meet a first preset requirement, repeatedly performing the following steps until the image parameters of the target observation area in any subsequent endoscopic image to be tested meet the first preset requirement: adjusting the focusing parameters of the image acquisition device based on a preset focusing step to control the focusing of the image acquisition device; and determining whether the image parameters of the target observation area in subsequent endoscopic images to be tested obtained after focusing meet the first preset requirement.

[0016] Exemplarily, the image acquisition device includes a deformable lens driven by an electrical signal, and adjusting the focus parameter of the image acquisition device based on a preset focusing step includes: adjusting the electrical signal applied to the deformable lens based on the preset focusing step.

[0017] Exemplarily, the image parameters include brightness; the preset requirements include a second preset requirement; the preset difference threshold includes a second difference threshold; determining whether the image parameters of the target observation area meet the preset requirements includes: determining whether the difference between the brightness of the target observation area and the baseline brightness is less than the second difference threshold; if the difference between the brightness of the target observation area and the baseline brightness is less than the second difference threshold, determining that the image parameters of the target observation area meet the second preset requirements.

[0018] Exemplarily, the imaging parameters include one or more of the following: the exposure time of an image acquisition device in the endoscope system, the analog gain of the image acquisition device, and the light intensity of an illumination device in the endoscope system; when the image parameters of the target observation area do not meet the preset requirements, adjusting the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, including: when the image parameters of the target observation area do not meet the second preset requirements, performing one or more of the following operations: exposure time adjustment operation: adjusting the exposure time of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted exposure time meet the second preset requirements or the adjusted exposure time reaches the exposure time threshold; light intensity adjustment operation: adjusting the light intensity of the illumination device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted light intensity meet the second preset requirements or the adjusted light intensity reaches the light intensity threshold; analog gain adjustment operation: adjusting the analog gain of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted analog gain meet the second preset requirements or the adjusted analog gain reaches the analog gain threshold.

[0019] Exemplarily, the exposure time adjustment operation is performed first, the light intensity adjustment operation is performed when the adjusted exposure time reaches the exposure time threshold and the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted exposure time do not meet the second preset requirement, and the analog gain adjustment operation is performed when the adjusted light intensity reaches the light intensity threshold and the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted light intensity do not meet the second preset requirement.

[0020] According to another aspect of the present application, an endoscopic observation area tracking system is also provided, comprising: a first acquisition module: used to acquire first feature information of a reference observation area in a reference endoscopic image; the reference endoscopic image contains a target observation object, the reference observation area is the area where the target observation object is located, and the first feature information includes at least one of the following features: a rotational invariance feature, a color feature, and a directional gradient feature; a second acquisition module: used to acquire an endoscopic image to be tested, the endoscopic image to be tested contains the target observation object; a first determination module: used to determine at least one candidate observation area from the endoscopic image to be tested; a third acquisition module: used to acquire second feature information of each candidate observation area in at least one candidate observation area; the second feature information is consistent with the type of feature contained in the first feature information; a calculation module: used to calculate the similarity between the candidate observation area and the reference observation area for each candidate observation area in at least one candidate observation area based on the second feature information and the first feature information; a second determination module: used to determine the target observation area based on the similarity corresponding to each of the at least one candidate observation areas, the target observation area being a candidate observation area containing the same target observation object as the reference observation area.

[0021] According to another aspect of the present application, there is also provided an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned endoscopic observation area tracking method when the processor is running.

[0022] According to another aspect of the present application, a non-volatile storage medium is provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned endoscopic observation area tracking method when running.

[0023] The above-mentioned endoscopic observation area tracking method calculates the similarity between each candidate observation area and the reference observation area by using the first feature information and the second feature information, and determines whether the candidate observation area and the reference observation area contain the same target observation object based on the similarity, thereby realizing automatic tracking and identification of the target observation object in different endoscopic images. This scheme helps to improve the efficiency of endoscopic inspection. In this technical scheme, the first feature information includes at least one of the rotational invariance feature, color feature and directional gradient feature. By using at least one of the rotational invariance feature, color feature and directional gradient feature of the reference observation area as the judgment basis for determining the target observation area in the subsequent steps, it helps to more accurately determine the target observation area in the endoscopic image to be tested.

[0024] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 A schematic flow chart showing an endoscope observation area tracking method according to an embodiment of the present application;

[0027] Figure 2 A schematic diagram showing a reference endoscopic image according to an embodiment of the present application;

[0028] Figure 3 A schematic flow chart showing an endoscope observation area tracking method according to a specific embodiment of the present application;

[0029] Figure 4 A schematic diagram showing a direction-sensitive feature according to an embodiment of the present application;

[0030] Figure 5 A schematic diagram showing a direction-insensitive feature according to an embodiment of the present application;

[0031] Figure 6 A schematic diagram showing an endoscopic image to be tested according to an embodiment of the present application;

[0032] Figure 7 A schematic flow chart of obtaining image parameters according to an embodiment of the present application is shown;

[0033] Figure 8 A schematic flow chart showing an endoscope observation area tracking method according to another specific embodiment of the present application;

[0034] Fig. 9 A schematic flow chart showing an endoscope observation area tracking method according to another specific embodiment of the present application;

[0035] Fig.10 A schematic block diagram showing an endoscope observation area tracking system according to an embodiment of the present application; and

[0036] Fig.11A schematic block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present application more obvious, the example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments described in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0038] An endoscope (i.e., "endoscope") can send an imaging element into a living body through the natural cavity of the living body, and return an image of the internal tissue of the living body, thereby realizing the inspection of the internal tissue of the living body. Among them, when there is an object that needs to be observed in the internal tissue of the living body (i.e., a target observation object, such as a lesion, an area where a lesion may exist, etc.), the user usually needs to conduct a detailed inspection around the target observation object. Since the biological cavity is tortuous, and when the user identifies the target observation object from the image of the body cavity, the user usually changes the observation angle of the endoscope by axially moving the distal end of the endoscope or rotating the distal end in the cavity to obtain more information about the target observation object. During the peristalsis of the biological cavity and / or the change of the angle of the endoscope, the position of the target observation object in the image will also change. The traditional positioning of the target observation object depends on the doctor's observation experience, but since the position of the target observation object in each frame of the image may not be the same, the user is required to re-search and locate the target observation object in each frame of the image, which seriously affects the efficiency of endoscopic inspection. And because the amount of information in the collected images is large, it is affected by the endoscopic inspection operation, observation time, the quality of the pictures taken by the camera, the similarity between the target observation object and the surrounding tissues, and the observer's recognition ability, resulting in inaccurate positioning of the target observation object in the image. In view of this, the present application provides an endoscopic observation area tracking method, an endoscopic observation area tracking system, an electronic device, and a non-volatile storage medium. The tracking method can more accurately identify the area where the target observation object is located in each frame of the endoscopic image, which helps to improve the efficiency of endoscopic inspection.

[0039] According to one aspect of the present application, an endoscopic observation area tracking method is provided. Figure 1 FIG. 2 is a schematic flow chart of an endoscope observation area tracking method according to an embodiment of the present application. Figure 1 As shown, the endoscopic observation area tracking method 100 may include but is not limited to the following steps S110, S120, S130, S140, S150 and S160.

[0040] In step S110, first feature information of a reference observation area in a reference endoscopic image is obtained; the reference endoscopic image includes a target observation object, the reference observation area is the area where the target observation object is located, and the first feature information includes at least one of the following features: rotational invariance feature, color feature and directional gradient feature.

[0041] Optionally, the target observation object may be a specific area in any specific part of the photographed cavity. The specific part may be the fundus of the stomach, the body of the stomach, the antrum of the stomach, the ascending colon, the transverse colon, the descending colon, etc. The specific area may be any area that the user needs to focus on observing, for example, it may be an area where a lesion occurs or may exist in a specific part. In a specific embodiment, the target observation object may be a tumor in the fundus of the stomach.

[0042] The reference observation area is the area where the target observation object is located. Optionally, the reference observation area can be determined by any existing or future developed area recognition method. For example, the user can mark the reference observation area in the reference endoscopic image based on experience. For another example, the reference observation area can be identified in the reference endoscopic image by a learned neural network model.

[0043] Optionally, the area of ​​the reference observation area may be smaller than the area of ​​the reference endoscopic image. It is understandable that the reference endoscopic image includes more image information, and only part of the image information may be related to the target observation object. In this embodiment, since the reference observation area is smaller, the image information obtained through the reference observation area contains less image information irrelevant to the target observation object, thereby helping to reduce the interference of irrelevant information on the target observation object.

[0044] In this solution, at least one of the rotation invariance feature, the color feature and the directional gradient feature is selected as the first feature information of the reference observation area, so as to be used for determining the target observation area in the endoscopic image to be tested in the subsequent steps. In some embodiments, the first feature information may include the rotation invariance feature. It can be understood that the endoscopic system often needs to change the angle during the observation process, which will cause the target observation object in the currently acquired endoscopic image to be tested to rotate at various angles. In image processing, some basic features may be difficult to maintain after the image is rotated. Therefore, the features with rotation invariance in the reference observation area can be extracted to ensure the accuracy of the determined target observation object. In other embodiments, the first feature information may include the color feature. When inspecting the target observation object in the internal tissue of a biological body (for example, for the inspection of the lesion area), the most important basis for judging whether the target observation object exists is the color difference between the target observation object and other parts. In this embodiment, by extracting the color feature in the reference observation area, the accuracy of the target observation area determined in the subsequent steps can be improved. In some other embodiments, the first feature information may include the directional gradient feature. When inspecting the target observation object in the internal tissue of a biological body, the texture (i.e., the directional gradient feature) of the target observation object relative to other parts is different. Taking the target observation object as the lesion area as an example, the texture change of the lesion area is the focus of clinical attention. In most cases, texture changes mean cell proliferation and there may be a risk of early cancer. Therefore, the texture of the reference observation area can be used as a basis for judging the target observation area in the endoscopic image to be tested. In this embodiment, by obtaining the directional gradient characteristics of the reference observation area, a more accurate basis can be provided for determining the target observation area in the subsequent steps.

[0045] In step S120, an endoscopic image to be tested is acquired, where the endoscopic image to be tested includes a target observation object.

[0046] Optionally, the endoscopic image to be tested can be acquired during the same endoscopic examination as the reference endoscopic image. The reference endoscopic image can be an image of the target observation object in any frame (e.g., the first frame) acquired during the endoscopic examination, and the endoscopic image to be tested can be an image of the target observation object in any frame acquired after the reference endoscopic image. In one embodiment, when the endoscope penetrates deep into the cavity being photographed and the target observation object is included in the acquired image, the image can be the reference endoscopic image. The image containing the target observation object obtained after the image is the endoscopic image to be tested.

[0047] Alternatively, the endoscopic image to be tested and the reference endoscopic image may be acquired during different endoscopic examinations. For example, the reference endoscopic image may be an endoscopic image containing the target observation object acquired by an endoscope before the endoscopic examination corresponding to the endoscopic image to be tested.

[0048] In step S130, at least one candidate observation region is determined from the endoscopic image to be tested.

[0049] Optionally, the size of each candidate observation area in the at least one candidate observation area may be the same as the size of the reference observation area. In this embodiment, the number of candidate observation areas in the at least one candidate observation area may be determined based on the area of ​​the endoscopic image to be tested and the area of ​​the candidate observation area. The method for determining at least one candidate observation area in the endoscopic image to be tested is described in detail below.

[0050] In step S140, for each candidate observation area in the at least one candidate observation area, second feature information of the candidate observation area is obtained. The second feature information is consistent with the type of feature included in the first feature information.

[0051] Optionally, the first feature information and the second feature information may both include at least one of a rotation invariant feature, a color feature, and a directional gradient feature. For example, the first feature information and the second feature information may both be rotation invariant features. For another example, the first feature information and the second feature information may both include a rotation invariant feature, a color feature, and a directional gradient feature.

[0052] Optionally, the second feature information can be obtained by any existing or future developed feature acquisition method. For example, when the second feature information includes rotational invariant features, the rotational invariant features of the candidate observation area can be extracted by a scale-invariant feature transform (SIFT) algorithm. When the second feature information includes color features, a color name (CN) feature extraction algorithm can be used to extract the color features of the candidate observation area. When the second feature information includes directional gradient features, a histogram of directional gradients (FHOG) feature extraction algorithm can be used to extract the directional gradient features of the candidate observation area. The specific feature extraction method is described in detail below.

[0053] In step S150 , for each candidate observation region of the at least one candidate observation region, the similarity between the candidate observation region and the reference observation region is calculated based on the second feature information and the first feature information.

[0054] The similarity between the candidate observation area and the reference observation area can be calculated by any existing or future similarity calculation method. For example, the similarity between the second feature information and the first feature information can be calculated by a similarity calculation method such as Euclidean distance as the similarity between the candidate observation area and the reference observation area.

[0055] In step S160 , based on the similarity corresponding to at least one candidate observation region, a target observation region is determined, where the target observation region is a candidate observation region that includes the same target observation object as the reference observation region.

[0056] For ease of description, in the following, the candidate observation area that contains the same target observation object as the reference observation area is referred to as the target observation area. It can be understood that for any candidate observation area in at least one candidate observation area, when the similarity corresponding to the candidate observation area is high, the possibility that the candidate observation area and the reference observation area contain the same target observation object is high. Therefore, it is possible to more accurately determine whether each candidate observation area in at least one candidate observation area is a target observation area based on the similarity corresponding to each of the at least one candidate observation areas. This solution helps to improve the accuracy of the target observation area in the endoscopic image to be tested.

[0057] In the above technical solution, the similarity between each candidate observation area and the reference observation area is calculated by using the first feature information and the second feature information, and whether the candidate observation area and the reference observation area contain the same target observation object is determined based on the similarity, thereby realizing automatic tracking and identification of the target observation object in different endoscopic images. This solution helps to improve the efficiency of endoscopic inspection. In this technical solution, the first feature information includes at least one of a rotational invariance feature, a color feature, and a directional gradient feature. By using at least one of the rotational invariance feature, the color feature, and the directional gradient feature of the reference observation area as a judgment basis for determining the target observation area in the subsequent steps, it helps to more accurately determine the target observation area in the endoscopic image to be tested.

[0058] Exemplarily, each of the at least one candidate observation area is the same size as the baseline observation area, the baseline observation area includes multiple first sub-areas, the first feature information includes multiple groups of feature information corresponding one-to-one to the multiple first sub-areas, each candidate observation area includes multiple second sub-areas, the multiple first sub-areas correspond one-to-one to the multiple second sub-areas and the corresponding first sub-areas and second sub-areas are the same size, and the second feature information includes multiple groups of feature information corresponding one-to-one to the multiple second sub-areas.

[0059] Calculating the similarity between the candidate observation area and the benchmark observation area based on the second feature information and the first feature information includes: calculating the sub-similarity between each second sub-area in the candidate observation area and the corresponding first sub-area in the benchmark observation area based on the feature information corresponding to each second sub-area in the candidate observation area and the feature information corresponding to the corresponding first sub-area in the benchmark observation area. The similarity between the candidate observation area and the benchmark observation area may specifically include the sub-similarity corresponding to each second sub-area in the candidate observation area.

[0060] Based on the similarities corresponding to at least one candidate observation area, a target observation area is determined, including: for each candidate observation area in the at least one candidate observation area, a target second sub-area whose corresponding sub-similarity is greater than a similarity threshold is determined; when the target second sub-area in the candidate observation area meets the target requirements, the candidate observation area is determined to be a target observation area that contains the same target observation object as the benchmark observation area.

[0061] Optionally, the size of the first sub-region can be selected as needed. It is understandable that if the first sub-region is too large, the extracted feature information may not be able to well characterize the features of the target observation object. If the first sub-region is too small, it may result in too many first sub-regions to be processed in the reference observation area, resulting in excessive calculations, which affects the detection efficiency. Therefore, the first sub-region of a suitable size can be selected by comprehensively considering the feature extraction accuracy and calculation efficiency. In a specific embodiment, the size of the first sub-region can be 16 pixels * 16 pixels.

[0062] Figure 2 A schematic diagram of a reference endoscopic image according to an embodiment of the present application is shown. As shown in the figure, the size of the reference observation region (ROI) is 96 pixels * 96 pixels, and the size of the first sub-region is 16 pixels * 16 pixels. In this embodiment, the reference observation region includes 36 first sub-regions. The division method of the second sub-region in the candidate observation region is similar to the division method of the first sub-region in the reference observation region, and will not be repeated.

[0063] The first sub-region corresponds to the second sub-region one-to-one. The sub-similarity between each first sub-region and the corresponding second sub-region can be calculated. Optionally, the sub-similarity can be obtained by using any existing or future developed algorithm for calculating the similarity between images. For example: cosine similarity algorithm, Gaussian similarity algorithm, etc. In one embodiment, the Gaussian similarity algorithm can be used to calculate the sub-similarity between the second sub-region and the corresponding first sub-region. In a specific embodiment, the sub-similarity K between the second sub-region and the corresponding first sub-region X1X2 It can be calculated using the following formula:

[0064]

[0065] Wherein, X1 represents the characteristic information of the first sub-region. X2 represents the characteristic information of the second sub-region. e is a natural constant. σ 2 are the parameters of the Gaussian similarity algorithm.

[0066] Optionally, the similarity threshold may be determined based on an empirical value. For example, the similarity threshold may be 0.5. In the above embodiment in which the Gaussian similarity algorithm is used to calculate the sub-similarity, when the corresponding sub-similarity is greater than 0.5, the second sub-region may be the target second sub-region.

[0067] As shown above, the feature information of the second sub-region may include multiple feature information. For example, the feature information of the second sub-region may include rotational invariance features, color features, and directional gradient features. Optionally, when the feature information of the second sub-region includes multiple feature information, the sub-similarity corresponding to each feature information may be calculated separately, and when the sub-similarity corresponding to any one of the multiple feature information is greater than the similarity threshold, the second sub-region is determined to be the target second sub-region. Alternatively, when the feature information of the second sub-region includes multiple feature information, the sub-similarity corresponding to each feature information may be calculated separately, and when the sub-similarity corresponding to each of the multiple feature information is greater than the similarity threshold, the second sub-region is determined to be the target second sub-region.

[0068] In the above technical solution, the similarity between the candidate observation area and the reference observation area is represented by the sub-similarity between each second sub-area and the corresponding first sub-area, and the corresponding candidate observation area is judged whether it contains the same target observation object as the reference observation area based on the similarity. This solution helps to effectively utilize the local information in the image by dividing the reference observation area into multiple first sub-areas and the candidate observation area into multiple second sub-areas. By comparing the similarity between the candidate observation area and the reference observation area based on local features, it helps to improve the accuracy of the judgment result.

[0069] Exemplarily, the target requirement may include: the number of target second sub-regions in the candidate observation region exceeds a preset number threshold, or the proportion of the area of ​​the target second sub-region in the candidate observation region to the area of ​​the candidate observation region exceeds a preset proportion threshold.

[0070] In one embodiment, the target requirement may include: the number of target second sub-areas in the candidate observation area exceeds a preset number threshold. The target requirement may be referred to as a number requirement. The preset number threshold may be determined based on the total number of second sub-areas in the candidate observation area. For example, if the total number of second sub-areas is 36, the preset number threshold may be any integer in the range of [25,36]. Specifically, for example, the preset number threshold may be 29.

[0071] In one embodiment, the target requirement may include: the ratio of the area of ​​the target second sub-region in the candidate observation area to the area of ​​the candidate observation area exceeds a preset ratio threshold. The target requirement may be referred to as a ratio requirement. The preset ratio threshold may be set as needed. In a specific embodiment, the preset ratio threshold may be 0.8. In this embodiment, the ratio of the area of ​​the target second sub-region to the area of ​​the candidate observation area may be represented by the ratio of the number of target second sub-regions to the total number of second sub-regions in the candidate observation area.

[0072] Figure 3 FIG. 2 is a schematic flow chart of an endoscope observation area tracking method according to a specific embodiment of the present application. Figure 3 As shown, first, step S310 is executed, image segmentation, to divide the endoscope image to be tested into at least one candidate observation area. Then, for each candidate observation area in the at least one candidate observation area, step S320 is executed, sub-area feature calculation, to calculate the feature information corresponding to each second sub-area in the candidate observation area. It can be understood that the candidate observation area can include multiple second sub-areas. Figure 3 In the illustrated embodiment, the number of second sub-regions is 144. Next, for each second sub-region, execute steps S330 to S370. In step S330, let i=1. In this step, the second sub-regions can be sorted in sequence. i=1 represents the first second sub-region in the candidate observation area. In step S340, the sub-similarity corresponding to the i-th second sub-region is calculated. In this step, the sub-similarity can be calculated according to the Gaussian similarity algorithm based on the feature information of the i-th second sub-region and the feature information of the corresponding i-th first sub-region. In step S350, the sub-similarity K is determined. XX Is it greater than the similarity threshold of 0.5? XX If the similarity K is greater than the similarity threshold 0.5, then step S360 is executed, and the number of the second sub-regions of the target N=N+1. Then step S370 is executed. XX If it is less than or equal to the similarity threshold of 0.5, then directly execute step S370. In step S370, determine whether i is greater than 144, that is, determine whether there is a second sub-region whose sub-similarity has not been calculated. If there is a second sub-region whose sub-similarity has not been calculated, set i++ and continue to execute steps S330 to S370 to calculate the sub-similarity corresponding to the next second sub-region. If the sub-similarity of all second sub-regions has been calculated, execute step S380 to determine whether the proportion of the area of ​​the target second sub-region in the candidate observation area to the area of ​​the candidate observation area exceeds the preset proportion threshold. Figure 3In the illustrated embodiment, the preset ratio threshold is 0.8, and the ratio of the area of ​​the target second sub-region to the area of ​​the candidate observation region can be represented by the ratio of the number of target second sub-regions to the total number of second sub-regions in the candidate observation region. That is, it can be determined whether N / 144 is greater than 0.8. If N / 144>0.8, it is determined to execute step S390 and mark the tracking region, that is, the candidate observation region can be marked as a target observation region containing the same target observation object as the reference observation region.

[0073] In the above technical solution, by using the quantity requirement or the ratio requirement, it is possible to more accurately determine whether the candidate observation area and the reference observation area contain the same target observation object. This solution helps to further improve the accuracy of the determination.

[0074] Exemplarily, determining the target observation area based on the similarities corresponding to at least one candidate observation area may also include the following steps: when none of the target second sub-areas in at least one candidate observation area meets the target requirements, selecting the candidate observation area with the largest number of target second sub-areas from at least one candidate observation area as the target observation area.

[0075] It can be understood that the position of the target observation object in the image will change during the peristalsis of the biological cavity and / or the change of the endoscope angle. In some endoscopic images, the target observation object may be located at the junction of multiple candidate observation areas in the endoscopic image. There may not be a candidate observation area in each candidate observation area whose target second sub-area meets the target requirements. In this example, the number of target second sub-areas of each candidate observation area in at least one candidate observation area can be compared, and the candidate observation area with the largest number of target second sub-areas can be selected as the target observation area. It can be understood that for a candidate observation area, when the number of target second sub-areas it includes is large, the candidate observation area can relatively accurately characterize the characteristics of the target observation object. In this scheme, by using the candidate observation area with the largest number of target second sub-areas as the target observation area, it helps to ensure that the target observation area more accurately characterizes the characteristics of the target observation object.

[0076] Exemplarily, step S110, acquiring first feature information of a reference observation area in a reference endoscopic image, may include but is not limited to the following steps S111, S112, and S113.

[0077] In step S111 , a reference endoscopic image is acquired.

[0078] In step S112, an initial observation area on the reference endoscopic image is determined. The initial observation area is an area including the target observation object.

[0079] In step S113, the boundary of the initial observation area is expanded to obtain a reference observation area.

[0080] In step S114, feature extraction is performed on the reference observation area to obtain first feature information.

[0081] As described above, the reference endoscopic image can be acquired during the same endoscopic examination as the endoscopic image to be tested, or can be acquired during a different endoscopic examination than the endoscopic image to be tested. The specific acquisition method has been described in detail above. For the sake of brevity, it will not be repeated here.

[0082] Optionally, the initial observation area on the reference endoscopic image may be determined by user input information. Alternatively, the initial observation area may be automatically determined on the reference endoscopic image using an auxiliary diagnosis device.

[0083] The user's input information may be identification information of the observation area or the area outline drawn by the user on the reference endoscopic image. Optionally, the user may select the initial observation area by selecting or inputting the identification information of the observation area. For example, the reference endoscopic image may be pre-divided into different observation areas, and each observation area may have different identification information. The user may select or input the identification information corresponding to any divided observation area by mouse click and / or keyboard input. In response to the user's operation, the observation area selected by the user may be used as the initial observation area. In this embodiment, the identification information input by the user is the input information. Alternatively, the user may also select the initial observation area by marking the outline of the initial observation area on the reference endoscopic image. For example, the user may draw the outline of any area in the reference endoscopic image by means of a mouse and / or keyboard and / or touch screen, and the area corresponding to the outline is the initial observation area. In this embodiment, the outline drawn by the user is the user's input information. Of course, the above-mentioned determination method of the initial observation area is only an example, and other suitable methods may be used to determine the initial observation area.

[0084] Optionally, the boundary of the initial observation area can be extended based on the position of the initial observation area in the reference endoscopic image. In some embodiments, the reference endoscopic image can be pre-divided into a plurality of regions of equal size according to a preset number, and the size of each region is larger than the size of the initial observation area. Then, it is determined which region the initial observation area belongs to in the reference endoscopic image, and the region to which the initial observation area belongs in the reference endoscopic image is the reference observation area obtained after the boundary of the initial observation area is extended. Alternatively, the boundary of the initial observation area can be extended based on the type of the target observation object. For example, assuming that the target observation object is the inflammation area of ​​the gastric body, since the inflammation area is usually dispersed, the boundary of the initial observation area can be extended to a larger range to avoid missing some inflammation areas. For another example, assuming that the target observation object is a calculus in the bladder, since the calculus is generally concentrated, the boundary of the initial observation area can be extended to a smaller range. Alternatively, the boundary of the initial observation area can be extended directly with any point on the initial observation area as the center and any multiple greater than 1. For example, any point on the initial observation area can be any point among the boundary point, center point, etc. of the initial observation area.

[0085] In this example, it can be considered that the initial observation area only includes image information related to the target observation object. It can be understood that the position of the target observation object in the endoscopic image is constantly changing due to the influence of the peristalsis of the biological cavity and the change of the endoscope angle. And because it is sometimes necessary to supply water and air to the target observation object during the endoscopic examination, the target observation object may undergo significant changes or be obscured. In this example, by expanding the boundaries of the initial observation area, the image information around the target observation object can be introduced as the basis for the subsequent determination of the target observation area, which helps to more accurately determine the target observation area in the endoscopic image to be tested and ensure the tracking effect of the target observation object.

[0086] Similar to the method of obtaining the second characteristic information, the first characteristic information can also be obtained by using any existing or future developed characteristic acquisition method. The specific acquisition method will be described in detail below.

[0087] In the above technical scheme, by determining the initial observation area in the baseline endoscopic image and expanding the boundaries of the initial observation area to obtain the baseline observation area, characteristic information around the target observation object can be introduced as a basis for subsequent determination of the target observation area, which helps to improve the accuracy of the target observation area determined in the subsequent step (determination of the target observation area in the endoscopic image to be tested).

[0088] Exemplarily, step S113, expanding the boundaries of the initial observation area to obtain a reference observation area, may include the following steps: taking the center of the initial observation area as the center, expanding the boundaries of the initial observation area based on a preset multiple to obtain a reference observation area.

[0089] Optionally, the preset multiple can be set as needed. For example, the size of the initial observation area can be expanded to an integer multiple of the size of the first sub-area according to the size of the first sub-area. In a specific embodiment, the size of the first sub-area is 16 pixels*16 pixels, and the length and width of the initial observation area can be expanded to an integer multiple of 16.

[0090] In one embodiment, the boundary of the initial observation area is expanded based on a preset multiple with the center of the initial observation area as the center, which may include the following steps: First, record the coordinates (x0, y0) of the upper left corner of the initial observation area, as well as the length l and width w of the initial observation area. Then, based on the preset multiple c, calculate the length L and width W of the expanded reference observation area. Next, with the center coordinates of the initial observation area as the origin, expand the initial observation area c times to obtain the reference observation area. The coordinates of the upper left corner of the reference observation area are (x'0, y'0). L, W, and the coordinates of the upper left corner of the reference observation area can be calculated by the following formula:

[0091]

[0092] In the above formula, floor() is a floor rounding function. In a specific embodiment, the preset multiple c is 3. The coordinates of the upper left corner of the initial observation area are (200,300), and the area size is 65 pixels * 50 pixels. After calculation by the above formula, the size of the expanded reference observation area is 192 pixels * 144 pixels, and the coordinates of the upper left corner are (132,252).

[0093] In the above technical solution, the boundary of the initial observation area is directly expanded with the center of the initial observation area as the center, which has simple steps and strong applicability.

[0094] Exemplarily, the first feature information includes a rotation invariant feature, a color feature, and a directional gradient feature. Step S110, obtaining the first feature information of the reference observation area in the reference endoscopic image, may include the following steps: respectively obtaining the rotation invariant feature, the color feature, and the directional gradient feature in the reference observation area; aggregating the rotation invariant feature, the color feature, and the directional gradient feature to obtain the first feature information.

[0095] In one embodiment, obtaining the rotation invariant features in the reference observation area may include obtaining the rotation invariant features of each first sub-area in the reference observation area. For a first sub-area, obtaining the rotation invariant features of the first sub-area may include the following steps: first, performing step a) scale space extremum detection. In this step, images in all scale spaces of the reference observation area are searched, and potential scale and rotation invariant interest points are identified by Gaussian differential functions. Then, step b) feature point positioning is performed. In this step, the position scale can be determined by a fitting fine model at each candidate position, and the selection of key points depends on their stability. Next, step c) feature direction assignment is performed. Based on the local gradient direction of the image, one or more directions are assigned to each key point position, and all subsequent operations are to transform the direction, scale and position of the key point, thereby providing the invariance of these features. Finally, step d) feature point description is performed. In this step, the local gradient of the image is measured at a selected scale in the neighborhood around each feature point, and these gradients are transformed into a representation that allows relatively large local shape deformation and illumination changes. By describing the feature points, the 128-dimensional SIFT features of the first sub-region can be obtained.

[0096] In one embodiment, obtaining the color features in the reference observation area may include obtaining the color features of each first sub-area in the reference observation area. For a first sub-area, the features in the RGB space may be mapped to 11-dimensional (black, yellow, brown, gray, green, orange, pink, purple, red, white, yellow) CN features by calling a mapping matrix between the trained RGB space and the CN space.

[0097] In one embodiment, obtaining the directional gradient feature in the reference observation area may include obtaining the directional gradient feature of each first sub-area in the reference observation area. For a first sub-area, obtaining the directional gradient feature of the first sub-area may include the following steps:

[0098] Step a) Color space normalization. In this step, the first sub-region can be converted from a color image to a grayscale image. The normalization process can be implemented by the following formula:

[0099] Gray=0.3*R+0.59*G+0.11*B

[0100] In the formula, Gray represents grayscale, R represents red, G represents green, and B represents blue.

[0101] Step b) Calculate the gradient. For the image of the first sub-region that has been normalized in color space, obtain its gradient and gradient direction, calculate in the horizontal direction and vertical direction respectively, and calculate the magnitude and direction of the gradient. The horizontal gradient G can be calculated by the following formula: x (x,y) and the vertical gradient G y (x,y):

[0102]

[0103] In the above formula, I() represents the grayscale value of the pixel.

[0104] The magnitude G(x,y) and direction θ(x,y) of the gradient can be calculated by the following formula:

[0105]

[0106] Step c) Discretize the gradient. In the FHOG feature, the direction-sensitive feature B1 and the direction-insensitive feature B2 are used simultaneously. The direction-sensitive feature B1 and the direction-insensitive feature B2 can be calculated by the following formula:

[0107]

[0108] Figure 4 FIG. 1 is a schematic diagram showing a direction-sensitive feature according to an embodiment of the present application. Figure 4 As shown, the gradient direction of each point can be dispersed into p angles. Figure 4 In the above equation, p is 18. That is, according to the direction of the gradient, 0 to 2π is evenly divided into 18 intervals, which are marked as interval 1 (z1 block), interval 2 (z2 block), ..., interval 18 (z 18 The gradient direction is divided into the nearest neighbor interval. Figure 5 A schematic diagram showing a direction-insensitive feature according to an embodiment of the present application is shown. Figure 5 In , p is 9, that is, according to the direction of the gradient, 0 to 2π is evenly divided into 9 intervals, and marked as interval 1 (z1 block), interval 2 (z2 block), ..., interval 9 (z9 block). The gradient direction is divided into the nearest interval. Figure 4 and Figure 5 The division of direction-sensitive features and direction-insensitive features shows that direction-insensitive features can be obtained through direction-sensitive features. It is only necessary to add the two-dimensional features specified in the 18-dimensional direction-sensitive features in pairs, that is, the 1st dimension and the 10th dimension are added as the 1st dimension of the direction-insensitive feature... The 9th dimension and the 18th dimension are added as the 9th dimension of the direction-insensitive feature. Therefore, the direction-sensitive features can be calculated first, and then the direction-insensitive features can be obtained.

[0109] Step d) calculates the gradient direction histogram. In this step, for a first sub-region, the first sub-region can be divided into multiple cell units (cells). Adjacent cells do not overlap, and within the cell, each pixel uses the gradient amplitude as a weight to vote on the gradient direction histogram. Each pixel uses the gradient amplitude as a weight to vote for the histogram, and histogram statistics are performed separately in each cell. Since the histogram is divided into 18 intervals (bins), an 18-dimensional direction-sensitive feature can be obtained in the end. Through the 18-dimensional direction-sensitive features that have been obtained, a 9-dimensional direction-insensitive feature vector can be obtained by adding the corresponding bins in pairs.

[0110] The size of the cell can be selected as needed. For example, the size of the cell can be 8*8. If the size of the first sub-region is 16*16, the first sub-region can be divided into 4 cells. Figure 2 The embodiment shown illustrates the division of the cell unit. Figure 2 In the example, the size of the first sub-region is 16*16, and the size of the cell is 8*8. The first sub-region can be divided into 4 cells.

[0111] Step e) Gradient energy calculation: The gradient energy N of the first sub-region can be calculated by the following formula:

[0112] N σ,γ (i,j)=(C2(i,j) 2 +C2(i+σ,j) 2 +C2(i,j+γ) 2 +C2(i+σ,j+γ) 2 +ε 2 ) 1 / 2

[0113] Where C2(i,j) is a 9-dimensional direction-insensitive eigenvector, σ,γ∈{-1>1}.

[0114] Step f) Feature normalization. In this step, an 18-dimensional normalized direction-sensitive feature vector R1 and a 9-dimensional direction-insensitive feature vector R2 are calculated. For each normalization, all gradient directions are accumulated to obtain a 4-dimensional texture feature vector R3. R1 and R2 can be calculated by the following formula:

[0115]

[0116]

[0117] Where C1(i,j) is the eigenvector of R1. C2(i,j) is the eigenvector of R2. σ,γ (i,j) is the gradient energy. T αIt is a cutoff function, which means that for values ​​greater than 0.2, it is assigned a value of 0.2.

[0118] Step g) Feature concatenation: In this step, the normalized R1, R2 and R3 can be concatenated to obtain the 31-dimensional FHOG feature of the first sub-region.

[0119] Optionally, after obtaining SIFT features, CN features and FHOG features, the three features may be aggregated to obtain 170-dimensional feature information of the first sub-region. Feature aggregation is similar to the above-mentioned “concatenation”, which may be to splice the channels of each feature together to form a new feature.

[0120] In this example, the method for extracting feature information in the first sub-region is described in detail. It can be understood that the method for extracting feature information in the second sub-region is similar to that in the first sub-region. For the sake of brevity, this article will not repeat it.

[0121] According to the above technical solution, by acquiring the SIFT features, CN features and FHOG features of the first sub-region, it is helpful to provide a reliable basis for the subsequent step (determining the target observation region in at least one candidate observation region).

[0122] Exemplarily, step S130, determining at least one candidate observation area from the endoscopic image to be tested, may include the following steps: dividing at least one candidate observation area on the endoscopic image to be tested; wherein, each candidate observation area in the at least one candidate observation area has the same size; when the number of candidate observation areas in the at least one candidate observation area is multiple, the multiple candidate observation areas do not overlap with each other. It can be understood that since the size of the endoscopic image to be tested may not be an integer multiple of the size of the candidate observation area, after the division is completed, there may be an edge portion in the endoscopic image to be tested that does not belong to any candidate observation area. In this example, the edge portion can be directly discarded.

[0123] For example, the size of each candidate observation area in at least one candidate observation area may be the same as the size of the reference observation area. The size of the reference observation area may be used as a reference, and a non-overlapping division method may be used to divide the area on the endoscopic image to be tested, so as to divide at least one candidate observation area. Figure 6 FIG. 1 is a schematic diagram showing an endoscopic image to be tested according to an embodiment of the present application. Figure 6 As shown, the endoscopic image to be tested can be divided into four candidate observation areas whose sizes are consistent with the reference observation area and do not overlap with each other. Each candidate observation area includes multiple second sub-areas. For simplicity, only one of the candidate observation areas is marked in the figure. Figure 6 In the example, the right edge is not large enough to be divided into a candidate observation area alone, so it can be directly discarded.

[0124] In the above scheme, at least one candidate observation area is divided on the endoscopic image to be tested in a non-overlapping manner, which helps to reduce the amount of calculation and improve the calculation efficiency.

[0125] Exemplarily, step S130, determining at least one candidate observation region from the endoscopic image to be tested, may include the following steps: acquiring at least one candidate observation region by sliding a sliding window on the endoscopic image to be tested.

[0126] Optionally, the sliding step of the sliding window can be set as needed. The smaller the sliding step, the more candidate observation areas there are, and the more accurate the determined target observation area is. But at the same time, more candidate observation areas will also increase the amount of calculation, affecting the efficiency of endoscopic examination. Therefore, the accuracy of determining the target observation area and the calculation efficiency can be comprehensively considered to select a suitable sliding step. In a specific embodiment, the sliding step can be 10.

[0127] In the above solution, the candidate observation areas are obtained by sliding the sliding window, and compared with the non-overlapping method, there is no need to discard the edge part. This solution helps to more accurately determine the target area in at least one candidate observation area.

[0128] When determining the candidate observation area, at least one candidate observation area can be divided on the entire endoscopic image to be tested by non-overlapping divisions, or at least one candidate observation area can be obtained by sliding a sliding window. The former has a small amount of calculation and high calculation efficiency, while the latter has a higher accuracy in determining the target observation area.

[0129] Exemplarily, the method 100 may further include the following steps S170 , S180 , and S190 .

[0130] In step S170 , image parameters of the reference observation area and image parameters of the target observation area are acquired respectively.

[0131] Optionally, the image parameter may be any parameter that can reflect the imaging quality, for example, it may include but is not limited to clarity, brightness, contrast, etc. In some embodiments, the image parameter may include only clarity. In other embodiments, the image parameter may also include only brightness. In still other embodiments, the image parameter may include clarity and brightness. In an optional embodiment, the type of image parameter may be determined according to the function of the corresponding endoscope system. For example, when the endoscope system has a focusing function, the image parameter may include clarity and brightness. When the endoscope system does not have a focusing function, the image parameter may include only brightness.

[0132] Optionally, the method for acquiring the image parameters may adopt any existing or future developed method for calculating image parameters.

[0133] In an optional embodiment, the image parameters include clarity. In this embodiment, the clarity of the target observation area can be calculated by any one of the Tenengrad gradient method, the Laplacian gradient method, the frequency distribution calculation method, etc. In a specific embodiment, the image of the target observation area can be grayed first, and then the gradient map can be calculated. In this step, the Sobel operator can be used to calculate the horizontal and vertical gradients of the target observation area respectively. Finally, the average grayscale value of the target observation area is calculated. It can be understood that in the same scene, the higher the gradient value, the clearer the image of the target observation area. In this embodiment, the indicator for measuring image clarity is the average grayscale value of the image processed by the Sobel operator. The larger the average grayscale value, the clearer the target observation area.

[0134] In another optional embodiment, the image parameter includes brightness. In this embodiment, the brightness value of each pixel in the target observation area can be calculated first. Then, the brightness value of each pixel can be compared, and some pixels with larger brightness values ​​can be selected to calculate the average brightness value of the pixels. The brightness of the target observation area can be represented by the average value. The brightness value of each pixel in the target observation area can be calculated by the following formula:

[0135] Gray=0.3*R+0.59*G+0.11*B

[0136] Among them, Gray represents the grayscale of the image. R represents red, G represents green, and B represents blue. In this embodiment, the grayscale value is used to represent the brightness value. The larger the grayscale value, the larger the brightness value of the corresponding pixel. In a specific embodiment, first, the brightness value of each pixel in the target observation area is calculated. Then, based on the brightness value of each pixel, the brightness histogram of the target observation area is calculated. Finally, the average brightness value of the first 60% of the pixels with larger brightness values ​​is calculated, and the average value is used as the brightness of the target observation area.

[0137] Figure 7 FIG. 2 shows a schematic flow chart of obtaining image parameters according to an embodiment of the present application. In this embodiment, the image parameter is brightness. Figure 7 As shown, first, determine the image of interest (ROI) (i.e., the image corresponding to the target observation area) in the endoscopic image to be tested. Then, calculate the brightness value of each pixel in the ROI image. Next, based on the brightness value of each pixel, calculate the brightness histogram of the target observation area. After obtaining the brightness histogram, select the first 60% of pixels with larger brightness values ​​in the brightness histogram. And calculate the average brightness of the first 60% of pixels. This average is the brightness of the target observation area.

[0138] The method for acquiring the image parameters of the reference observation area is similar to the method for acquiring the image parameters of the target observation area, and will not be described in detail.

[0139] In step S180, it is determined whether the image parameters of the target observation area meet preset requirements, wherein the preset requirements include: the image parameter difference between the image parameters of the target observation area and the image parameters of the reference observation area is less than a preset difference threshold.

[0140] In this article, the reference endoscopic image refers to an endoscopic image in which the reference observation area meets the user's observation requirements. Therefore, the image effect of the reference observation area in the reference endoscopic image can be used as a judgment basis for the image effect of the target observation area in the endoscopic image to be tested. According to the judgment basis, it can be more accurately determined whether the target observation area in the endoscopic image to be tested meets the user's observation requirements. For simplicity, the image parameters of the reference observation area are referred to as reference image parameters below.

[0141] Optionally, the preset difference threshold can be set as needed. For example, it can be set according to the user's requirements for image effects. The higher the requirements for image effects, the smaller the preset difference threshold can be; otherwise, vice versa.

[0142] Optionally, the preset difference threshold may include difference thresholds corresponding to different image parameters, for example, when the image parameters include clarity and brightness, the preset difference threshold may include a first difference threshold corresponding to clarity and a second difference threshold corresponding to brightness.

[0143] In step S190, when the image parameters of the target observation area do not meet the preset requirements, the imaging parameters of the endoscope system are adjusted so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements.

[0144] Optionally, the imaging parameters of the endoscope may include any one or more of the focus parameters, exposure time, analog gain of an image acquisition device in the endoscope system, and the light intensity of an illumination device in the endoscope system. In some embodiments, the imaging parameters of the endoscope may be determined based on the image parameters of the target observation area. For example, if the image parameter of the target observation area is clarity, the imaging parameters of the endoscope may include imaging parameters that affect the image clarity, such as the focus parameters of the image acquisition device. If the image parameter of the target observation area is brightness, the imaging parameters of the endoscope may include imaging parameters that affect the image brightness, such as any one or more of the exposure time, analog gain of the image acquisition device, and the light intensity of an illumination device in the endoscope system.

[0145] In the related art, the endoscope can automatically adjust the imaging parameters according to the comparison results between the image parameters of the current endoscopic image and the preset image parameters. However, since this adjustment method focuses on the global information of the endoscopic image, the adjusted image parameters are affected by the image area outside the target observation area, and the image parameters are difficult to ensure that the target observation area always has a better imaging effect. In addition, the image parameters of the current endoscopic image are mainly adjusted based on the preset image parameters in the related art. This method can only ensure that the image parameters of each endoscopic image after adjustment are maintained near the preset image parameters, but there may still be large differences in the image effects between each endoscopic image. For example, the image parameters of one frame of the adjusted endoscopic image may be within the negative deviation range of the preset image parameters, and the image parameters of another frame of the adjusted endoscopic image may be within the positive deviation range of the preset image parameters. Although both frames of endoscopic images are within the deviation range of the preset image parameters, the image parameter difference between the two frames of endoscopic images may be large. In short, the imaging parameter adjustment method in the related art cannot meet user needs and affects the user experience. In the above technical solution of the present application, when the image parameters of the target observation area do not meet the preset requirements, the image parameters of the target observation area in the subsequent endoscopic image to be tested are adjusted by adjusting the imaging parameters of the endoscope system. This solution mainly focuses on the image parameters of the target observation area in the endoscopic image to be tested. By adjusting the imaging parameters, it helps to make the target observation area in the endoscopic image to be tested have a better image effect, thereby helping to meet the user's observation needs. At the same time, this solution uses the comparison result between the image parameters of the reference observation area and the image parameters of the target observation area as the basis for adjusting the imaging parameters, thereby helping to ensure the consistency of the image effect of the target observation area with the image effect of the reference observation area, so that the user always has a satisfactory and stable observation effect when observing the target object.

[0146] Exemplarily, the image parameters include clarity. The preset requirements include a first preset requirement. The preset difference threshold includes a first difference threshold. Step S180, determining whether the image parameters of the target observation area meet the preset requirements, may include the following steps: determining whether the difference between the clarity of the target observation area and the clarity of the reference observation area is less than the first difference threshold. If the difference between the clarity of the target observation area and the clarity of the reference observation area is less than the first difference threshold, it is determined that the image parameters of the target observation area meet the first preset requirement. For ease of description, the clarity of the reference observation area may be referred to as the reference clarity.

[0147] Optionally, the first difference threshold value can be set as required. For example, it can be set according to the user's requirements for image effects. The higher the user's requirements for image effects, the smaller the first difference threshold value can be; otherwise, vice versa.

[0148] According to the above technical solution, by comparing the clarity of the target observation area with the benchmark clarity, it can be more accurately determined whether the image parameters of the target observation area meet the first preset requirements, thereby providing a more reliable basis for subsequent steps (such as step S190), which is conducive to ensuring that the clarity of the target observation area is consistent with the benchmark clarity.

[0149] Exemplarily, the imaging parameters include focus parameters of an image acquisition device in an endoscope system. Step S190, when the image parameters of the target observation area do not meet the preset requirements, adjust the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic images to be tested meet the preset requirements, which may specifically include the following steps: when the image parameters of the target observation area do not meet the first preset requirements, repeat the following steps until the image parameters of the target observation area in any subsequent endoscopic images to be tested meet the first preset requirements: adjust the focus parameters of the image acquisition device based on the preset focusing step to control the focusing of the image acquisition device. Determine whether the image parameters of the target observation area in the subsequent endoscopic images to be tested obtained after focusing meet the first preset requirements.

[0150] Optionally, the preset focusing step length may be a fixed value. The preset focusing step length may be set as required. The smaller the preset focusing step length, the higher the focusing accuracy; otherwise, vice versa. In a specific embodiment, the preset focusing step length may be 1 millimeter (mm). Alternatively, the preset focusing step length may be a non-fixed value. For example, the preset focusing step length may be determined based on the difference between the clarity of the target observation area and the reference clarity.

[0151] Optionally, the type of the preset focusing step can be determined according to the type of focusing member in the image acquisition device. In one embodiment, the image acquisition device may include a deformable lens driven by an electrical signal. In this embodiment, the focusing parameter of the image acquisition device may be represented by an electrical signal applied to the deformable lens. Based on the electrical signal before focusing and the preset focusing step, the corresponding new electrical signal after focusing can be determined. The new electrical signal can be used to drive the deformable lens to deform, thereby achieving focusing. In another embodiment, the image acquisition device may include an ultrasonic motor and a movable lens. It can be understood that the ultrasonic motor can utilize the characteristic that the input voltage of the piezoelectric material will produce deformation, so that it can generate mechanical vibrations of ultrasonic frequencies, and then through the friction drive design, the ultrasonic motor can be rotated or linearly moved like an electromagnetic motor, thereby driving the movable lens to move and achieve rapid focusing. In this embodiment, the focusing parameter of the image acquisition device can be represented by the driving current of the ultrasonic motor, and the preset focusing step can correspond to the change in the driving current. The driving current of the ultrasonic motor can be adjusted based on the preset focusing step to control the focusing of the image acquisition device. The preset focusing step can be set as needed. For example, when the preset focus step size corresponds to the amount of change in the driving current of the ultrasonic motor, the preset focus step size may be 10 milliamperes (mA).

[0152] Optionally, the image acquisition device includes an ultrasonic motor and a movable lens. The focus parameter includes a driving current of the ultrasonic motor. Adjusting the focus parameter of the image acquisition device based on a preset focus step to control the focus of the image acquisition device may include the following steps: adjusting the current driving current according to the preset focus step. The adjustment may include increasing and decreasing. The driving current corresponding to the difference between the clarity of the target observation area and the reference clarity is less than the first difference threshold can be found by gradually increasing or gradually decreasing the current driving current. Exemplarily, when the current driving current is increased according to the preset focus step, it can be determined whether the increased driving current is greater than the maximum current threshold. When the increased driving current is less than or equal to the maximum current threshold, the ultrasonic motor is controlled to move according to the increased driving current. When the increased driving current is greater than the maximum current threshold, the current driving current is reset to the minimum current threshold, the current driving current is increased from the minimum current threshold to the new driving current according to the preset focus step, and the ultrasonic motor is controlled to move according to the new driving current. It can be understood that both the minimum current threshold and the maximum current threshold can be determined according to the working parameters of the ultrasonic motor. In this embodiment, when the increased driving current is greater than the maximum current threshold, the driving current is reset to the minimum current threshold, and adjustment is performed from the minimum current threshold based on the preset focusing step, thereby helping to increase the range of clarity adjustment and improve the accuracy of clarity adjustment. Similarly, when the current driving current is reduced according to the preset focusing step, it can be determined whether the reduced driving current is less than the minimum current threshold. When the reduced driving current is greater than or equal to the maximum current threshold, the ultrasonic motor is controlled to move according to the reduced driving current. When the reduced driving current is less than the minimum current threshold, the current driving current is reset to the maximum current threshold, the current driving current is reduced from the maximum current threshold to the new driving current according to the preset focusing step, and the ultrasonic motor is controlled to move according to the new driving current.

[0153] Figure 8 A schematic flow chart of an endoscopic observation area tracking method according to another specific embodiment of the present application is shown. In this embodiment, the image acquisition device includes an ultrasonic motor and a movable lens. The imaging parameter is a focus parameter of the image acquisition device. The focus parameter is a driving current of the ultrasonic motor. The preset focus step is 10 milliamperes (mA). Figure 8 As shown, first, execute step S810 to determine the target observation area. Then, execute step S820 to calculate the clarity value F of the target observation area. Then, execute step S830 to determine whether the difference between F and the reference clarity F0 is less than the first difference threshold TH1. When the difference between F and F0 is greater than or equal to TH1, execute step S840 to set the current driving current I=I+10. Then, execute step S850 to determine whether the increased driving current I is less than the maximum current threshold Imax . If I<I max , then based on the increased driving current, return to step S810. Otherwise, execute step S860 and set the driving current I=I min +10, that is, reset the current driving current to the minimum current threshold I min , according to the preset focusing step, the current driving current is increased from the minimum current threshold I min The new driving current is increased, and based on the new driving current, the execution returns to step S810. When the difference between F and F0 is less than TH1, it is determined that the focusing is completed. That is, it is determined that the image parameters of the target observation area in the subsequent endoscope image to be tested obtained after focusing meet the first preset requirement.

[0154] In the above technical solution, when the image parameters do not meet the first preset requirement, the image acquisition device is controlled to focus based on the preset focusing step length so that the image parameters of the target observation area in the endoscopic image to be tested meet the first preset requirement. This solution has a simple adjustment method and helps to improve the focusing efficiency.

[0155] Exemplarily, the image acquisition device includes a deformable lens driven by an electrical signal, and adjusting the focus parameter of the image acquisition device based on a preset focus step may specifically include the following steps: adjusting the electrical signal applied to the deformable lens based on the preset focus step; wherein the focus parameter is represented by the electrical signal.

[0156] In this embodiment, the image acquisition device adjusts the deformation amount of the deformable lens by changing the driving signal of the deformable lens (i.e., the electrical signal applied to the deformable lens), thereby changing the focal length of the image acquisition device. Therefore, in this embodiment, the focus parameter can be represented by the electrical signal applied to the deformable lens. The electrical signal applied to the deformable lens can be adjusted based on the preset focusing step to control the focusing of the image acquisition device. Among them, the deformable lens can be any lens that changes the curvature of the optical surface under electrical control, such as a liquid lens or a piezoelectrically actuated deformable lens. Preferably, it is a piezoelectrically actuated deformable lens, which has a higher response speed and can achieve fast focusing. The specific focusing process is similar to the focusing process of the ultrasonic motor and is not repeated here.

[0157] Exemplarily, the image parameter includes brightness. The preset requirement includes a second preset requirement. The preset difference threshold includes a second difference threshold. Step S180, determining whether the image parameter of the target observation area meets the preset requirement, may include the following steps: determining whether the difference between the brightness of the target observation area and the brightness of the reference observation area is less than the second difference threshold. If the difference between the brightness of the target observation area and the brightness of the reference observation area is less than the second difference threshold, it is determined that the image parameter of the target observation area meets the second preset requirement. For ease of description, the brightness of the reference observation area may be referred to as the reference brightness.

[0158] Similar to the first difference threshold, the second difference threshold can be set as needed. For example, it can be set according to the user's requirements for image effects. The higher the user's requirements for image effects, the smaller the second difference threshold can be; otherwise, vice versa.

[0159] According to the above technical solution, by comparing the brightness of the target observation area with the reference brightness, it is possible to more accurately determine whether the image parameters of the target observation area meet the second preset requirement, thereby providing a more reliable basis for subsequent steps (such as step S190). Furthermore, this solution is conducive to ensuring that the brightness of the target observation area is always maintained within a better range, and the brightness consistency of the target observation area in multiple frames of endoscopic images to be tested is high, which is conducive to reducing the brightness and darkness changes in the target observation area.

[0160] Exemplarily, the imaging parameters include one or more of the following: exposure time of an image acquisition device in the endoscope system, analog gain of the image acquisition device, and light intensity of an illumination device in the endoscope system. Step S190, when the image parameters of the target observation area do not meet the preset requirements, adjust the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, which may include the following steps: when the image parameters of the target observation area do not meet the second preset requirements, perform one or more of the following operations: exposure time adjustment operation, light intensity adjustment operation, and analog gain adjustment operation.

[0161] The exposure time adjustment operation may include the following steps: adjusting the exposure time of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired under the adjusted exposure time meet the second preset requirements or the adjusted exposure time reaches the exposure time threshold.

[0162] The light intensity adjustment operation may include the following steps: adjusting the light intensity of the lighting device until the image parameters of the target observation area in the endoscopic image to be tested collected under the adjusted light intensity meet the second preset requirement or the adjusted light intensity reaches the light intensity threshold.

[0163] The analog gain adjustment operation may include the following steps: adjusting the analog gain of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired under the adjusted analog gain meet the second preset requirements or the adjusted analog gain reaches the analog gain threshold.

[0164] In some embodiments, when the image parameters of the target observation area do not meet the second preset requirement, only any one of the exposure time adjustment operation, the light intensity adjustment operation, and the analog gain adjustment operation may be performed. In other embodiments, any multiple of the exposure time adjustment operation, the light intensity adjustment operation, and the analog gain adjustment operation may be performed. For example, the exposure time adjustment operation, the light intensity adjustment operation, and the analog gain adjustment operation may be performed simultaneously. For another example, the exposure time adjustment operation, the light intensity adjustment operation, and the analog gain adjustment operation may be performed sequentially according to a preset order. The preset order may be set as needed.

[0165] Optionally, step S190, when the image parameters of the target observation area do not meet the preset requirements, adjust the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, and may also include the following steps: when the current imaging parameters reach the imaging parameter threshold, determine that the image parameters of the target observation area in the endoscopic image to be tested corresponding to the imaging parameter threshold meet the preset requirements. The imaging parameter threshold includes a first imaging parameter threshold and a second imaging parameter threshold. The first imaging parameter threshold represents the maximum value of the imaging parameter, and the second imaging parameter threshold represents the minimum value of the imaging parameter. In one embodiment, the imaging parameters include exposure time, light intensity and analog gain, and the imaging parameter threshold includes an exposure time threshold, a light intensity threshold and an analog gain threshold. Among them, the exposure time threshold includes a maximum exposure time and a minimum exposure time. The light intensity threshold includes a maximum light intensity and a minimum light intensity. The analog gain threshold includes a maximum analog gain and a minimum analog gain. When the current exposure time is the exposure time threshold, the light intensity is the light intensity threshold, and the analog gain is the analog gain threshold, directly determine that the image parameters of the target observation area in the endoscopic image to be tested corresponding to the imaging parameter meet the preset requirements. It is understandable that when adjusting the imaging parameters, it is usually based on only one parameter adjustment direction (e.g., increase or decrease). After the imaging parameters reach the imaging parameter threshold, it means that the imaging parameters can no longer be adjusted during the current adjustment process. At this time, the image effect of the target observation area in the endoscopic image to be tested has reached the best level that can be adjusted at present. Therefore, it can be directly considered that the image parameters of the target observation area in the endoscopic image to be tested meet the image requirements.

[0166] According to the above technical scheme, by performing one or more of the exposure time adjustment operation, the light intensity adjustment operation and the analog gain adjustment operation, the image parameters (brightness in this scheme) of the endoscope system can be adjusted more accurately, thereby helping to obtain a target observation area that meets the user's observation needs.

[0167] Exemplarily, the exposure time adjustment operation is performed first. The light intensity adjustment operation is performed when the adjusted exposure time reaches the exposure time threshold and the image parameters of the target observation area in the endoscopic image to be tested collected at the adjusted exposure time do not meet the second preset requirement. The analog gain adjustment operation is performed when the adjusted light intensity reaches the light intensity threshold and the image parameters of the target observation area in the endoscopic image to be tested collected at the adjusted light intensity do not meet the second preset requirement.

[0168] It is understandable that for the endoscope system, the core of the brightness adjustment mechanism is stability, rapid response and no obvious flicker. In this example, only one imaging parameter is adjusted at a time, which can effectively reduce the complexity of the calculation, ensure timely response speed, and at the same time ensure the stability of the dimming process and reduce the mutual influence between parameters.

[0169] In this example, the exposure parameters, light intensity, and analog gain are adjusted in sequence. It can be understood that both the exposure time and the analog gain act directly on the sensor of the endoscope system, so the response is faster. However, the gain adjustment will cause changes in image noise, which will significantly affect the observation effect of the image. Therefore, this example adopts the adjustment sequence of exposure parameters, light intensity, and analog gain.

[0170] Fig. 9 A schematic flow chart of an endoscope observation area tracking method according to another specific embodiment of the present application is shown. In this embodiment, the imaging parameters include the exposure time of the image acquisition device in the endoscope system, the analog gain of the image acquisition device, and the light intensity of the lighting device in the endoscope system. Fig. 9 As shown, first, execute step S910 to determine the target observation area. That is, in the endoscopic image to be tested corresponding to the current imaging parameters, determine the target observation area. Then, execute step S920 to calculate the brightness value B of the area (that is, the target observation area). Next, execute step S930 to determine whether the difference between B and the reference brightness B0 is less than the second difference threshold TH2. If the difference between B and B0 is less than TH2, maintain the current imaging parameters. At this time, the image parameters of the target observation area meet the second preset requirement. If the difference between B and B0 is greater than or equal to TH2, execute step S940 to perform the exposure time adjustment operation. Next, execute step S950 to determine whether the adjusted exposure time T reaches the exposure time threshold T M , that is, whether T = T M The exposure time threshold T M Including the maximum exposure time T1 and the minimum exposure time T2. If T is not equal to T M (ie, T2<T<T1), then return to step S910. M(ie, T = T1 or T = T2), then execute step S960 to perform the light intensity adjustment operation. Then execute step S970 to determine whether the adjusted light intensity L reaches the light intensity threshold L. M , that is, whether L = L M The light intensity threshold L M Includes maximum light intensity L1 and minimum light intensity L2. If L is not equal to L M (ie L2<L<L1), then return to step S910. M (ie L = L1 or L = L2), then execute step S980 to perform the analog gain adjustment operation. Then execute step S990 to determine whether the adjusted analog gain M reaches the analog gain threshold M. M , that is, whether M=M M The analog gain threshold M M Includes the maximum analog gain M1 and the minimum analog gain M2. If M is not equal to M M (ie, M2<M<M1), then return to step S910. M (ie, M=M1 or M=M2), then maintain the imaging parameter thresholds, that is, maintain the current exposure time that reaches the exposure time threshold, the light intensity that reaches the light intensity threshold, and the analog gain that reaches the analog gain threshold. M , L M and M M As the imaging parameter of the endoscope system, the target observation area in the endoscope image to be measured corresponding to the imaging parameter is used as the observation area that meets the preset requirements.

[0171] According to the above technical solution, by adjusting only one imaging parameter at a time, the complexity of calculation can be effectively reduced, timely response speed can be ensured, and the stability of the dimming process can be ensured, and the mutual influence between parameters can be reduced. By adopting the imaging parameter adjustment sequence of exposure parameter, light intensity and analog gain, it is helpful to improve the adjustment efficiency, reduce the adjustment time, and help ensure the observation effect of the image.

[0172] Exemplarily, the exposure time adjustment operation may include the following steps: adjusting the current exposure time to a new exposure time according to a first preset adjustment amount. Acquiring a new endoscopic image to be tested corresponding to the new exposure time. Determining whether the image parameters of the target observation area in the new endoscopic image to be tested meet the second preset requirement. If the image parameters of the target observation area in the new endoscopic image to be tested do not meet the second preset requirement, returning to the step of adjusting the current exposure time to a new exposure time according to the first preset adjustment amount.

[0173] Optionally, before acquiring a new endoscopic image to be tested corresponding to the new exposure time, the exposure time adjustment operation may further include the following steps: determining whether the new exposure time reaches an exposure time threshold. If the new exposure time reaches the exposure time threshold, the exposure time threshold is determined to be the new exposure time. It is understandable that after the exposure time reaches the exposure time threshold, the image brightness cannot be adjusted based on the exposure time. Since the longer the exposure time, the higher the image brightness. Therefore, the exposure time threshold may be determined as the exposure time of the subsequent endoscopic image to be tested.

[0174] Optionally, the first preset adjustment amount can be set as needed. The larger the first preset adjustment amount, the lower the accuracy of the image brightness adjustment based on the exposure time; the smaller the first preset adjustment amount, the higher the accuracy of the image brightness adjustment based on the exposure time. In a specific embodiment, the first preset adjustment amount can be determined based on the actual working parameters of the image acquisition device of the endoscope system and / or the image brightness difference between the target observation area and the reference observation area.

[0175] Optionally, the adjustment may include increasing and decreasing. It is understood that the exposure time is positively correlated with the image brightness. In some embodiments, when the brightness of the target observation area is greater than the brightness of the reference observation area, the current exposure time is reduced to a new exposure time according to a first preset adjustment amount. In other embodiments, when the brightness of the target observation area is less than the brightness of the reference observation area, the current exposure time is increased to a new exposure time according to a first preset adjustment amount.

[0176] According to the above technical solution, by gradually adjusting the exposure time according to the first preset adjustment amount, the image brightness can be adjusted with higher accuracy, thereby helping to quickly and accurately obtain the endoscopic image to be tested that meets the user's observation needs.

[0177] Exemplarily, the light intensity adjustment operation may include the following steps: adjusting the current light intensity to a new light intensity according to a second preset adjustment amount. Acquiring a new endoscopic image to be tested corresponding to the new light intensity. Determining whether the image parameters of the target observation area in the new endoscopic image to be tested meet the second preset requirement. If the image parameters of the target observation area in the new endoscopic image to be tested do not meet the second preset requirement, returning to the step of adjusting the current light intensity to the new light intensity according to the second preset adjustment amount.

[0178] Optionally, before acquiring a new endoscopic image to be tested corresponding to the new light intensity, the light intensity adjustment operation may further include the following steps: determining whether the new light intensity reaches a light intensity threshold. If the new light intensity reaches the light intensity threshold, the light intensity threshold is determined to be the new light intensity. It is understandable that after the light intensity reaches the light intensity threshold, the image brightness cannot be adjusted based on the light intensity. Since the greater the light intensity, the higher the image brightness. Therefore, the light intensity threshold may be determined as the light intensity of the subsequent endoscopic image to be tested.

[0179] Optionally, the second preset adjustment amount can be set as needed. The larger the second preset adjustment amount, the lower the accuracy of image brightness adjustment based on light intensity; the smaller the second preset adjustment amount, the higher the accuracy of image brightness adjustment based on light intensity. In a specific embodiment, the second preset adjustment amount can be determined based on actual working parameters of the lighting device of the endoscope system and / or the image brightness difference between the target observation area and the reference observation area.

[0180] Optionally, the adjustment may include increasing and decreasing. It is understood that the light intensity is positively correlated with the image brightness. In some embodiments, when the brightness of the target observation area is greater than the brightness of the reference observation area, the current light intensity is reduced to a new light intensity according to a second preset adjustment amount. In other embodiments, when the brightness of the target observation area is less than the brightness of the reference observation area, the current light intensity is increased to a new light intensity according to a second preset adjustment amount.

[0181] According to the above technical solution, by gradually adjusting the light intensity according to the second preset adjustment amount, the image brightness can be adjusted with higher accuracy, thereby helping to quickly and accurately obtain the endoscopic image to be tested that meets the user's observation needs.

[0182] Exemplarily, the analog gain adjustment operation may include the following steps: adjusting the current analog gain to a new analog gain according to a third preset adjustment amount. Acquiring a new endoscopic image to be tested corresponding to the new analog gain. Determining whether the image parameters of the target observation area in the new endoscopic image to be tested meet the second preset requirement. If the image parameters of the target observation area in the new endoscopic image to be tested do not meet the second preset requirement, returning to the step of adjusting the current analog gain to a new analog gain according to the third preset adjustment amount.

[0183] Optionally, before acquiring a new endoscopic image to be tested corresponding to the new analog gain, the analog gain adjustment operation may further include the following steps: determining whether the new analog gain reaches an analog gain threshold. If the new analog gain reaches the analog gain threshold, the analog gain threshold is determined to be the new analog gain. It is understandable that after the analog gain reaches the analog gain threshold, the image brightness cannot be adjusted based on the analog gain. Since the larger the analog gain, the higher the image brightness. Therefore, the analog gain threshold may be determined as the analog gain of the subsequent endoscopic image to be tested.

[0184] Optionally, the third preset adjustment amount can be set as needed. The larger the third preset adjustment amount, the lower the accuracy of the image brightness adjustment based on the analog gain; the smaller the third preset adjustment amount, the higher the accuracy of the image brightness adjustment based on the analog gain. In a specific embodiment, the third preset adjustment amount can be determined based on the actual working parameters of the image acquisition device of the endoscope system and / or the image brightness difference between the target observation area and the reference observation area.

[0185] Optionally, the adjustment may include increasing and decreasing. It is understood that the analog gain is positively correlated with the image brightness. In some embodiments, when the brightness of the target observation area is greater than the brightness of the reference observation area, the current analog gain is reduced to a new analog gain according to a third preset adjustment amount. In other embodiments, when the brightness of the target observation area is less than the brightness of the reference observation area, the current analog gain is increased to a new analog gain according to a third preset adjustment amount.

[0186] According to the above technical solution, by gradually adjusting the analog gain according to the third preset adjustment amount, the image brightness can be adjusted with higher accuracy, thereby helping to quickly and accurately obtain the endoscopic image to be tested that meets the user's observation needs.

[0187] Exemplarily, the image parameters include clarity and brightness. The preset requirements include a first preset requirement and a second preset requirement. The preset difference threshold includes a first preset difference threshold and a second preset difference threshold. Step S180, determining whether the image parameters of the target observation area meet the preset requirements, may include the following steps: when the image parameters of the target observation area meet the first preset requirement and the second preset requirement at the same time, determining that the image parameters of the target observation area meet the preset requirements. The specific determination method has been described in detail above. For the sake of brevity, it will not be repeated here. In this example, considering the clarity and brightness of the target observation area at the same time helps to further improve the image effect of the target observation area.

[0188] Exemplarily, the image parameters include clarity and brightness. The preset requirements include a first preset requirement and a second preset requirement. Step S190, when the image parameters of the target observation area do not meet the preset requirements, adjust the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, which may include the following steps: adjusting the focus parameters of the image acquisition device based on a preset focusing step until the image parameters of the target observation area meet the first preset requirements. After the image parameters of the target observation area meet the first preset requirements, perform one or more of the following operations: exposure time adjustment operation, light intensity adjustment operation, and analog gain adjustment operation, so that the image parameters of the target observation area meet the second preset requirements. In this example, the brightness of the target observation area is adjusted first, and then the clarity of the target observation area is adjusted. Thereby, it is helpful to quickly and accurately obtain the endoscopic image to be tested that meets the user's observation needs.

[0189] According to another aspect of the present application, an endoscopic observation area tracking system is provided. Fig.10 FIG. 2 is a schematic block diagram of an endoscope observation area tracking system according to an embodiment of the present application. Fig.10 As shown, the endoscopic observation area tracking system 1000 may include: a first acquisition module 1010 , a second acquisition module 1020 , a first determination module 1030 , a third acquisition module 1040 , a calculation module 1050 and a second determination module 1060 .

[0190] The first acquisition module 1010 is used to acquire first feature information of a reference observation area in a reference endoscopic image; the reference endoscopic image includes a target observation object, the reference observation area is the area where the target observation object is located, and the first feature information includes at least one of the following features: rotational invariance feature, color feature and directional gradient feature.

[0191] The second acquisition module 1020 is used to acquire the endoscopic image to be tested, where the endoscopic image to be tested contains the target observation object.

[0192] The determination module 1030 is used to determine at least one candidate observation area from the endoscopic image to be tested.

[0193] The third acquisition module 1040 is used to acquire second feature information of each candidate observation area in at least one candidate observation area; the second feature information is consistent with the type of the feature included in the first feature information.

[0194] The calculation module 1050 is used to calculate, for each candidate observation area in the at least one candidate observation area, a similarity between the candidate observation area and the reference observation area based on the second feature information and the first feature information.

[0195] The second determination module 1060 is configured to determine a target observation region based on the similarity corresponding to at least one candidate observation region, where the target observation region is a candidate observation region that contains the same target observation object as the reference observation region.

[0196] Exemplarily, each of the at least one candidate observation area has the same size as the benchmark observation area, the benchmark observation area includes a plurality of first sub-areas, the first feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of first sub-areas, each candidate observation area includes a plurality of second sub-areas, the plurality of first sub-areas correspond one-to-one to the plurality of second sub-areas and the corresponding first sub-areas and second sub-areas have the same size, the second feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of second sub-areas, wherein the calculation module 1050 includes: a calculation sub-module for calculating the feature information corresponding to each second sub-area in the candidate observation area based on the feature information corresponding to each second sub-area in the benchmark observation area The feature information corresponding to a sub-region is used to calculate the sub-similarity between each second sub-region in the candidate observation region and the corresponding first sub-region in the benchmark observation region, and the similarity includes the sub-similarity corresponding to each second sub-region in the candidate observation region one by one; the second determination module 1060 includes: a first determination sub-module, used to determine, for each candidate observation region in at least one candidate observation region, a target second sub-region whose corresponding sub-similarity in the candidate observation region is greater than a similarity threshold, and when the target second sub-region in the candidate observation region meets the target requirement, determine that the candidate observation region is a target observation region that contains the same target observation object as the benchmark observation region.

[0197] Exemplarily, the second determination module 1060 also includes: a selection submodule, which is used to select a candidate observation area with the largest number of target second subareas from at least one candidate observation area as the target observation area when none of the target second subareas in at least one candidate observation area meets the target requirement.

[0198] Exemplarily, the first acquisition module 1010 includes: a first acquisition submodule, used to acquire a baseline endoscopic image; a second determination submodule, used to determine an initial observation area on the baseline endoscopic image, the initial observation area being an area containing a target observation object; an expansion submodule, used to expand the boundaries of the initial observation area to obtain a baseline observation area; and a second acquisition submodule, used to extract features of the baseline observation area to acquire first feature information.

[0199] Exemplarily, the expansion submodule includes: an expansion unit, which is used to expand the boundary of the initial observation area based on a preset multiple with the center of the initial observation area as the center, so as to obtain a reference observation area.

[0200] Exemplarily, the determination module 1030 includes: a division submodule, used to divide at least one candidate observation area on the endoscopic image to be tested; wherein, each candidate observation area in at least one candidate observation area has the same size; when the number of candidate observation areas in at least one candidate observation area is multiple, the multiple candidate observation areas do not overlap with each other; or, a sliding acquisition submodule, used to acquire at least one candidate observation area by sliding on the endoscopic image to be tested through a sliding window.

[0201] Exemplarily, the first feature information includes rotational invariance features, color features and directional gradient features; the first acquisition module 1010 includes: a third acquisition submodule, which respectively acquires the rotational invariance features, color features and directional gradient features in the reference observation area; and an aggregation submodule, which is used to aggregate the rotational invariance features, color features and directional gradient features to obtain the first feature information.

[0202] Exemplarily, the endoscopic observation area tracking system 1000 may also include: a fourth acquisition module, used to respectively acquire the image parameters of the reference observation area and the image parameters of the target observation area; a judgment module, used to judge whether the image parameters of the target observation area meet preset requirements, wherein the preset requirements include: the image parameter difference between the image parameters of the target observation area and the reference image parameters is less than a preset difference threshold; an adjustment module, used to adjust the imaging parameters of the endoscope system when the image parameters of the target observation area do not meet the preset requirements, so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements.

[0203] Exemplarily, the image parameters include clarity; the preset requirements include a first preset requirement; the preset difference threshold includes a first difference threshold; the judgment module includes: a first judgment submodule, used to judge whether the difference between the clarity of the target observation area and the clarity of the benchmark observation area is less than the first difference threshold; a first determination submodule, used to determine that the image parameters of the target observation area meet the first preset requirements if the difference between the clarity of the target observation area and the clarity of the benchmark observation area is less than the first difference threshold.

[0204] Exemplarily, the imaging parameters include focusing parameters of an image acquisition device in an endoscope system, and the adjustment module includes: a first execution submodule, for, when the image parameters of a target observation area do not meet a first preset requirement, repeatedly executing the following steps until the image parameters of the target observation area in any subsequent endoscopic image to be tested meet the first preset requirement: adjusting the focusing parameters of the image acquisition device based on a preset focusing step to control the focusing of the image acquisition device; and determining whether the image parameters of the target observation area in a subsequent endoscopic image to be tested obtained after focusing meet the first preset requirement.

[0205] Exemplarily, the image acquisition device includes a deformable lens driven by an electrical signal, and the first execution submodule includes: a focusing unit for adjusting the electrical signal applied to the deformable lens based on a preset focusing step.

[0206] Exemplarily, the image parameters include brightness; the preset requirement includes a second preset requirement; the preset difference threshold includes a second difference threshold; the judgment module includes: a second judgment submodule, used to judge whether the difference between the brightness of the target observation area and the brightness of the benchmark observation area is less than the second difference threshold; a second determination submodule, used to determine that the image parameters of the target observation area meet the second preset requirement if the difference between the brightness of the target observation area and the brightness of the benchmark observation area is less than the second difference threshold.

[0207] Exemplarily, the imaging parameters include one or more of the following: exposure time of an image acquisition device in the endoscope system, analog gain of the image acquisition device, and light intensity of an illumination device in the endoscope system; the adjustment module includes: a second execution submodule, used to perform one or more of the following operations when the image parameters of the target observation area do not meet the second preset requirement: exposure time adjustment operation: adjusting the exposure time of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted exposure time meet the second preset requirement or the adjusted exposure time reaches an exposure time threshold; light intensity adjustment operation: adjusting the light intensity of the illumination device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted light intensity meet the second preset requirement or the adjusted light intensity reaches the light intensity threshold; analog gain adjustment operation: adjusting the analog gain of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired at the adjusted analog gain meet the second preset requirement or the adjusted analog gain reaches the analog gain threshold.

[0208] According to yet another aspect of the present application, an electronic device is also provided. Fig.11 A schematic block diagram of an electronic device according to an embodiment of the present application is shown. As shown in the figure, the electronic device 1100 includes a processor 1110 and a memory 1120, wherein the memory 1120 stores computer program instructions, and the computer program instructions are used by the processor 1110 to execute the endoscopic observation area tracking method and / or imaging parameter adjustment method in any of the above embodiments when the computer program instructions are executed.

[0209] According to another aspect of the present application, a non-volatile storage medium is also provided, on which program instructions are stored, and the program instructions are used to execute the endoscopic observation area tracking method and / or imaging parameter adjustment method in any of the above embodiments when running. The non-volatile storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0210] A person skilled in the art can understand the specific implementation scheme of the above-mentioned endoscopic observation area tracking and imaging parameter adjustment system, electronic device and non-volatile storage medium by reading the above-mentioned description of the endoscopic observation area tracking and imaging parameter adjustment method, which will not be elaborated here for the sake of brevity.

[0211] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0212] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0213] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0214] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0215] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0216] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0217] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0218] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the endoscopic observation area tracking according to the embodiments of the present application. The present application can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0219] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0220] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for tracking an endoscope observation area, characterized in that: include: Acquiring first feature information of a reference observation area in a reference endoscopic image; The reference endoscopic image includes a target observation object, the reference observation area is an area where the target observation object is located, and the first feature information includes at least one of the following features: a rotation invariance feature, a color feature, and a directional gradient feature; Acquiring an endoscopic image to be tested, wherein the endoscopic image to be tested includes the target observation object; Determine at least one candidate observation area from the endoscopic image to be tested; For each candidate observation area in the at least one candidate observation area, Acquire second feature information of the candidate observation area; the second feature information is consistent with the type of features included in the first feature information; Calculating the similarity between the candidate observation area and the reference observation area based on the second feature information and the first feature information; Based on the similarity corresponding to each of the at least one candidate observation regions, a target observation region is determined, where the target observation region is a candidate observation region that includes the same target observation object as the reference observation region.

2. The endoscopic observation area tracking method according to claim 1, characterized in that: Each of the at least one candidate observation area has the same size as the reference observation area, the reference observation area includes a plurality of first sub-areas, the first feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of first sub-areas, each candidate observation area includes a plurality of second sub-areas, the plurality of first sub-areas correspond one-to-one to the plurality of second sub-areas and the corresponding first sub-areas and second sub-areas have the same size, the second feature information includes a plurality of sets of feature information corresponding one-to-one to the plurality of second sub-areas, wherein, The calculating the similarity between the candidate observation area and the reference observation area based on the second feature information and the first feature information includes: Calculate the sub-similarity between each second sub-region in the candidate observation region and the corresponding first sub-region in the benchmark observation region based on the feature information corresponding to each second sub-region in the candidate observation region and the feature information corresponding to the corresponding first sub-region in the benchmark observation region, wherein the similarity includes the sub-similarity corresponding to each second sub-region in the candidate observation region one by one; The determining the target observation area based on the similarity corresponding to each of the at least one candidate observation area comprises: For each candidate observation area among the at least one candidate observation area, determine a target second sub-area whose corresponding sub-similarity in the candidate observation area is greater than a similarity threshold; when the target second sub-area in the candidate observation area meets the target requirement, determine that the candidate observation area is a target observation area that contains the same target observation object as the benchmark observation area.

3. The endoscopic observation area tracking method according to claim 2, characterized in that: The target requirement includes: the number of the target second sub-regions in the candidate observation region exceeds a preset number threshold, or the proportion of the area of ​​the target second sub-region in the candidate observation region to the area of ​​the candidate observation region exceeds a preset proportion threshold.

4. The endoscopic observation area tracking method according to claim 2, characterized in that: The determining of the target observation area based on the similarity corresponding to each of the at least one candidate observation area further includes: When none of the target second sub-regions in the at least one candidate observation region meets the target requirement, a candidate observation region with the largest number of target second sub-regions is selected from the at least one candidate observation region as the target observation region.

5. The endoscopic observation area tracking method according to any one of claims 1 to 4, characterized in that: The step of acquiring first feature information of a reference observation area in a reference endoscopic image includes: acquiring the reference endoscopic image; Determining an initial observation area on the reference endoscopic image, wherein the initial observation area is an area including the target observation object; Expanding the boundary of the initial observation area to obtain the reference observation area; Feature extraction is performed on the reference observation area to obtain the first feature information.

6. The endoscopic observation area tracking method according to claim 5, characterized in that: The step of expanding the boundary of the initial observation area to obtain the reference observation area includes: Taking the center of the initial observation area as the center, the boundary of the initial observation area is expanded based on a preset multiple to obtain the reference observation area.

7. The endoscopic observation area tracking method according to any one of claims 1 to 4, characterized in that: The step of determining at least one candidate observation area from the endoscopic image to be tested comprises: Dividing the at least one candidate observation area on the endoscopic image to be tested; wherein each candidate observation area in the at least one candidate observation area has the same size; when there are multiple candidate observation areas in the at least one candidate observation area, the multiple candidate observation areas do not overlap with each other; or, The at least one candidate observation area is acquired by sliding a sliding window on the endoscopic image to be tested.

8. The endoscopic observation area tracking method according to any one of claims 1 to 4, characterized in that: The first feature information includes rotation invariance feature, color feature and directional gradient feature; The step of acquiring first feature information of a reference observation area in a reference endoscopic image includes: Respectively acquiring rotation invariance features, color features, and directional gradient features in the reference observation area; The rotation invariant feature, the color feature, and the directional gradient feature are aggregated to obtain the first feature information.

9. The endoscopic observation area tracking method according to any one of claims 1 to 8, characterized in that: Also includes: Respectively acquiring image parameters of the reference observation area and image parameters of the target observation area; Determining whether the image parameters of the target observation area meet preset requirements, wherein the preset requirements include: the image parameter difference between the image parameters of the target observation area and the image parameters of the reference observation area is less than or equal to a preset difference threshold; When the image parameters of the target observation area do not meet the preset requirements, the imaging parameters of the endoscope system are adjusted so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements.

10. The endoscopic observation area tracking method according to claim 9, characterized in that: The image parameters include clarity; the preset requirements include a first preset requirement; The preset difference threshold comprises a first difference threshold; The determining whether the image parameters of the target observation area meet preset requirements includes: Determining whether a difference between the clarity of the target observation area and the clarity of the reference observation area is less than the first difference threshold; If the difference between the clarity of the target observation area and the clarity of the reference observation area is less than the first difference threshold, it is determined that the image parameters of the target observation area meet the first preset requirement.

11. The endoscopic observation area tracking method according to claim 10, characterized in that: The imaging parameters include focus parameters of the image acquisition device in the endoscope system, and when the image parameters of the target observation area do not meet the preset requirements, adjusting the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, including: When the image parameters of the target observation area do not meet the first preset requirement, the following steps are repeatedly performed until the image parameters of the target observation area in any subsequent endoscopic image to be tested meet the first preset requirement: Adjusting the focus parameter of the image acquisition device based on a preset focus step length to control the focus of the image acquisition device; Determine whether the image parameters of the target observation area in the subsequent endoscopic image to be tested obtained after focusing meet the first preset requirement.

12. The endoscopic observation area tracking method according to claim 11, characterized in that: The image acquisition device includes a deformable lens driven by an electrical signal, and the step of adjusting the focus parameter of the image acquisition device based on a preset focus step length includes: Based on the preset focusing step, the electrical signal applied to the deformable lens is adjusted; wherein the focusing parameter is represented by the electrical signal.

13. The endoscopic observation area tracking method according to any one of claims 9 to 12, characterized in that: The image parameter includes brightness; the preset requirement includes a second preset requirement; the preset difference threshold includes a second difference threshold; The determining whether the image parameters of the target observation area meet preset requirements includes: determining whether a difference between the brightness of the target observation area and the brightness of the reference observation area is less than the second difference threshold; If the difference between the brightness of the target observation area and the brightness of the reference observation area is less than the second difference threshold, it is determined that the image parameter of the target observation area meets the second preset requirement.

14. The endoscopic observation area tracking method according to claim 13, characterized in that: The imaging parameters include one or more of the following: exposure time of the image acquisition device in the endoscope system, analog gain of the image acquisition device, and light intensity of the lighting device in the endoscope system; when the image parameters of the target observation area do not meet the preset requirements, adjusting the imaging parameters of the endoscope system so that the image parameters of the target observation area in the subsequent endoscopic image to be tested meet the preset requirements, including: When the image parameters of the target observation area do not meet the second preset requirement, performing one or more of the following operations: Exposure time adjustment operation: adjusting the exposure time of the image acquisition device until the image parameters of the target observation area in the endoscope image to be tested acquired under the adjusted exposure time meet the second preset requirement or the adjusted exposure time reaches the exposure time threshold; Light intensity adjustment operation: adjusting the light intensity of the lighting device until the image parameters of the target observation area in the endoscope image to be tested collected under the adjusted light intensity meet the second preset requirement or the adjusted light intensity reaches the light intensity threshold; Analog gain adjustment operation: adjusting the analog gain of the image acquisition device until the image parameters of the target observation area in the endoscopic image to be tested acquired under the adjusted analog gain meet the second preset requirement or the adjusted analog gain reaches the analog gain threshold.

15. The endoscopic observation area tracking method according to claim 14, characterized in that: The exposure time adjustment operation is performed first, The light intensity adjustment operation is performed when the adjusted exposure time reaches the exposure time threshold and the image parameters of the target observation area in the endoscope image to be tested acquired under the adjusted exposure time do not meet the second preset requirement, The analog gain adjustment operation is performed when the adjusted light intensity reaches the light intensity threshold and the image parameters of the target observation area in the endoscopic image to be tested acquired under the adjusted light intensity do not meet the second preset requirement.

16. An endoscopic observation area tracking system, characterized in that: include: A first acquisition module: used for acquiring first characteristic information of a reference observation area in a reference endoscopic image; The reference endoscopic image includes a target observation object, the reference observation area is an area where the target observation object is located, and the first feature information includes at least one of the following features: a rotation invariance feature, a color feature, and a directional gradient feature; A second acquisition module is used to acquire an endoscopic image to be tested, wherein the endoscopic image to be tested includes the target observation object; A first determination module: used to determine at least one candidate observation area from the endoscopic image to be tested; A third acquisition module: configured to acquire, for each candidate observation area in the at least one candidate observation area, second feature information of the candidate observation area; the second feature information is consistent with the type of features included in the first feature information; A calculation module: configured to calculate, for each candidate observation area in the at least one candidate observation area, a similarity between the candidate observation area and the reference observation area based on the second feature information and the first feature information; The second determination module is used to determine a target observation area based on the similarity corresponding to each of the at least one candidate observation area, where the target observation area is a candidate observation area that contains the same target observation object as the reference observation area.

17. An electronic device comprising a processor and a memory, wherein: The memory stores computer program instructions, which are used by the processor to execute the endoscopic observation area tracking method according to any one of claims 1 to 15 when the processor is running the computer program instructions.

18. A non-volatile storage medium having program instructions stored thereon, wherein the program instructions are used to execute the endoscopic observation area tracking method according to any one of claims 1 to 15 when running.

Images