Method and system for processing panoramic image of endoscope

Abstract

The invention relates to a method and a system for processing a panoramic image of an endoscope. The method comprises the following steps: acquiring a plurality of endoscope images to be spliced; respectively inputting each to-be-spliced endoscope image into a pre-trained endoscope image detection model for focus area detection to obtain a focus area detection result of at least one to-be-spliced endoscope image; wherein the lesion area detection result comprises a probability used for representing that the lesion candidate area selected by the detection frame is the lesion area; determining the lesion candidate area with the probability greater than a preset probability as a lesion area; splicing the plurality of to-be-spliced endoscope images to obtain a panoramic image, and determining the position of the lesion area in the panoramic image based on the position information of the lesion area; and based on the position of the focus area in the panoramic image, prompting and displaying are carried out in a real inner cavity three-dimensional model obtained by fusing the endoscope image and the inner cavity three-dimensional model, so that the accuracy of lesion detection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscope image detection, and in particular to a method and system for processing an endoscope panoramic image. Background Art

[0002] With the development of medical technology, endoscopes have been widely used in various minimally invasive surgeries, such as heart surgery, liver, gallbladder, pancreas, spleen and kidney surgery, etc. In this process, endoscopes provide doctors with a clear field of vision while reducing the area of ​​intraoperative trauma and improving postoperative recovery.

[0003] However, the existing endoscopic diagnosis and treatment technology still has the following problems in its application: for complex lesions, doctors need to rely on rich experience to make accurate judgments, which to a certain extent limits the popularization and application of diagnosis and treatment technology. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method and system for processing an endoscopic panoramic image, which solves the technical problem of inaccurate lesion identification in the prior art.

[0006] (II) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, an embodiment of the present invention provides a method for processing an endoscopic panoramic image, comprising: acquiring a plurality of endoscopic images to be stitched; inputting each of the plurality of endoscopic images to be stitched into a pre-trained endoscopic image detection model for lesion area detection, and obtaining a lesion area detection result of at least one endoscopic image to be stitched; wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area; determining a lesion candidate area with a probability greater than a preset probability as a lesion area; stitching the plurality of endoscopic images to be stitched to obtain a panoramic image, and determining a position of the lesion area in the panoramic image based on the position information of the lesion area; and displaying a prompt in a real endoscopic three-dimensional model obtained by fusing the endoscopic image and the endoscopic three-dimensional model based on the position of the lesion area in the panoramic image.

[0009] In a possible embodiment, the endoscope image detection model includes a Backbone module and a Neck module, the Backbone module includes a SimSPPF module, and the Neck module includes a BiFPN module.

[0010] In a possible embodiment, multiple endoscopic images to be stitched are stitched to obtain a panoramic image, including: preprocessing two endoscopic images to be stitched to obtain two preprocessed images, and performing feature matching and spatial transformation alignment on the two preprocessed images to obtain two registered and aligned images; using a SURF algorithm to perform feature point detection on the two registered and aligned images to obtain feature points of the two registered and aligned images; using a nearest neighbor and next nearest neighbor ratio algorithm to perform coarse matching on the feature points to obtain feature points after coarse matching; using a RANSAC algorithm to eliminate mismatched feature point pairs, and performing image stitching to obtain a panoramic image.

[0011] In one possible embodiment, the preprocessing includes clarity screening, and the clarity screening process includes: graying each of the two endoscopic images to be spliced ​​to obtain two grayscale images; performing Gaussian blur processing on each of the two grayscale images using two different Gaussian kernels to obtain two Gaussian blurred images; subtracting the two Gaussian blurred images corresponding to each grayscale image to obtain the number of pixels with a pixel value of 0 in each grayscale image, and calculating the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels of the corresponding grayscale image; based on the ratio, performing corresponding processing on the grayscale image.

[0012] In one possible embodiment, the grayscale image is processed accordingly based on the ratio, including: comparing the ratio corresponding to the current grayscale image with a preset ratio; if the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, retaining the endoscopic image; if the ratio corresponding to the current grayscale image is greater than the preset ratio, deleting the endoscopic image to be stitched corresponding to the current grayscale image.

[0013] In a second aspect, an embodiment of the present invention provides an endoscopic panoramic image processing system, comprising: an acquisition module for acquiring multiple endoscopic images to be stitched; an input module for inputting each of the multiple endoscopic images to be stitched into a pre-trained endoscopic image detection model for lesion area detection, and obtaining a lesion area detection result of at least one endoscopic image to be stitched; wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area; a determination module for determining a lesion candidate area with a probability greater than a preset probability as a lesion area; a stitching module for stitching the multiple endoscopic images to be stitched to obtain a panoramic image, and determining the position of the lesion area in the panoramic image based on the position information of the lesion area; a prompt display module for prompting and displaying, based on the position of the lesion area in the panoramic image, in a real endoscopic three-dimensional model obtained by fusing the endoscopic image and the endoscopic three-dimensional model.

[0014] In a possible embodiment, the endoscope image detection model includes a Backbone module and a Neck module, the Backbone module includes a SimSPPF module, and the Neck module includes a BiFPN module.

[0015] In one possible embodiment, the stitching module is specifically used to: preprocess two endoscopic images to be stitched to obtain two preprocessed images, and perform feature matching and spatial transformation alignment on the two preprocessed images to obtain two registered and aligned images; use the SURF algorithm to detect feature points of the two registered and aligned images to obtain feature points of the two registered and aligned images; use the nearest neighbor and next nearest neighbor ratio algorithm to roughly match the feature points to obtain feature points after rough matching; use the RANSAC algorithm to eliminate mismatched feature point pairs, and perform image stitching to obtain a panoramic image.

[0016] In one possible embodiment, the preprocessing includes clarity screening; a stitching module is specifically used to perform a clarity screening process, and the clarity screening process includes: graying each of the two endoscopic images to be stitched to obtain two grayscale images; performing Gaussian blur processing on each of the two grayscale images using two different Gaussian kernels to obtain two Gaussian blurred images; subtracting the two Gaussian blurred images corresponding to each grayscale image to obtain the number of pixels with a pixel value of 0 in each grayscale image, and calculating the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels of the corresponding grayscale image; based on the ratio, performing corresponding processing on the grayscale image.

[0017] In one possible embodiment, the stitching module is specifically used to: compare the ratio corresponding to the current grayscale image with a preset ratio; if the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, retain the endoscopic image; if the ratio corresponding to the current grayscale image is greater than the preset ratio, delete the endoscopic image to be stitched corresponding to the current grayscale image.

[0018] In a third aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to execute the method described in the first aspect or any optional implementation of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the method described in the first aspect or any optional implementation method of the first aspect is performed.

[0020] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0021] (III) Beneficial effects

[0022] The beneficial effects of the present invention are:

[0023] The embodiment of the present application provides a method and system for processing an endoscopic panoramic image, by acquiring a plurality of endoscopic images to be spliced, and inputting each of the plurality of endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model for lesion area detection, thereby obtaining a lesion area detection result of at least one endoscopic image to be spliced, wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area, and determining a lesion candidate area with a probability greater than a preset probability as a lesion area; splicing the plurality of endoscopic images to be spliced ​​to obtain a panoramic image, and determining a position of the lesion area in the panoramic image based on the position information of the lesion area, and displaying the lesion area in a real three-dimensional model of the inner cavity obtained by fusing the endoscopic image and the three-dimensional model of the inner cavity based on the position of the lesion area in the panoramic image, thereby improving the accuracy of lesion detection.

[0024] In order to make the above-mentioned objectives, features and advantages to be achieved by the embodiments of the present application more obvious and understandable, the following specifically cites preferred embodiments and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flowchart of a method for processing an endoscope panoramic image provided by an embodiment of the present application is shown;

[0027] Figure 2 A schematic structural diagram of a lesion detection model provided in an embodiment of the present application is shown;

[0028] Figure 3 A flow chart of an endoscopic image stitching method provided in an embodiment of the present application is shown;

[0029] Figure 4 A structural block diagram of an endoscope panoramic image processing system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0031] In order to solve the problem of inaccurate lesion identification in the prior art, an embodiment of the present application provides a method for processing an endoscopic panoramic image, by acquiring multiple endoscopic images to be spliced, and inputting each of the multiple endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model for lesion area detection, thereby obtaining a lesion area detection result of at least one endoscopic image to be spliced, wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area, and determining a lesion candidate area with a probability greater than a preset probability as a lesion area; splicing the multiple endoscopic images to be spliced ​​to obtain a panoramic image, and based on the position information of the lesion area, determining the position of the lesion area in the panoramic image, and based on the position of the lesion area in the panoramic image, prompting and displaying it in a real inner cavity three-dimensional model obtained by fusing the endoscopic image and the inner cavity three-dimensional model, thereby improving the accuracy of lesion detection.

[0032] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0033] See also Figure 1 , Figure 1 A flowchart of a method for processing an endoscope panoramic image provided by an embodiment of the present application is shown. Specifically, the processing method can be executed by a controller, and the specific device of the controller can be set according to actual needs, and the embodiment of the present application is not limited thereto; the processing method includes:

[0034] Step S110, acquiring a plurality of endoscopic images to be stitched.

[0035] Specifically, when the endoscope is inserted into the patient's body, images of the positioning area are captured and transmitted to the endoscopic diagnosis and treatment navigation system in the form of a video stream through the HDMI interface, thereby obtaining multiple endoscopic images for stitching panoramic images.

[0036] Step S120, input each of the multiple endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model to perform lesion area detection, and obtain a lesion area detection result of at least one endoscopic image to be spliced. The lesion area detection result includes position information of a detection frame for selecting a candidate lesion area and a probability for indicating that the candidate lesion area selected by the detection frame is a lesion area.

[0037] It should be understood that the specific model type and structure of the lesion detection model can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0038] Optionally, in the case where the lesion detection model can be a modified YOLOv8 model, see Figure 2 , Figure 2 FIG. 2 shows a schematic diagram of the structure of a lesion detection model provided in an embodiment of the present application. Figure 2 As shown, the lesion detection model includes a Backbone module, a Neck module and a Head module, the SPPF module in the Backbone module is replaced with a pyramid pooling SimSPPF module, and the Neck module includes a BiFPN module.

[0039] Step S130, determining the candidate lesion region with a probability greater than a preset probability as the lesion region. The specific probability of the preset probability can be set according to actual needs, and the embodiment of the present application is not limited thereto.

[0040] Step S140 , stitching the multiple endoscopic images to be stitched together to obtain a panoramic image, and determining the position of the lesion area in the panoramic image based on the position information of the lesion area.

[0041] Step S150 , based on the position of the lesion area in the panoramic image, a prompt is displayed in the real three-dimensional model of the inner cavity obtained by fusing the endoscope image and the three-dimensional model of the inner cavity.

[0042] It should be understood that the real three-dimensional model of the inner cavity is a three-dimensional model that can display the real-time position reached by the tip of the endoscope.

[0043] It should also be understood that the specific display method of the prompt display can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0044] Optionally, based on the position of the lesion area in the panoramic image, the corresponding area of ​​the lesion area in the real three-dimensional model is determined, and the corresponding area in the real three-dimensional model is displayed as a prompt. The prompt display may include at least one of a plurality of prompt display modes such as color distinction, voice broadcast, and area flashing.

[0045] Furthermore, the specific process of stitching the multiple endoscopic images to be stitched in step S140 is as follows:

[0046] See also Figure 3 , Figure 3 A flow chart of an endoscopic image stitching method provided in an embodiment of the present application is shown. Specifically, the endoscopic image stitching method can also be executed by a controller; the endoscopic image stitching method includes:

[0047] Step S310 , preprocessing the two endoscopic images to be stitched to obtain two preprocessed images, and performing feature matching and spatial transformation alignment on the two preprocessed images to obtain two registered and aligned images.

[0048] It should be understood that the processes included in the preprocessing process can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0049] Optionally, the preprocessing process may include distortion correction, image noise reduction and clarity screening.

[0050] Among them, the clarity screening process includes: graying each of the two endoscopic images to be spliced ​​to obtain two grayscale images, and each of the two grayscale images corresponds to an endoscopic image to be spliced. Then, Gaussian blurring is performed on each grayscale image through two different Gaussian kernels (that is, it includes a first Gaussian kernel and a second Gaussian kernel, and the first Gaussian kernel is greater than the second Gaussian kernel, and the first Gaussian kernel and the second Gaussian kernel are both odd numbers) to obtain two Gaussian blurred images, and each grayscale image corresponds to the two Gaussian blurred images. Then, the two Gaussian blurred images corresponding to each grayscale image are subtracted to obtain the number of pixels with a pixel value of 0 in each grayscale image. Then, the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels in the corresponding grayscale image is calculated. Finally, the ratio corresponding to the current grayscale image is compared with the preset ratio. If the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, it is determined that the endoscopic image to be spliced ​​corresponding to the current grayscale image is relatively clear, so that the endoscopic image is retained for subsequent splicing; if the ratio corresponding to the current grayscale image is greater than the preset ratio, it is determined that the endoscopic image to be spliced ​​corresponding to the current grayscale image is blurred, and the endoscopic image to be spliced ​​corresponding to the current grayscale image is deleted. Among them, the specific value of the preset ratio can be set according to actual needs, and the embodiment of the present application is not limited to this. For example, the preset ratio can be 2.

[0051] In addition, if the endoscopic image to be stitched is deleted, the user can be prompted to collect it again.

[0052] It should also be understood that the specific process of distortion correction and the specific process of image noise reduction can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0053] Step S320: Use the SURF algorithm to perform feature point detection on the two registered and aligned images to obtain feature points of the two registered and aligned images.

[0054] Step S330, using the nearest neighbor and next nearest neighbor ratio algorithm to perform rough matching on the feature points to obtain rough matched feature point pairs.

[0055] Step S340: Use the RANSAC algorithm to remove mismatched feature point pairs, and perform image stitching to obtain a panoramic image.

[0056] It should be noted here that Figure 3 The process of stitching two endoscopic images to be stitched is described below. Correspondingly, other endoscopic images are similar and will not be described in detail.

[0057] Therefore, with the help of the above-mentioned technical scheme, the embodiment of the present application obtains a plurality of endoscopic images to be stitched, and inputs each of the plurality of endoscopic images to be stitched into a pre-trained endoscopic image detection model for lesion area detection, thereby obtaining a lesion area detection result of at least one endoscopic image to be stitched, wherein the lesion area detection result includes the position information of a detection frame for selecting a lesion candidate area and the probability for indicating that the lesion candidate area selected by the detection frame is a lesion area, and determining the lesion candidate area with a probability greater than a preset probability as the lesion area; stitching the plurality of endoscopic images to be stitched together to obtain a panoramic image, and based on the position information of the lesion area, determining the position of the lesion area in the panoramic image, and based on the position of the lesion area in the panoramic image, displaying the lesion area in the real three-dimensional model of the inner cavity obtained by fusing the endoscopic image and the three-dimensional model of the inner cavity, thereby improving the accuracy of lesion detection.

[0058] It should be understood that the above-mentioned method for processing endoscopic panoramic images is only exemplary, and those skilled in the art may make various modifications based on the above-mentioned method, and the modified schemes also fall within the protection scope of the present application.

[0059] See also Figure 4 , Figure 4 The structural block diagram of a system 400 for processing an endoscope panoramic image provided by an embodiment of the present application is shown. It should be understood that the processing system 400 is capable of executing each step in the above method embodiment. The specific functions of the processing system 400 can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here. The processing system 400 includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the processing system 400. Specifically, the processing system 400 includes:

[0060] An acquisition module 410 is used to acquire a plurality of endoscopic images to be stitched;

[0061] An input module 420 is used to input each of the multiple endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model to perform lesion area detection, and obtain a lesion area detection result of at least one endoscopic image to be spliced; wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area;

[0062] A determination module 430, configured to determine a candidate lesion region with a probability greater than a preset probability as a lesion region;

[0063] A stitching module 440 is used to stitch a plurality of endoscopic images to be stitched together to obtain a panoramic image, and determine the position of the lesion area in the panoramic image based on the position information of the lesion area;

[0064] The prompt display module 450 is used to display prompts in the real three-dimensional inner cavity model obtained by fusing the endoscope image and the three-dimensional inner cavity model based on the position of the lesion area in the panoramic image.

[0065] In a possible embodiment, the endoscope image detection model includes a Backbone module and a Neck module, the Backbone module includes a SimSPPF module, and the Neck module includes a BiFPN module.

[0066] In one possible embodiment, the stitching module 440 is specifically used to: preprocess two endoscopic images to be stitched to obtain two preprocessed images, and perform feature matching and spatial transformation alignment on the two preprocessed images to obtain two registered images; use the SURF algorithm to detect feature points of the two registered images to obtain feature points of the two registered images; use the nearest neighbor and next nearest neighbor ratio algorithm to roughly match the feature points to obtain feature points after rough matching; use the RANSAC algorithm to eliminate mismatched feature point pairs, and perform image stitching to obtain a panoramic image.

[0067] In one possible embodiment, the preprocessing includes clarity screening; the stitching module 440 is specifically used to perform a clarity screening process, and the clarity screening process includes: graying each of the two endoscopic images to be stitched to obtain two grayscale images; performing Gaussian blur processing on each of the two grayscale images using two different Gaussian kernels to obtain two Gaussian blurred images; subtracting the two Gaussian blurred images corresponding to each grayscale image to obtain the number of pixels with a pixel value of 0 in each grayscale image, and calculating the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels of the corresponding grayscale image; based on the ratio, performing corresponding processing on the grayscale image.

[0068] In one possible embodiment, the stitching module 440 is specifically used to: compare the ratio corresponding to the current grayscale image with a preset ratio; if the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, retain the endoscopic image; if the ratio corresponding to the current grayscale image is greater than the preset ratio, delete the endoscopic image to be stitched corresponding to the current grayscale image.

[0069] Since the device described in the above embodiment of the present invention is a device used to implement the method of the above embodiment of the present invention, based on the method described in the above embodiment of the present invention, a person skilled in the art can understand the specific structure and deformation of the device, so it is not described here. All devices used in the method of the above embodiment of the present invention belong to the scope of protection of the present invention.

[0070] The present application provides a storage medium having a computer program stored thereon. The computer program is executed by a processor to execute the method described in the embodiment.

[0071] The present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.

[0072] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0074] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0075] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0076] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0077] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A method for processing an endoscope panoramic image, characterized in that: include: Acquire multiple endoscopic images to be stitched; Inputting each of the multiple endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model to perform lesion area detection, and obtaining a lesion area detection result of at least one of the endoscopic images to be spliced; wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area; Determine the candidate lesion region whose probability is greater than the preset probability as the lesion region; Stitching the multiple endoscopic images to be stitched together to obtain a panoramic image, and determining the position of the lesion area in the panoramic image based on the position information of the lesion area; Based on the position of the lesion area in the panoramic image, it is displayed in the real three-dimensional model of the inner cavity obtained by fusing the endoscope image and the three-dimensional model of the inner cavity.

2. The processing method according to claim 1, characterized in that: The endoscope image detection model includes a Backbone module and a Neck module, the Backbone module includes a SimSPPF module, and the Neck module includes a BiFPN module.

3. The processing method according to claim 1, characterized in that: The step of stitching the plurality of endoscopic images to be stitched together to obtain a panoramic image includes: Preprocessing the two endoscopic images to be spliced ​​to obtain two preprocessed images, and performing feature matching and spatial transformation alignment on the two preprocessed images to obtain two registered and aligned images; Using the SURF algorithm to perform feature point detection on the two registered and aligned images, and obtaining feature points of the two registered and aligned images; The feature points are roughly matched using the nearest neighbor and next nearest neighbor ratio algorithm to obtain the roughly matched feature points; The RANSAC algorithm is used to remove mismatched feature point pairs, and image stitching is performed to obtain the panoramic image.

4. The processing method according to claim 3, characterized in that: The preprocessing includes clarity screening, and the process of clarity screening includes: Performing grayscale processing on each of the two endoscopic images to be spliced ​​to obtain two grayscale images; Performing Gaussian blur processing on each of the two grayscale images using two different Gaussian kernels to obtain two Gaussian blurred images; Subtracting the two Gaussian blurred images corresponding to each grayscale image from each other to obtain the number of pixels with a pixel value of 0 in each grayscale image, and calculating the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels in the corresponding grayscale image; Based on the ratio, the grayscale image is processed accordingly.

5. The processing method according to claim 4, characterized in that: The grayscale image is processed accordingly based on the ratio, including: The ratio corresponding to the current grayscale image is compared with the preset ratio. If the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, the endoscopic image is retained; if the ratio corresponding to the current grayscale image is greater than the preset ratio, the endoscopic image to be spliced ​​corresponding to the current grayscale image is deleted.

6. A system for processing an endoscope panoramic image, characterized in that: include: An acquisition module, used for acquiring a plurality of endoscopic images to be stitched; An input module, used to input each of the multiple endoscopic images to be spliced ​​into a pre-trained endoscopic image detection model to perform lesion area detection, and obtain a lesion area detection result of at least one of the endoscopic images to be spliced; wherein the lesion area detection result includes position information of a detection frame for selecting a lesion candidate area and a probability for indicating that the lesion candidate area selected by the detection frame is a lesion area; A determination module, used for determining the candidate lesion region with the probability greater than the preset probability as the lesion region; A stitching module, used for stitching the multiple endoscopic images to be stitched to obtain a panoramic image, and determining the position of the lesion area in the panoramic image based on the position information of the lesion area; A prompt display module is used to display a prompt in a real three-dimensional inner cavity model obtained by fusing the endoscope image and the three-dimensional inner cavity model based on the position of the lesion area in the panoramic image.

7. The processing system according to claim 6, characterized in that The endoscope image detection model includes a Backbone module and a Neck module, the Backbone module includes a SimSPPF module, and the Neck module includes a BiFPN module.

8. The processing system according to claim 6, characterized in that The stitching module is specifically used to: pre-process the two endoscopic images to be stitched to obtain two pre-processed images, and perform feature matching and spatial transformation alignment on the two pre-processed images to obtain two registered and aligned images; The SURF algorithm is used to detect feature points of the two registered and aligned images to obtain feature points of the two registered and aligned images; the feature points are roughly matched using the nearest neighbor and next nearest neighbor ratio algorithm to obtain feature points after rough matching; the RANSAC algorithm is used to eliminate mismatched feature point pairs, and images are stitched to obtain the panoramic image.

9. The processing system according to claim 8, characterized in that The pre-processing includes clarity screening; The stitching module is specifically used to perform a clarity screening process, and the clarity screening process includes: graying each of the two endoscopic images to be stitched to obtain two grayscale images; performing Gaussian blur processing on each of the two grayscale images using two different Gaussian kernels to obtain two Gaussian blurred images; subtracting the two Gaussian blurred images corresponding to each grayscale image to obtain the number of pixels with a pixel value of 0 in each grayscale image, and calculating the ratio of the number of pixels with a pixel value of 0 in each grayscale image to the number of all pixels of the corresponding grayscale image; and performing corresponding processing on the grayscale image based on the ratio.

10. The processing system according to claim 9, characterized in that The stitching module is specifically used to: compare the ratio corresponding to the current grayscale image with a preset ratio, and if the ratio corresponding to the current grayscale image is less than or equal to the preset ratio, retain the endoscopic image; If the ratio corresponding to the current grayscale image is greater than the preset ratio, the endoscopic image to be spliced ​​corresponding to the current grayscale image is deleted.

Images