Image processing method and processing system for rectal panoramic images

Abstract

The application provides an image processing method and an image processing system for a rectal panoramic image. The method comprises: synchronously collecting multiple images in a rectal cavity by using multiple image collection units of a panoramic rectal scope, wherein two images collected by two adjacent image collection units have an overlapping area; mapping the multiple images to a cylindrical area of a same cylindrical coordinate system based on a pre-constructed mapping relationship, and determining overlapping areas between multiple cylindrical development images; mutually matching multiple key points in the overlapping areas of adjacent first and second cylindrical development images to obtain multiple target key point pairs; determining correction parameters for aligning the adjacent first and second cylindrical development images based on pose differences of two key points of the multiple target key point pairs; and splicing the adjacent first and second cylindrical development images based on the correction parameters to obtain a target panoramic image.

Description

Technical Field

[0001] This application relates to the fields of image processing and intelligent medical devices, and in particular to an image processing method and system for panoramic images of the rectum. Background Technology

[0002] With the improvement of living standards and the increasing aging of the population, colorectal cancer has become one of the most common malignant tumors worldwide. Rectal lesions are often examined endoscopically. However, current endoscopic image stitching techniques rely heavily on complex motion deduction or simple direct superposition. The presence of intestinal peristalsis interference can lead to errors and jumps in the stitched images, affecting the overall stitching quality. Summary of the Invention

[0003] In view of this, this application provides an image processing method and system for panoramic rectal images.

[0004] One aspect of this application provides an image processing method for panoramic rectal images. The method includes: simultaneously acquiring multiple images within the rectal cavity using multiple image acquisition units of a panoramic proctoscope, wherein two images acquired by two adjacent image acquisition units have overlapping regions; mapping the multiple images to cylindrical regions of the same cylindrical coordinate system based on a pre-constructed mapping relationship to determine the overlapping regions between the multiple cylindrical unfolded images; matching multiple key points in the overlapping regions of adjacent first and second cylindrical unfolded images to obtain multiple target key point pairs, wherein the multiple key points are multiple corner points of the rectal texture that satisfy predetermined texture change conditions; determining correction parameters for aligning adjacent first and second cylindrical unfolded images based on the pose differences determined by the multiple target key points for each pair of key points; and stitching adjacent first and second cylindrical unfolded images together based on the correction parameters to obtain a target panoramic image.

[0005] According to an embodiment of this application, multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded images and second cylindrical unfolded images are matched with each other to obtain multiple target key point pairs. This includes: extracting features from pixels of multiple key points in a predetermined range area of ​​the first cylindrical unfolded image and the second cylindrical unfolded image to obtain multiple first descriptor features of the first cylindrical unfolded image and multiple second descriptor features of the second cylindrical unfolded image; and matching multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded images and second cylindrical unfolded images with each other based on the matching degree between the multiple first descriptor features and the multiple second descriptor features to obtain multiple target key point pairs.

[0006] According to embodiments of this application, based on the matching degree between multiple first descriptor features and multiple second descriptor features, multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded images and second cylindrical unfolded images are matched to obtain multiple target key point pairs. This includes: determining target second descriptor features that match each first descriptor feature from the multiple second descriptor features based on the matching degree between the multiple first descriptor features and multiple second descriptor features; determining target first descriptor features that match each second descriptor feature from the multiple first descriptor features; determining multiple first key point pairs based on the multiple first descriptor features and their corresponding target second descriptor features; determining multiple second key point pairs based on the multiple second descriptor features and their corresponding target first descriptor features; and determining target key point pairs based on the key point pairs that are identical between the multiple first key point pairs and the multiple second key point pairs.

[0007] According to an embodiment of this application, the target second descriptor feature includes a best-matching second descriptor feature and a second-best-matching second descriptor feature determined according to the matching degree; based on multiple first descriptor features and their respective corresponding target second descriptor features, multiple first key point pairs are determined, including: for any first descriptor feature, determining a test ratio based on the distance between the first descriptor feature and the best-matching second descriptor feature and the distance between the first descriptor feature and the second-best-matching second descriptor feature; if the test ratio is less than a predetermined ratio test threshold, obtaining a first key point pair based on the first descriptor feature and the best-matching second descriptor feature.

[0008] According to an embodiment of this application, the method further includes: identifying key points in the overlapping area of ​​the first cylindrical unfolded image and the second cylindrical unfolded image respectively, and determining multiple candidate key points for each of the first cylindrical unfolded image and the second cylindrical unfolded image; filtering the multiple candidate key points for each of the multiple candidate key points based on their reflective features and texture reliability features, and determining multiple key points for each of the first cylindrical unfolded image and the second cylindrical unfolded image, wherein the reflective features are determined based on the specular reflectivity of the wetted surface of the rectum, and the texture reliability features are determined based on the texture complexity of the rectum.

[0009] According to an embodiment of this application, the method further includes: adjusting the correction parameters to basic correction parameters that characterize not performing correction when the number of target keypoint pairs does not meet a first threshold, or the ratio of multiple keypoints to multiple candidate keypoints does not meet a second threshold, or the average error of reprojection of multiple target keypoint pairs after adjustment by correction parameters is greater than a third threshold.

[0010] According to an embodiment of this application, based on correction parameters, adjacent first cylindrical unfolded images and second cylindrical unfolded images are stitched together to obtain a target stitched image. This includes: aligning adjacent first cylindrical unfolded images and second cylindrical unfolded images in an overlapping region based on correction parameters to obtain an aligned overlapping region; constructing a seam cost map of the aligned overlapping region based on the stitching constraint features of multiple pixels in the aligned overlapping region, the seam cost map indicating the unsuitability of any pixel in the aligned overlapping region as a stitching path; determining the target stitching seam based on the seam cost map; and performing multi-band fusion processing on the aligned overlapping regions on both sides of the target stitching seam to obtain a target panoramic image.

[0011] According to an embodiment of this application, the pixel stitching constraint features include gradient differences, color differences, reflective features, wrinkle structure features, and stability features of the pixels in the first cylindrical unfolded map and the second cylindrical unfolded map, as well as the stability features of the pixels across multiple frames. Based on the pixel stitching constraint features of the first cylindrical unfolded map and the second cylindrical unfolded map in the aligned overlapping region, a stitch cost map of the aligned overlapping region is constructed, including: for any pixel in the aligned overlapping region, a weighted summation of gradient differences, color differences, reflective features, wrinkle structure features, and stability features of the pixels across multiple frames is performed to determine the stitch cost of the pixel; and a stitch cost map is constructed based on the stitch costs of multiple pixels.

[0012] According to an embodiment of this application, based on a pre-constructed mapping relationship, multiple images are mapped to cylindrical regions of the same cylindrical coordinate system to obtain multiple cylindrical unfolded images. This includes: based on the pre-constructed mapping relationship, mapping the coordinate sets of each of the multiple images to cylindrical regions of the same cylindrical coordinate system to determine the cylindrical coordinate sets of each of the multiple images; and based on the pre-calibrated array extrinsic parameters of each of the multiple image acquisition units and the field of view coverage angle of each of the multiple image acquisition units, mapping the cylindrical coordinate sets of each of the multiple images to a panoramic canvas coordinate system to determine the overlapping areas between the multiple cylindrical unfolded images.

[0013] Another aspect of this application provides an image processing system for panoramic rectal images. The system includes: a panoramic proctoscope, comprising multiple image acquisition units, each image acquisition unit including multiple first image acquisition units arranged circumferentially and a second image acquisition unit located on top of the multiple first image acquisition units. The multiple first image acquisition units are used to acquire images of the rectal wall, and the second image acquisition unit is used to acquire images of the panoramic proctoscope in the forward direction; and an electronic device electrically connected to the panoramic proctoscope for receiving images acquired by the panoramic proctoscope and performing the aforementioned image processing method.

[0014] According to the technical solution of this application, multiple original images are mapped to the same cylindrical coordinate system through a pre-constructed mapping relationship, effectively eliminating the distortion of the original planar images and generating a cylindrical unfolded image that fits the true morphology of the rectal cavity. Then, based on the target key point pairs in the overlapping areas of adjacent cylindrical unfolded images, pose differences are calculated and correction parameters are determined. These correction parameters can accurately eliminate pose deviations between adjacent unfolded images, achieving seamless alignment and stitching of each unfolded image, ultimately generating a complete, clear, and blind-spot-free panoramic image of the rectal cavity. This at least solves the problem that the presence of intestinal peristalsis interference can cause errors and jumps in the stitched images resulting from the above stitching method. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0016] Figure 1 A flowchart of an image processing method for panoramic rectal images according to an embodiment of this application is shown;

[0017] Figure 2a A schematic diagram of image stitching according to a related embodiment is shown;

[0018] Figure 2b A schematic diagram of image stitching according to an embodiment of this application is shown;

[0019] Figure 2c A schematic diagram illustrating the transformation of cylindrical coordinates into a panoramic canvas coordinate system according to an embodiment of this application is shown;

[0020] Figure 3 A schematic diagram of a panoramic proctoscope according to an embodiment of this application is shown;

[0021] Figure 4 A partially enlarged view of the camera unit according to an embodiment of this application is shown;

[0022] Figure 5 A cross-sectional view of the camera unit according to an embodiment of this application is shown;

[0023] Figure 6 A schematic diagram of the observation area of ​​the camera unit according to an embodiment of this application is shown;

[0024] Figure 7 A block diagram of an electronic device suitable for implementing an image processing method for panoramic rectal images according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or first components, but do not exclude the presence or addition of one or more other features, steps, operations, or first components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] Currently, rectal cancer screening technology faces three main bottlenecks: First, limitations in anatomical structure and field of view. The rectal wall has complex folds such as the semilunar folds. Traditional proctoscopy or colonoscopy typically only has one forward-facing camera, resulting in a limited field of view and making it difficult to simultaneously observe the anterior and posterior aspects of the intestinal lumen, easily leading to missed diagnoses. Second, operational difficulty and image cognitive load. To observe the entire circumference of the intestinal wall, doctors need to repeatedly rotate the scope and adjust the bends, which is cumbersome and increases patient discomfort. Third, inadequacies in image stitching technology. Existing endoscopic image stitching technologies mostly rely on complex motion deduction or simple direct superposition. The former is computationally intensive and easily affected by intestinal peristalsis, while the latter is prone to producing obvious ghosting, misalignment, and brightness jumps when dealing with areas of large parallax and weak texture, failing to form a geometrically correct and visually natural complete panoramic image for diagnosis. Therefore, there is an urgent need for a proctoscopy system that can cover the entire circumference with a single insertion and generate a high-quality two-dimensional panoramic unfolded image in real time using efficient algorithms.

[0030] In view of this, this application provides an image processing method and image processing system for panoramic rectal images, in order to solve at least one of the above-mentioned problems.

[0031] Figure 1 A flowchart of an image processing method for panoramic rectal images according to an embodiment of this application is shown.

[0032] like Figure 1 As shown, the method includes operations S110 to S150.

[0033] During operation of S110, multiple images are simultaneously acquired within the rectal cavity using multiple image acquisition units of the panoramic proctoscope. Among these images, there is an overlapping area between the two images acquired by two adjacent image acquisition units.

[0034] In operation S120, based on the pre-built mapping relationship, multiple images are mapped to cylindrical regions of the same cylindrical coordinate system, and the overlapping areas between multiple cylindrical unfolded images are determined.

[0035] In operation S130, multiple key points in the overlapping area of ​​the adjacent first cylindrical unfolded map and second cylindrical unfolded map are matched with each other to obtain multiple target key point pairs. The multiple key points are multiple corner points of the rectum texture that satisfy the predetermined texture change conditions.

[0036] In operation S140, based on the pose difference determined by multiple target key points for each of the two key points, correction parameters are determined for aligning adjacent first cylindrical unfolded diagrams and second cylindrical unfolded diagrams.

[0037] In operation S150, based on the correction parameters, the adjacent first cylindrical unfolded image and second cylindrical unfolded image are stitched together to obtain the target panoramic image.

[0038] In embodiments of this application, the multiple image acquisition units of the panoramic proctoscope may include multiple first image acquisition units arranged circumferentially and a second image acquisition unit located on top of the multiple first image acquisition units. The multiple first image acquisition units are used to acquire images of the inner wall of the rectum, and the second image acquisition unit is used to acquire images of the panoramic proctoscope in the forward direction, forming a "top view + circumferential side view" multi-channel verification array layout.

[0039] For example, before acquiring images using multiple image acquisition units of a panoramic proctoscope, the internal parameters of the five lenses can be completed during the initialization phase. Distortion parameters With array extrinsic parameters Precise calibration provides the geometric basis for subsequent mapping and stitching. To address inconsistencies in brightness and color caused by differences in lighting from multiple lenses and variations in the angle of the intracavity light source, brightness equalization can be performed first. Brightness equalization achieves cross-viewpoint color consistency calibration by dynamically adjusting the Gamma value and matching it with the histogram.

[0040] For example, a field-programmable gate array (FPGA) can be used to generate a unified master clock and send commands to multiple image acquisition units simultaneously via hardware trigger lines to achieve microsecond-level frame synchronization acquisition. This helps suppress time differences between viewing angles caused by rapid endoscope movement or intestinal peristalsis.

[0041] A two-way mapping relationship can be established between the coordinates of the original acquired image and cylindrical coordinates, enabling reversible geometric transformations within the same cylindrical coordinate system. Cylindrical coordinates are used... The output resolution uses For each image acquisition unit, a bidirectional mapping table can be generated offline and stored as a lookup table. During the online image acquisition phase, the images captured at time t... The cylindrical unfolded image is obtained by mapping. .

[0042] The overlapping areas between multiple cylindrical unfolded diagrams include the overlapping area between two adjacent side views, and the overlapping area between the top view and the side view. The overlapping area between adjacent first and second cylindrical unfolded diagrams (denoted as...) ),extract Corner points within an image can be points in the image where the texture changes drastically or where there are obvious turning points, such as the intersection of rectal mucosal folds or the branching points of blood vessels. Corner points are unique and stable and can be used as core references for stitching.

[0043] The pre-defined texture variation conditions are pre-set screening criteria used to exclude points with flat textures and no obvious features, such as smooth and wrinkle-free mucosal areas, to ensure that the extracted key points can be accurately matched and to avoid false matches.

[0044] The key points in the overlapping areas of the first and second cylindrical unfolded diagrams are matched with each other. Each pair of corresponding key points, that is, a corner point in the first cylindrical unfolded diagram and a corner point in the corresponding position in the second cylindrical unfolded diagram, constitutes a target key point pair.

[0045] Due to factors such as acquisition perspective, intestinal peristalsis, and mapping errors, there may be certain pose differences between two key points in a target key point pair. Pose differences include position deviation and orientation deviation. Position deviation is the coordinate offset of the two points in the image coordinate system, while orientation deviation is the difference in the angle and scaling ratio of the corresponding textures of the two points.

[0046] For example, algorithms such as least squares method and random sampling consensus can be used to estimate correction parameters for multiple target key point pairs. Correction parameters It is used to compensate for minor relative pose disturbances caused by assembly errors, temperature drift, and slight creep.

[0047] For example, constraints can be imposed on the correction magnitude. . This represents the correction amplitude threshold. This constraint avoids introducing large-scale deformation fitting and improves stability under conditions of weak intestinal texture and reflectivity.

[0048] The correction parameters can be applied to the adjacent first and second cylindrical unfolded diagrams. The position, angle rotation, and scaling of one of the unfolded diagrams can be adjusted to make the overlapping areas of the two unfolded diagrams completely aligned (eliminating pose deviations). Then, the overlapping areas are fused, such as by weighted fusion, to avoid obvious discontinuities and ghosting at the splicing point, and to seamlessly stitch the two unfolded diagrams into a whole.

[0049] Following the above procedure, all adjacent cylindrical unfolded images can be stitched and fused sequentially, ultimately integrating all unfolded images to obtain a target panoramic image covering the entire inner wall of the rectal cavity. This target panoramic image is free of blind spots and distortion, completely and clearly presenting the true morphology of the rectal cavity, providing intuitive and accurate imaging evidence for clinical diagnosis.

[0050] According to embodiments of this application, multiple acquired original images are mapped to the same cylindrical coordinate system through a pre-constructed mapping relationship, effectively eliminating distortions in the original planar images and generating a cylindrical unfolded image that closely matches the true morphology of the rectal cavity. Then, based on target key point pairs in the overlapping areas of adjacent cylindrical unfolded images, pose differences are calculated and correction parameters are determined. These correction parameters can accurately eliminate pose deviations between adjacent unfolded images, achieving seamless alignment and stitching of each unfolded image, ultimately generating a complete, clear, and blind-spot-free panoramic image of the rectal cavity. This at least addresses the problem that the presence of intestinal peristalsis interference can cause errors and abrupt changes in the stitched images resulting from the aforementioned stitching method.

[0051] According to embodiments of this application, mapping multiple images to cylindrical regions of the same cylindrical coordinate system based on a pre-built mapping relationship to obtain multiple cylindrical unfolded images may include: mapping the coordinate sets of each of the multiple images to cylindrical regions of the same cylindrical coordinate system based on a pre-built mapping relationship to determine the cylindrical coordinate sets of each of the multiple images; and mapping the cylindrical coordinate sets of each of the multiple images to a panoramic canvas coordinate system based on the pre-calibrated array extrinsic parameters of each of the multiple image acquisition units and the field of view coverage angle of each of the multiple image acquisition units to determine the overlapping areas between the multiple cylindrical unfolded images.

[0052] For the multiple original images acquired, all pixel coordinates of each image are extracted to form their respective coordinate sets. Then, through a pre-constructed mapping relationship, the coordinate sets of each image are mapped one by one to the corresponding cylindrical regions of the same cylindrical coordinate system. After coordinate transformation, the cylindrical coordinate set corresponding to each image is obtained. This eliminates the distortion of the original planar image and ensures that the image coordinates accurately correspond to the actual spatial location of the rectal cavity.

[0053] For example, when the second image acquisition unit is an ultra-wide-angle fisheye lens, traditional fisheye image correction methods usually use an irreversible checkerboard method to eliminate barrel distortion. The top view acquired by the second image acquisition unit is used as the main view. The side view image is prone to deformation and distortion during the stitching process, which affects the accuracy of lesion identification.

[0054] According to the solution of this application embodiment, by establishing a bidirectional mapping relationship between cylindrical coordinates and fisheye coordinates, the top view can be horizontally expanded into a long strip area occupying the center of the panoramic image, thereby achieving reversible geometric transformation while preserving the original pixel information.

[0055] Array extrinsic parameters are pre-calibrated parameters characterizing the spatial position and orientation of each acquisition unit, used to define the relative positional relationships between different acquisition units. The field of view coverage angle is the angle of the rectal cavity range that each acquisition unit can capture. Based on the pre-calibrated array extrinsic parameters and field of view coverage angle, the images of each cylindrical path are unfolded. Placed into a unified panoramic canvas coordinate system, a coarse stitching result is obtained. It also calculates the overlapping areas between multiple cylindrical unfolded diagrams.

[0056] According to the embodiments of this application, by gradually transforming the original image into cylindrical coordinates and then into panoramic canvas coordinates, image distortion is eliminated through cylindrical mapping. Furthermore, by using the calibration parameters and field of view of the acquisition unit, the overlapping area between the cylindrical unfolded images is accurately located, laying the foundation for subsequent determination of correction parameters and seamless image stitching.

[0057] Figure 2a A schematic diagram of image stitching according to a related embodiment is shown. Figure 2b A schematic diagram of image stitching according to an embodiment of this application is shown. Figure 2c A schematic diagram illustrating the transformation of cylindrical coordinates into a panoramic canvas coordinate system according to an embodiment of this application is shown.

[0058] like Figure 2aAs shown, the top original view is used as the main view, and the top original view, the sitting original view, and the right original view are stitched together with the top original view using a checkerboard method to obtain a stitched panoramic image. However, this method will cause the loss of edge information, and the side view image is prone to deformation and distortion during the stitching process, which will affect the accuracy of lesion identification.

[0059] like Figure 2b As shown, this embodiment employs a horizontal unfolding method, expanding the central circular fisheye view into a long strip area occupying the center of the panoramic image, thus avoiding the loss of edge information. This is achieved by establishing cylindrical coordinates. fisheye coordinates The bidirectional mapping relationship between them enables reversible geometric transformations while preserving the original pixel information.

[0060] like Figure 2c As shown, for a given three-dimensional point Perform spherical coordinate transformation ,satisfy:

[0061]

[0062] Then the spherical coordinates are mapped to the pixel coordinates of the equidistant cylindrical projection plane. ,satisfy:

[0063]

[0064] in For panoramic image resolution, the inverse transformation from the equidistant cylindrical projection plane to spherical coordinates is readily obtained as:

[0065]

[0066] According to an embodiment of this application, before matching multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded images and second cylindrical unfolded images, the method may further include: identifying key points in the overlapping area of ​​the first cylindrical unfolded image and second cylindrical unfolded image, and determining multiple candidate key points for each of the first cylindrical unfolded image and second cylindrical unfolded image; filtering the candidate key points for each of the multiple candidate key points based on their reflective features and texture reliability features, and determining multiple key points for each of the first cylindrical unfolded image and second cylindrical unfolded image, wherein the reflective features are determined based on the specular reflectivity of the wetted surface of the rectum, and the texture reliability features are determined based on the texture complexity of the rectum.

[0067] For example, the specular reflectivity of the moistened surface of the rectum can be determined by combining a brightness threshold with saturation. Based on the specular reflectivity of the moistened surface of the rectum, reflective masks can be constructed for each candidate key point. The value can be 0 or 1, where 1 represents a reflective or highlighted area and 0 represents a non-reflective area. The reflective characteristics of each candidate keypoint are determined based on the reflective mask of each candidate keypoint.

[0068] For example, the complexity of each candidate keypoint can be determined based on its local gradient energy or structural response, thereby determining the texture reliability features of each candidate keypoint. The value ranges from 0 to 1. The smaller the value, the weaker the texture and the lower the reliability.

[0069] For example, a reflective mask can be used. Texture reliability features Gating of candidate pixels. This will satisfy... Regions that meet the criteria are considered unreliable matching regions and are removed or downweighted in subsequent feature detection and matching. Alternatively, regions that satisfy the criteria can be... Less than the reliability threshold The affected areas are considered weak texture degradation zones, and only low-degree-of-freedom stitching methods are allowed to prevent high-degree-of-freedom stitching from causing drift and amplification under weak textures. An effective matching domain for overlapping areas can be defined. .

[0070] The reflective coverage ratio can be further calculated. and effective texture ratio It can be and As part of the evaluation metrics for matching quality.

[0071] By analyzing the reflectivity and texture reliability characteristics of multiple candidate keypoints, keypoints that are non-reflective and have high texture reliability can be identified. Alternatively, the weights of each candidate keypoint can be determined based on their respective reflectivity and texture reliability characteristics; these weights can then be used as weights for calculating correction parameters based on the target keypoint.

[0072] According to embodiments of this application, by screening candidate key points based on reflective features and texture reliability features, false corners and unreliable texture regions with intestinal wet reflectivity can be explicitly suppressed, thereby improving the accuracy of candidate key point matching.

[0073] According to embodiments of this application, matching multiple key points in the overlapping region of adjacent first cylindrical unfolded images and second cylindrical unfolded images to obtain multiple target key point pairs may include: extracting features from pixels of multiple key points in a predetermined range area of ​​the first cylindrical unfolded image and the second cylindrical unfolded image to obtain multiple first descriptor features of the first cylindrical unfolded image and multiple second descriptor features of the second cylindrical unfolded image; and matching multiple key points in the overlapping region of adjacent first cylindrical unfolded images and second cylindrical unfolded images based on the matching degree between the multiple first descriptor features and the multiple second descriptor features to obtain multiple target key point pairs.

[0074] The predetermined range region can be a local pixel region defined around each keypoint, for example, a 3×3 or 5×5 pixel block centered on the keypoint. The descriptor feature represents the feature vector after quantizing the texture, grayscale, edge, and other information of this local pixel region.

[0075] Define several key points of the first cylindrical surface development diagram as follows: Several key points of the second cylindrical surface development diagram are For multiple key points and several key points By extracting features from pixels within a predetermined range, multiple descriptor sets of the first cylindrical unfolded image can be obtained. The second cylindrical surface development diagram .

[0076] For example, non-maximum suppression can be applied near reflective boundaries to reduce false corners caused by highlight edges. Alternatively, a weak texture fallback implementation can be used to... The segmentation is divided into small overlapping patches, and descriptor features are extracted using a contrastive learning encoder.

[0077] Two descriptor features and The higher the degree of matching between them, the more similar the local textures and details of the two key points are, and the more likely these two key points are to be key points in the same location.

[0078] By comparing the matching degree of all first descriptor features with second descriptor features, feature pairs with a matching degree higher than a preset threshold can be selected. Each successfully matched descriptor feature corresponds to a pair of key points in the overlapping area of ​​the two unfolded images, and this pair of key points is a target key point pair.

[0079] According to embodiments of this application, by first extracting descriptive features of key points and then matching key points using descriptive features, rich reference features can be provided for key point matching, ensuring that the matched target key points accurately correspond to the same area of ​​the rectal cavity.

[0080] According to embodiments of this application, based on the matching degree between multiple first descriptor features and multiple second descriptor features, matching multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded images and second cylindrical unfolded images to obtain multiple target key point pairs may include: determining target second descriptor features that match each first descriptor feature from the second descriptor features based on the matching degree between the multiple first descriptor features and multiple second descriptor features; determining target first descriptor features that match each second descriptor feature from the first descriptor features; determining multiple first key point pairs based on the multiple first descriptor features and their corresponding target second descriptor features; determining multiple second key point pairs based on the multiple second descriptor features and their corresponding target first descriptor features; and determining target key point pairs based on the key point pairs that are the same between the multiple first key point pairs and the multiple second key point pairs.

[0081] For example, the matching degree between each first descriptor feature and all second descriptor features can be calculated, and the second descriptor feature with the highest matching degree and above a preset threshold can be selected as the target second descriptor feature corresponding to the first descriptor feature. At the same time, the matching degree between each second descriptor feature and all first descriptor features can be calculated in reverse, and the first descriptor feature with the highest matching degree and above a preset threshold can be selected as the target first descriptor feature corresponding to the second descriptor feature.

[0082] Each first descriptor feature is associated with its corresponding target second descriptor feature. Since each descriptor feature corresponds to a keypoint in a unfolded graph, each associated "first descriptor feature - target second descriptor feature" corresponds to a keypoint in the first cylinder unfolded graph and a keypoint in the second cylinder unfolded graph, thus obtaining multiple first keypoint pairs. Similarly, multiple second keypoint pairs can also be obtained.

[0083] Keypoint pairs that match each other and pass two-way verification can be used as target keypoint pairs. Matching pairs that only exist in a certain set of keypoint pairs are judged as one-way false matches and are removed.

[0084] According to embodiments of this disclosure, a bidirectional matching mechanism is used to verify each pair of key points bidirectionally, eliminating false matching pairs that may occur in unidirectional matching due to "similar features but not in the same location", such as mismatches caused by similar textures of rectal mucosa, ensuring that the final target key point pairs are key points in the same location of the rectal cavity within the overlapping area of ​​the two unfolded images, thereby reducing matching errors.

[0085] According to an embodiment of this application, the target second descriptor feature includes a best-matching second descriptor feature and a second-best-matching second descriptor feature determined according to the matching degree; based on multiple first descriptor features and their respective corresponding target second descriptor features, multiple first key point pairs are determined, including: for any first descriptor feature, determining a test ratio based on the distance between the first descriptor feature and the best-matching second descriptor feature and the distance between the first descriptor feature and the second-best-matching second descriptor feature; if the test ratio is less than a predetermined ratio test threshold, obtaining a first key point pair based on the first descriptor feature and the best-matching second descriptor feature.

[0086] For example, the Euclidean distance between the first descriptor feature and the best-matching second descriptor feature, and the Euclidean distance between the first descriptor feature and the second-best-matching second descriptor feature can be calculated. Then, the ratio of these two distances is calculated to obtain the test ratio. The calculation formula is as follows:

[0087] ;

[0088] When the test ratio is less than a predetermined test threshold In this case, it indicates that the similarity between the best matching second descriptor feature and the first descriptor feature is much higher than that between the second matching second descriptor feature. This match has uniqueness and reliability. At this time, based on the first descriptor feature and its corresponding best matching second descriptor feature, a pair of first keypoints is determined.

[0089] When the test ratio is greater than or equal to a predetermined test threshold. This indicates that the similarity between the most matching and the second most matching features is not significant, the uniqueness of the match is insufficient, and there is a risk of false matching. In this case, the matching relationship is removed, and the corresponding first keypoint pair is not generated.

[0090] According to embodiments of this application, bidirectional matching combined with ratio testing can effectively suppress duplicate textures and false matches, thereby further improving matching accuracy.

[0091] According to an embodiment of this application, after obtaining multiple target keypoint pairs, the method may further include: if the number of target keypoint pairs does not meet a first threshold, or the ratio of multiple keypoints to multiple candidate keypoints does not meet a second threshold, or the average error of the reprojection of multiple target keypoint pairs after adjustment by the correction parameters is greater than a third threshold, the correction parameters are adjusted to the basic correction parameters that characterize not performing correction.

[0092] The first threshold is used to check the number of target keypoint pairs, representing the minimum number of valid matching pairs required. Only when the number of target keypoint pairs reaches or exceeds the first threshold can sufficient reliable data be provided for the calculation of correction parameters, avoiding excessive deviation of correction parameters due to too few matching pairs.

[0093] The second threshold is used to test the ratio of multiple key points to multiple candidate key points, representing the minimum requirement for matching success rate. If the ratio is greater than or equal to the second threshold, it indicates that the effective matching rate of the initial key points is high and the matching result is reliable; if the ratio is too low, it indicates that a large number of initial key points have not been matched successfully and the matching result is abnormal.

[0094] The third threshold is used to check the average reprojection error of all target keypoints after adjustment by the correction parameters. If the average reprojection error is less than the third threshold, it indicates that the correction parameters can effectively align the two unfolded images, and the correction effect meets the standard. If the average reprojection error is greater than the third threshold, it indicates that the correction effect does not meet the standard.

[0095] If any of the above conditions are met, the correction parameters calculated based on the target key points will be adjusted to the basic correction parameters. That is, no position, angle, or scaling corrections will be performed on adjacent cylindrical unfolded images. The cylindrical unfolded images obtained from the mapping relationship can be directly stitched together, thus avoiding reliance on large-scale feature tracking under creeping and reflective conditions and suppressing drift accumulation.

[0096] For example, the correction parameters can also be adjusted. Introduce time smoothing, and according to Update the correction parameters to suppress inter-frame jitter.

[0097] According to an embodiment of this application, stitching adjacent first cylindrical unfolded images and second cylindrical unfolded images together based on correction parameters to obtain a target stitched image may include: aligning adjacent first cylindrical unfolded images and second cylindrical unfolded images in an overlapping region based on correction parameters to obtain an aligned overlapping region; constructing a seam cost map of the aligned overlapping region based on the stitching constraint features of multiple pixels in the aligned overlapping region, the seam cost map indicating the unsuitability of any pixel in the aligned overlapping region as a stitching path; determining the target stitching seam based on the seam cost map; and performing multi-band fusion processing on the aligned overlapping regions on both sides of the target stitching seam to obtain a target panoramic image.

[0098] For example, it can be done in each overlapping area Internal suture cost and solution for optimal suture This reduces the interference of local misalignment caused by creep deformation and viewing angle differences on lesion identification. Let the two aligned cylindrical images be... and .

[0099] The pixel stitching constraint features can include the gradient difference, color difference, reflectivity features, wrinkle structure features, and stability features of pixels in the first cylindrical unfolded map and the second cylindrical unfolded map, as well as the stability features of pixels across multiple frames.

[0100] According to an embodiment of this application, constructing a stitch cost map for the aligned and overlapping region based on the pixel splicing constraint features of the first cylindrical unfolded map and the second cylindrical unfolded map in the aligned and overlapping region may include: for any pixel in the aligned and overlapping region, performing a weighted summation of gradient difference, color difference, reflective features, wrinkle structure features, and stability features of the pixel across multiple frames to determine the stitch cost of the pixel; and constructing a stitch cost map based on the stitch costs of multiple pixels.

[0101] For example, the formula for the suture cost diagram is as follows:

[0102] ;

[0103] in, The image representing time t is in The stitch cost per pixel; a larger value indicates that the stitch is less likely to pass through that pixel. This indicates the weight of each item. Represents the gradient operator; Higher penalties are applied to areas with prominent folds and vascular patterns to prevent sutures from crossing critical structures and causing ghosting. A higher penalty is applied to reflective areas to suppress stitching through specular highlights. This reduces the cost of areas near the stitching in the previous frame, thus suppressing stitch jitter in the video stream.

[0104] For example, graphical cutting can be used to solve for the optimal suture line. This can be done... Create a pixel grid map Each pixel p∈ A node in the corresponding pixel grid map The spatial relationship between adjacent pixels corresponds to the edges of the graph. It can be done for each pixel. Assign binary labels And construct the energy function:

[0105] ;

[0106] in, This represents the graph cut energy function, and the optimal label is obtained through the minimum cut. . Pixel label. Indicates that the pixel comes from the view. or view . This represents the neighborhood system, which can be either 4-neighborhood or 8-neighborhood. Data items can represent pixels. Assign tags The cost of the stitches. This indicates a smoothing term, used to penalize inconsistent labels between adjacent pixels and encourage smooth boundary lines. Indicator function. Returns 1 if the labels are different, otherwise returns 0.

[0107] The solution can be obtained using minimum cut and maximum flow algorithms. Find the minimum value to obtain the optimal label allocation. Its boundary set. Specifically, virtual source point s (representing label i) and sink point t (representing label j) can be added to the pixel mesh graph. Each pixel node Connect nodes s and t, and find a set of edges (cuts) that divide the graph into two parts (the set to which s belongs and the set to which t belongs) such that the sum of the cut weights is minimized. This cut corresponds to: node label i on the s side, node label j on the t side; edges between pixel nodes and s / t (data cost) and edges between adjacent pixels (smoothing cost). According to the maximum flow-minimum cut theorem, the weight of the minimum cut is equal to the maximum flow value from s to t. By solving for the maximum flow using standard algorithms, the minimum cut can be obtained, thus determining the optimal label assignment L* and the set of boundary edges (i.e., cut edges).

[0108] The set of boundary edges can be defined as the optimal seam. Preferred boundary weights In non-overlapping areas, use data items By forcing pixels to select a unique source, data items are made equivalent to two labels within the overlap band, thus making the suture position primarily determined by the suture cost map.

[0109] In image stitching, the target stitching seam is the path with the least difference in the overlapping area of ​​the two images, which can reduce stitching artifacts.

[0110] After obtaining the suture, multi-band fusion is performed in a banded region adjacent to the suture, thereby smoothing the transition of low-frequency brightness while maintaining the continuity of high-frequency details. A banded fusion region can be defined. .in This represents half the bandwidth, preferably w=64 pixels, but can be selected from 32 pixels to 64 pixels. It can be used only... The Laplace pyramid fusion is performed inside the strip area, while outside the strip area, it is categorized by label. Directly retrieve the corresponding view pixels. The preferred pyramid layer count is L=5, but 4 to 6 layers are also possible. This can be applied to... Generated fusion mask Building the Gauss Pyramid And construct the Laplace pyramid from the two images. and You can press... The fused image is obtained by fusing layers one by one and reconstructing from top to bottom. The final output is a panoramic unfolded image. .

[0111] According to embodiments of this application, a seam cost map is constructed using gradient differences, color differences, reflective features, wrinkle structure features, and pixel stability features across multiple frames. A graph cut algorithm is then used to solve for the globally optimal seam, enabling the stitching boundary to actively avoid wrinkle edges and highlight areas. Furthermore, multi-band fusion is performed in the seam neighborhood band region, achieving smooth transitions in low-frequency brightness and continuity in high-frequency details, significantly reducing ghosting, skipped seams, and localized tearing.

[0112] This application also provides an image processing system for panoramic rectal images. The system includes: a panoramic proctoscope comprising multiple image acquisition units, each image acquisition unit including multiple first image acquisition units arranged circumferentially and a second image acquisition unit located on top of the multiple first image acquisition units. The multiple first image acquisition units are used to acquire images of the rectal wall, and the second image acquisition unit is used to acquire images of the panoramic proctoscope in the forward direction. An electronic device, electrically connected to the panoramic proctoscope, is used to receive the images acquired by the panoramic proctoscope and perform the above-described image processing method.

[0113] Figure 3 A schematic diagram of a panoramic proctoscope according to an embodiment of this application is shown.

[0114] In one example, such as Figure 3 As shown, the panoramic proctoscope can include a camera unit, a curved section, and an insertion tube. The camera unit includes multiple image acquisition units. The curved section 2 uses a medical-grade stainless steel braided tube 20 and a snake-bone joint 21 structure, providing high flexibility and high strength mechanical support. The curved section 2 uses a snake-bone joint 21 structure, with four sets of nickel-titanium alloy drive wires 22 circumferentially distributed at 90° intervals. By changing the length of the four sets of drive wires 22, the curved section 2 adjusts the gap between the snake-bone joints 21, allowing the camera end 1 to deflect in four directions within the lumen. The curved section 2 fine-tunes the overall posture of the camera unit 1 to avoid excessive contact between one side of the camera and the intestinal wall due to its own weight or the patient's position. During adjustment, the center of the second image acquisition unit points towards the intestinal lumen. The insertion tube 3 has an effective working length of 18cm, and its flexible and slender shape allows for non-invasive insertion under natural relaxation of the anal sphincter.

[0115] Figure 4 A partially enlarged view of the camera unit according to an embodiment of this application is shown.

[0116] In one example, such as Figure 4As shown, the multiple image acquisition units of the camera unit can be an integrated multi-camera array. The second image acquisition unit is an ultra-wide-angle fisheye lens 11 located at the geometric center of the endoscope's top. Its optical axis is parallel to the endoscope's axis, and its focal length is set to 1.2mm, providing a hemispherical field of view of approximately 220° to 230°. It is mainly responsible for establishing a global coordinate system reference and covering the deep field of view in the direction of intestinal lumen movement. On the side of the top, four lateral wide-angle lenses 10a, 10b, 10c, and 10d are evenly distributed at 90° intervals along the circumference, forming an orthogonal observation matrix, serving as multiple first image acquisition units. The optical axis of the first image acquisition unit is perpendicular to the endoscope's axis, with a single lens having a field of view of approximately 150° and a focal length of approximately 4mm. It is used to capture details near the intestinal wall and behind anatomical folds, avoiding blind spots.

[0117] The illumination system features an adjustable-intensity xenon lamp cold light source. A beam splitter distributes the dominant beam to four ball-head optical fibers 12a, 12b, 12c, and 12d, creating a ring-shaped illumination field around each lens to ensure consistent color temperature and uniform brightness across multiple viewing angles. Inflation of the intestinal lumen via nozzle 13 fully expands the rectal mucosa, facilitating observation and image acquisition. The front-end sensor is a high-sensitivity CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, supporting high dynamic range acquisition to adapt to the complex lighting environment within the intestine, where reflective and dark areas coexist.

[0118] Figure 5 A cross-sectional view of the camera unit according to an embodiment of this application is shown.

[0119] In one example, such as Figure 5 As shown, considering the limitations of physical space, the fisheye lens 11 is located at the center of symmetry of the rectangular cross-section, as shown in the cross-sectional view of the camera end. Figure 4 Pinhole lenses 10a, 10b, 10c, and 10d are compactly located on the side edges of the rectangular cross-section, while ball-head optical fibers 12a, 12b, 12c, and 12d are located at the four vertices of the rectangular cross-section. Preferably, four sets of pinhole lenses are distributed to facilitate the control of the pose of the four-way camera by four sets of drive wires 22 evenly distributed in the circumferential direction of the snake-bone structure.

[0120] Figure 6 A schematic diagram of the observation area of ​​the camera unit according to an embodiment of this application is shown.

[0121] In one example, such as Figure 6 As shown, the field of view of the top fisheye lens 11 is at the apex angle of... The spherical fan-shaped coverage, the circumferential array of pinhole lenses 10 respectively covering the apex corner is The image is covered in a spherical fan shape. Multiple images are acquired in real time through the image acquisition device, including the original top view captured by the fisheye lens 11, and the original top, bottom, left, and right views captured by the side pinhole lenses respectively.

[0122] According to embodiments of this application, a composite multi-view optical array is constructed by employing a central ultra-wide-angle fisheye lens and circumferentially distributed lateral wide-angle lenses. The central lens provides forward-facing overall scene perception. The circumferential lenses supplement the acquisition of images from the surrounding intestinal wall and the dorsal folded region. The multiple fields of view are spatially complementary and form a continuous overlapping band. Hardware-level synchronous triggering enables simultaneous acquisition of multiple video streams in the same frame, reducing misalignment caused by cross-view time differences due to peristalsis and endoscopic movement from the source. This structure achieves blind-spot-free visualization of the rectal circumference and significantly improves the observability of the lower rectum and the dorsal folded region.

[0123] The image processing system of this application can output an intuitive two-dimensional panoramic video stream and static images of the rectal circumference, facilitating doctors to quickly complete circumferential observation and locate suspected lesion areas in a single view. Furthermore, based on the correction and degradation mechanism of this application, it can maintain continuous and stable display under complex conditions such as enhanced reflection or insufficient texture, reducing the operational burden on doctors caused by image jitter and stitching failures. This solution has controllable computational power dependence, is suitable for bedside rapid examination and large-scale screening scenarios, and is feasible for promotion in primary healthcare institutions.

[0124] Figure 7 A block diagram of an electronic device suitable for implementing an image processing method for panoramic rectal images according to an embodiment of the present disclosure is shown schematically.

[0125] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0126] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0127] Multiple first components in electronic device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of display, speaker, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the log diagnostic method. For example, in some embodiments, the log diagnostic method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the log diagnostic method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the log diagnostic method by any other suitable means (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable test apparatus, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0132] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. An image processing method for panoramic rectal images, characterized in that, The method includes: Multiple images are simultaneously acquired within the rectal cavity using multiple image acquisition units of a panoramic proctoscope, with overlapping areas between images acquired by two adjacent image acquisition units. Based on the pre-built mapping relationship, multiple images are mapped to cylindrical regions of the same cylindrical coordinate system to determine the overlapping areas between multiple cylindrical unfolded diagrams. Multiple key points in the overlapping area of ​​adjacent first and second cylindrical unfolded diagrams are matched to obtain multiple target key point pairs. The multiple key points are multiple corner points of the rectum whose texture satisfies the predetermined texture change conditions. Based on the pose differences determined by multiple target key points for each of the two key points, correction parameters are determined for aligning adjacent first cylindrical surface unfolded diagrams and second cylindrical surface unfolded diagrams. Based on the correction parameters, adjacent first cylindrical surface unfolded images and second cylindrical surface unfolded images are stitched together to obtain a panoramic image of the target.

2. The method according to claim 1, characterized in that, The process involves matching multiple key points in the overlapping region of adjacent first and second cylindrical unfolded diagrams to obtain multiple pairs of target key points, including: Feature extraction is performed on pixels of multiple key points in a predetermined range area for each of the first and second cylindrical unfolded images to obtain multiple first descriptor features of the first cylindrical unfolded image and multiple second descriptor features of the second cylindrical unfolded image. Based on the matching degree between multiple first descriptor features and multiple second descriptor features, multiple key points in the overlapping area of ​​adjacent first cylindrical unfolded maps and second cylindrical unfolded maps are matched with each other to obtain multiple target key point pairs.

3. The method according to claim 2, characterized in that, Based on the matching degree between multiple first descriptor features and multiple second descriptor features, multiple key points in the overlapping area of ​​adjacent first and second cylindrical unfolded images are matched to obtain multiple target key point pairs, including: Based on the matching degree between multiple first descriptor features and multiple second descriptor features, target second descriptor features that match each first descriptor feature are determined from the multiple second descriptor features, and target first descriptor features that match each second descriptor feature are determined from the multiple first descriptor features. Based on multiple first descriptor features and their corresponding target second descriptor features, multiple first keypoint pairs are determined. Based on multiple second descriptor features and their corresponding target first descriptor features, multiple pairs of second key points are determined. The target keypoint pair is determined based on the same keypoint pairs among multiple first keypoint pairs and multiple second keypoint pairs.

4. The method according to claim 3, characterized in that, The target second descriptor feature includes the best-matching second descriptor feature and the second-best-matching second descriptor feature determined according to the matching degree; Based on multiple first descriptor features and their corresponding target second descriptor features, multiple pairs of first keypoints are determined, including: For any of the first descriptor features, a test ratio is determined based on the distance between the first descriptor feature and the best-matching second descriptor feature and the distance between the first descriptor feature and the second-best-matching second descriptor feature; If the test ratio is less than a predetermined test ratio threshold, the first key point pair is obtained based on the first descriptor feature and the best matching second descriptor feature.

5. The method according to claim 1, characterized in that, The method further includes: Key points in the overlapping area of ​​the first cylindrical unfolded diagram and the second cylindrical unfolded diagram are identified to determine multiple candidate key points for each of the first cylindrical unfolded diagram and the second cylindrical unfolded diagram. Based on the reflective features and texture reliability features of each of the multiple candidate key points, multiple candidate key points of each of the first cylindrical unfolded image and the second cylindrical unfolded image are screened to determine multiple key points of each of the first cylindrical unfolded image and the second cylindrical unfolded image. The reflective features are determined based on the specular reflectivity of the wetted surface of the rectum, and the texture reliability features are determined based on the texture complexity of the rectum.

6. The method according to claim 5, characterized in that, The method further includes: If the number of target keypoint pairs does not meet the first threshold, or the ratio of multiple keypoints to multiple candidate keypoints does not meet the second threshold, or the average error of the reprojection of multiple target keypoint pairs after adjustment by the correction parameters is greater than the third threshold, the correction parameters are adjusted to the basic correction parameters that characterize not performing correction.

7. The method according to claim 1, characterized in that, Based on the correction parameters, adjacent first cylindrical surface unfolded images and second cylindrical surface unfolded images are stitched together to obtain a target stitched image, including: Based on the correction parameters, the adjacent first cylindrical surface unfolded diagrams and second cylindrical surface unfolded diagrams are aligned in the overlapping area to obtain the aligned overlapping area; Based on the stitching constraint features of multiple pixels in the alignment and overlap region, a suture cost map of the alignment and overlap region is constructed. The suture cost map indicates the degree of unsuitability of any pixel in the alignment and overlap region as a stitching path. Based on the suture cost map, the target splicing suture is determined; Multi-band fusion processing is performed on the aligned and overlapping areas on both sides of the target stitching seam to obtain a panoramic image of the target.

8. The method according to claim 7, characterized in that, The pixel splicing constraint features include the gradient difference, color difference, reflective features, wrinkle structure features of the first cylindrical unfolded map and the second cylindrical unfolded map of the pixel, as well as the stability features of the pixel across multiple frames of images; The step of constructing a stitch cost map for the aligned and overlapping region based on the pixel splicing constraint features of the first and second cylindrical unfolded maps in the aligned and overlapping region includes: For any pixel in the alignment and overlap region, the gradient difference, color difference, reflective features, wrinkle structure features, and stability features of the pixel across multiple frames are weighted and summed to determine the stitch cost of the pixel. A suture cost map is constructed based on the suture cost of each of the multiple pixels.

9. The method according to claim 1, characterized in that, Based on pre-built mapping relationships, multiple images are mapped to cylindrical regions within the same cylindrical coordinate system, resulting in multiple cylindrical unfolded images, including: Based on the pre-built mapping relationship, the coordinate sets of multiple images are mapped to cylindrical regions of the same cylindrical coordinate system, thereby determining the cylindrical coordinate sets of multiple images. Based on the pre-calibrated array extrinsic parameters of each of the multiple image acquisition units and the field of view coverage angle of each of the multiple image acquisition units, the cylindrical coordinate sets of each of the multiple images are mapped to the panoramic canvas coordinate system to determine the overlapping area between the multiple cylindrical unfolded images.

10. An image processing system for panoramic rectal images, characterized in that, The system includes: A panoramic proctoscope includes multiple image acquisition units, each comprising multiple first image acquisition units arranged circumferentially and a second image acquisition unit located on top of the multiple first image acquisition units. The multiple first image acquisition units are used to acquire images of the inner wall of the rectum, and the second image acquisition unit is used to acquire images of the panoramic proctoscope in the forward direction. An electronic device, electrically connected to the panoramic proctoscope, is used to receive images acquired by the panoramic proctoscope and to perform the image processing method according to any one of claims 1 to 9.

Images