Vessel detection and modeling enhancement method based on higher order moment constraint and projection feedback

Abstract

The present application relates to the technical field of instrument image recognition, and particularly relates to a kind of based on high-order moment constraint and projection feedback's ware detection and modeling enhancement method. Including the following steps: receiving ware continuous image, the moving track and posture of camera in three dimensions are obtained, initialization dual quadratic surface, and projection obtains dual quadratic curve;According to ware continuous image, binary mask is obtained by mask extraction;According to binary mask and dual quadratic curve calculation, residual constraint is carried out, and the optimized dual quadratic surface is obtained;Dual quadratic surface is projected to plane again, and the optimized binary mask is obtained;The optimized binary mask is obtained, and the dual quadratic surface of bidirectional enhancement after is obtained, and the position detection result of ware is obtained according to dual quadratic surface.The present application is finely described to shape distribution by high-order moment, effectively overcome the depth dependence and rotation ambiguity problem, and the estimation accuracy of ellipsoid rotation angle and size parameter is significantly improved.

Description

Technical Field

[0001] This invention relates to the field of instrument image recognition technology, and in particular to a method for enhancing the detection and modeling of vessels based on higher-order moment constraints and projection feedback. Background Technology

[0002] In the context of chemical laboratory automation, 3D perception of transparent glassware such as beakers, test tubes, and flasks is a key challenge. Existing technologies typically employ a "detect first, reconstruct later" pipeline: first, deep learning (such as YOLO, Mask R-CNN) is used to detect objects on 2D images, and then depth maps or multi-view geometry are combined to reconstruct 3D bounding boxes.

[0003] Current mainstream visual 3D object detection and modeling methods are mainly divided into two categories: end-to-end methods and geometric constraint methods. End-to-end methods typically predict the projection center point and eight corner points of a 3D cuboid model onto a 2D image, and combine this with depth information provided by a depth sensor or multi-frame consistency constraints to jointly estimate the 6D pose and size of the object. However, transparent glassware, due to its highly transparent surface and lack of texture, easily blends with the background color, leading to unstable detection results or even complete loss of detection. Simultaneously, depth sensors (such as RGB-D cameras) often experience measurement failures when dealing with transparent materials—light penetrates the container walls, capturing the depth of the bottom or background rather than the surface of the container itself, making the observation data unreliable and difficult to maintain a stable 3D model continuously, sometimes even leading to modeling failure. On the other hand, geometric constraint methods typically first use a deep learning model to obtain a 2D bounding box and segmentation mask, and then fit a general geometric model (such as a cuboid or ellipsoid) based on multi-view geometric relationships. However, such methods also face fundamental challenges: the smooth, textureless surface of transparent vessels makes it difficult for traditional feature point extraction algorithms to obtain stable and reliable key points; furthermore, the optimization process of existing models relies excessively on 2D bounding box constraints, resulting in significant rotational ambiguity and difficulty in accurately recovering the true orientation of the vessel. In summary, due to the combined difficulties of error-prone detection, unreliable depth information, missing feature points, and ambiguity in pose estimation, existing technologies struggle to achieve robust, accurate, and continuous 3D parametric modeling of transparent glass vessels, severely limiting the reliability and success rate of downstream tasks such as robotic arm grasping. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides an enhanced method for vessel detection and modeling based on higher-order moment constraints and projection feedback, which overcomes the problems of depth dependence and rotation ambiguity, and significantly improves the estimation accuracy of the rotation angle and size parameters of the ellipsoid.

[0005] This invention provides a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback, comprising: S1: Receiver receives continuous images, acquires the camera's movement trajectory and attitude in three dimensions, and obtains the dual quadratic curve; S2: Obtain a binary mask by performing mask extraction based on the continuous image of the vessel; S3: Calculate residual constraints based on the binary mask and dual quadratic curve to obtain the optimized dual quadratic surface; S4: Reproject the dual quadratic surface onto the plane to obtain the optimized binary mask; S5: The optimized binary mask is subjected to residual constraints based on the binary mask and the dual quadratic curve to obtain the bidirectional enhanced dual quadratic surface, and the position detection result of the vessel is obtained based on the dual quadratic surface.

[0006] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided. The calculation formula for step S1 is as follows: in, For dual conic sections, For dual quadric surfaces, For camera projection matrix, This is the matrix transpose.

[0007] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein step S3 includes: S31: Convert the dual quadratic curve into binary mask coordinates. S32: Calculate the observation moments of the binary mask based on the binary mask coordinates, where the observation moments are the first to fourth order geometric moments of the binary mask; S33: Calculate the analytical moments based on the dual quadratic surface, where the analytical moments are the theoretical first to fourth order geometric moments of the dual quadratic surface; S34: Construct higher-order moment constraints based on the analytical moments and observed moments; S35: Construct bounding box constraints based on the dual quadratic surface and binary mask coordinates; S36: Optimize the dual quadratic curves based on the higher-order moment constraints and boundary box constraints to obtain the optimized dual quadratic surface.

[0008] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein step S32 includes: S321: Calculate the first-order geometric moment of the observed moment: in, The first geometric moment of the observation moment. The second first-order geometric moment of the observation moment. Let x be the x-axis of the mask. The vertical axis of the mask is... It is a binary mask; S322: Calculate the second-order geometric moments of the observed moments: in, The first and second order geometric moments of the observation moments. The second geometric moment of the observation moment. The third and second order geometric moments of the observation moments; S323: Calculate the third geometric moment of the observed moment: in, The first and third geometric moments of the observation moments. The second and third geometric moments of the observation moments. The third geometric moment of the observation moment. The fourth and third geometric moments of the observation moments; S324: Calculate the fourth geometric moment of the observed moment: in, The first and fourth geometric moments of the observation moments. The second and fourth geometric moments of the observation moments. The third and fourth geometric moments of the observation moments, The fourth geometric moment of the observation moment. It is the fifth and fourth geometric moments of the observation moments.

[0009] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein step S33 includes: S331: Calculate the first-order geometric moment of the analytic moment: in, The first geometric moment of the analytic moment. Let x be the x-coordinate of the center of the dual conic section. The moment at the origin, The second first-order geometric moment of the analytic moment. The ordinate of the center of the dual conic section; S332: Calculate the second-order geometric moments of the analytic moments: in, These are the first and second order geometric moments of the analytic moments. The second second-order geometric moment is the analytic moment. The third and second order geometric moments of the analytic moments. Let be the semi-major axis length of the dual conic section. Let be the semi-minor axis length of the dual conic section. The rotation angle of the dual conic section; S333: Calculate the third geometric moment of the analytic moment: in, These are the first and third geometric moments of the analytic moments. These are the second and third order geometric moments of the analytic moments. The third and third geometric moments are the analytic moments. The fourth and third geometric moments are the analytic moments; S334: Calculate the fourth geometric moment of the analytic moment: in, The first and fourth geometric moments are the analytic moments. The second and fourth geometric moments are the analytic moments. These are the third and fourth order geometric moments of the analytic moments. The fourth and fourth geometric moments are the analytic moments. It is the fifth and fourth order geometric moment of the analytic moment.

[0010] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein the higher-order moment constraints in step S34 are... The calculation formula is: in, Let n be the order of the moment. For the first Weighting coefficients of the step moment, For the first The geometric moment of the observed moment of the order moment. For the first The analytic moment and the geometric moment of the first moment. Represents norm operations.

[0011] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein the bounding box constraint in step S35 is... The calculation formula is: in, The observation bounding box extracted by the mask. Based on ellipsoid parameters With camera pose The calculated theoretical projected boundary, Let covariance matrix be the variance matrix. For projection function, ellipsoid parameters The corresponding dual quadratic surface, Represents norm operations, For camera projection matrix, This is the matrix transpose.

[0012] According to the present invention, a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback is provided, wherein step S4 is as follows: S41: Project the optimized dual quadratic surface according to the current camera projection matrix to obtain the predicted projection region. : in, For the optimized dual quadratic surface, For the camera projection matrix at that time, This is the matrix transpose. S42: Input the predicted projection region into the projection function to obtain the optimized binary mask.

[0013] This invention provides a method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback, wherein the optimization function of the method is... for: in, The reprojection error constraint is used to ensure the consistency between the camera pose and the map point position. It is a higher-order moment constraint used to constrain the geometric consistency between the 3D ellipsoid model and the observation semantic mask; This is a bounding box constraint used to ensure consistency between the bounding rectangle of the 3D ellipsoid's projection onto the image and the 2D object detection box.

[0014] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: This invention provides an enhanced method for vessel detection and modeling based on higher-order moment constraints and projection feedback. It initially obtains the 2D bounding box and segmentation mask of the vessel using a target detection network, and then extracts first- to fourth-order original image moments from the segmentation mask to comprehensively characterize its geometric distribution characteristics on the image plane, including position, scale, orientation, skewness, and kurtosis. Simultaneously, it combines the back-projection planes corresponding to the four sides of the bounding box to jointly optimize the parameters of the 3D ellipsoid model represented by the dual quadratic surface. Unlike traditional methods that rely solely on loose bounding box constraints, this invention effectively overcomes the problems of depth dependence and rotation ambiguity through a detailed description of the shape distribution using higher-order moments, significantly improving the estimation accuracy of the ellipsoid's rotation angle and size parameters.

[0015] Building upon this foundation, a closed-loop enhancement mechanism of "detection-modeling-feedback-correction" is constructed: an optimized and stable 3D ellipsoidal model is forward-projected onto the image plane based on the current frame's camera pose, generating geometrically guided prior cues. These cues are then input into the Segment Anything Model (SAM) for secondary refined segmentation and used for detection guidance in the next frame. This mechanism achieves a two-way mutual benefit between the 3D model and 2D perception—the 3D model corrects noise and missed detections in 2D segmentation, improving detection recall and localization accuracy; higher-quality 2D segmentation results, in turn, provide more accurate observation constraints for the 3D model, driving continuous model optimization. Furthermore, the stable 3D ellipsoidal model can also assist in camera pose estimation, ensuring the consistency of the vessel's position, size, and orientation across multiple frames in the video sequence, thus providing a reliable and continuous 3D environmental perception foundation for robot operation.

[0016] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the enhanced vessel detection and modeling method based on higher-order moment constraints and projection feedback provided by the present invention.

[0019] Figure 2 It is the recognition result image after the image of the vessel has been processed.

[0020] Figure 3 This is a screenshot showing the results of vessel detection using the QuadricSLAM algorithm.

[0021] Figure 4 This is a diagram showing the detection effect of the vessel using the algorithm of this invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0023] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0024] The following is combined Figures 1 to 3 This invention is described.

[0025] Example like Figure 1 As shown, Figure 1 The flowchart of the vessel detection and modeling enhancement method based on higher-order moment constraints and projection feedback provided by the present invention includes the following steps: S1: Receive continuous images of the vessel to obtain the camera's movement trajectory and orientation in 3D, and obtain the dual quadratic curve. The continuous images of the vessel are a set of unprocessed images taken in the real world.

[0026] S2: Obtain a binary mask by performing mask extraction based on the continuous image of the vessel.

[0027] S3: Calculate residual constraints based on the binary mask and dual quadratic curve to obtain the optimized dual quadratic surface.

[0028] S4: Reproject the dual quadratic surface onto the plane to obtain the optimized binary mask; S5: The optimized binary mask is subjected to residual constraints based on the binary mask and the dual quadratic curve to obtain the bidirectional enhanced dual quadratic surface, and the position detection result of the vessel is obtained based on the dual quadratic surface.

[0029] Specifically, to closely fit rotationally symmetrical chemical vessels such as beakers and flasks, this invention uses an ellipsoid as its three-dimensional representation model and employs the "dual quadratic surface" form for parameterization. The advantage of this representation method is that when the ellipsoid in three-dimensional space is projected onto a two-dimensional image plane through a camera model, the projection result forms a quadratic curve under perspective projection conditions. Compared to directly using explicit ellipsoidal parameters or point cloud models, the dual quadratic surface form can directly establish an analytical mapping relationship between the three-dimensional model parameters and the two-dimensional projected quadratic curve without relying on the object's surface point cloud or depth information. This avoids modeling failures caused by missing depth or unstable edges in transparent glass vessels. The calculation formula for step S1 is as follows: in, This is a dual conic section, also known as a 2D bounding box. For dual quadric surfaces, For camera projection matrix, This is the matrix transpose.

[0030] Specifically, in step S2, this embodiment of the invention uses SAM (Segment Anything Model) to extract a binary mask based on the continuous image of the vessel.

[0031] To address the pose ambiguity caused by using only 2D bounding box and projected ellipse tangency constraints in traditional methods, this invention introduces both higher-order moment constraints and bounding box constraints to fully utilize the complete shape information of the segmentation mask.

[0032] Specifically, step S3 includes: S31: Convert the dual quadratic curve into binary mask coordinates. S32: Calculate the observation moments of the binary mask based on the binary mask coordinates, where the observation moments are the first to fourth order geometric moments of the binary mask; S33: Calculate the analytical moments based on the dual quadratic surface, where the analytical moments are the theoretical first to fourth order geometric moments of the dual quadratic surface; S34: Construct higher-order moment constraints based on the analytical moments and observed moments; S35: Construct bounding box constraints based on the dual quadratic surface and binary mask coordinates; S36: Optimize the dual quadratic curves based on the higher-order moment constraints and boundary box constraints to obtain the optimized dual quadratic surface.

[0033] Step S32 includes: S321: Calculate the first-order geometric moment of the observed moment: in, The first geometric moment of the observation moment. The second first-order geometric moment of the observation moment. Let x be the x-axis of the mask. The vertical axis of the mask is... This is a binary mask. The first-order geometric moments are the centroid positions, used to constrain the object's position in the image.

[0034] S322: Calculate the second-order geometric moments of the observed moments: in, The first and second order geometric moments of the observation moments. The second geometric moment of the observation moment. This is the third second-order geometric moment of the observation moments. The second-order geometric moments are used to describe the shape covariance and the ratio of the major and minor axes, constraining the scale and orientation of the ellipsoid.

[0035] S323: Calculate the third geometric moment of the observed moment: in, The first and third geometric moments of the observation moments. The second and third geometric moments of the observation moments. The third geometric moment of the observation moment. This is the fourth and third geometric moments of the observation moments. The third geometric moments capture the asymmetry (skewness) of the projected shape and can be used to help constrain the spatial attitude parameters of the ellipsoid.

[0036] S324: Calculate the fourth geometric moment of the observed moment: in, The first and fourth geometric moments of the observation moments. The second and fourth geometric moments of the observation moments. The third and fourth geometric moments of the observation moments, The fourth geometric moment of the observation moment. This is the fifth and fourth geometric moments of the observation moments. The fourth geometric moments are used to describe the steepness (kurtosis) of the shape distribution, which helps to distinguish the shape characteristics of the projection under different rotational attitudes.

[0037] Step S33 includes: S331: Calculate the first-order geometric moment of the analytic moment: in, The first geometric moment of the analytic moment. Let x be the x-coordinate of the center of the dual conic section. The moment at the origin, The second first-order geometric moment of the analytic moment. The ordinate of the center of the dual conic section; S332: Calculate the second-order geometric moments of the analytic moments: in, These are the first and second order geometric moments of the analytic moments. The second second-order geometric moment is the analytic moment. The third and second order geometric moments of the analytic moments. Let be the semi-major axis length of the dual conic section. Let be the semi-minor axis length of the dual conic section. The rotation angle of the dual conic section; S333: Calculate the third geometric moment of the analytic moment: in, These are the first and third geometric moments of the analytic moments. These are the second and third order geometric moments of the analytic moments. The third and third geometric moments are the analytic moments. The fourth and third geometric moments are the analytic moments; S334: Calculate the fourth geometric moment of the analytic moment: in, The first and fourth geometric moments are the analytic moments. The second and fourth geometric moments are the analytic moments. These are the third and fourth order geometric moments of the analytic moments. The fourth and fourth geometric moments are the analytic moments. It is the fifth and fourth order geometric moment of the analytic moment.

[0038] Specifically, the higher-order moment constraints in step S34 The calculation formula is: in, Let n be the order of the moment. For the first Weighting coefficients of the step moment, For the first The geometric moment of the observed moment of the order moment. For the first The geometric moments of the analytic moments. This residual term measures the difference in higher-order statistical characteristics between the observed shape and the theoretical shape of the ellipsoidal projection. Represents norm operations.

[0039] Specifically, in step S35, the bounding box constraint The calculation formula is: in, The observation bounding box extracted by the mask. Based on ellipsoid parameters With camera pose The calculated theoretical projected boundary, Let covariance matrix be the variance matrix. For projection function, ellipsoid parameters The corresponding dual quadric surface.

[0040] Specifically, the steps of the projection function are as follows: First, find the maximum and minimum extreme points of the four hyperbolas, without considering the image boundaries; Find all intersection points of the hyperbola with the image boundary, up to a maximum of 8; Remove false points and all points outside the boundary; The maximum and minimum coordinates among the remaining points are used as the final coordinates. Function result.

[0041] Specifically, step S4 aims to address the issue of unstable segmentation in SAM under complex scenarios, and the steps are as follows: S41: Project the optimized dual quadratic surface according to the current camera projection matrix to obtain the predicted projection region. : in, For the optimized dual quadratic surface, For the camera projection matrix at that time, This is the matrix transpose. S42: Input the predicted projection region into the projection function to obtain the optimized binary mask. In this embodiment of the invention, based on the method for determining the projection region boundary mentioned in step S35, the minimum bounding rectangle of the projection region in the image plane is calculated. The smallest bounding rectangle is then used as the Box Prompt of the SAM. The Prompt generated by the above geometry is then input into the SAM. Since this Prompt originates from a geometrically consistent 3D model across multiple frames, it forces the SAM to focus on the correct object region, suppressing oversegmentation (mask overflow) caused by reflections or undersegmentation caused by transparency.

[0042] Specifically, the optimization function of the vessel detection and modeling enhancement method. for: in, The reprojection error constraint is used to ensure the consistency between the camera pose and the map point position. It is a higher-order moment constraint used to constrain the geometric consistency between the 3D ellipsoid model and the observation semantic mask; This is a bounding box constraint used to ensure consistency between the bounding rectangle of the 3D ellipsoid's projection onto the image and the 2D object detection box.

[0043] Specifically, reprojection error constraints Used to constrain the consistency between camera pose and map point positions. Let... Let m be the 3D coordinates of the m-th map point. Let n be the pose (transformation matrix) of the nth keyframe. Let be the observed pixel coordinates of the map point on the image, then: in, For camera projection function, This is a robust kernel function.

[0044] Higher-order image moment constraint terms Used to constrain the geometric consistency between the 3D ellipsoid model and the observation semantic mask: in, Let n be the order of the moment. For the first Weighting coefficients of the step moment, For the first The geometric moment of the observed moment of the order moment. For the first The geometric moments of the analytic moments. This residual term measures the difference in higher-order statistical characteristics between the observed shape and the theoretical shape of the ellipsoidal projection. Represents norm operations.

[0045] Bounding box constraints Used to constrain the consistency between the bounding rectangle of the 3D ellipsoid's projection onto the image and the 2D object detection box. in, The observation bounding box extracted by the mask. Based on ellipsoid parameters With camera pose The calculated theoretical projected boundary, Let covariance matrix be the variance matrix. For projection function, ellipsoid parameters The corresponding dual quadratic surface, Represents norm operations, For camera projection matrix, This is the matrix transpose.

[0046] This invention overcomes the theoretical bottleneck of the traditional "boundary box back projection plane tangency" method, which cannot recover the accurate rotation attitude by utilizing higher-order moment constraints. This allows the recovered chemical vessel model to have the correct orientation in 3D space. Compared with the state-of-the-art algorithm QuadricSLAM, the average angle is reduced by 33.21°, and the overall orientation estimation error is reduced by about 88%, meeting the requirements for precise grasping.

[0047] Table 1 Comparison of Orientation Estimation for Chemical Glassware

[0048] As shown in Table 1, the solution provided by the present invention significantly reduces the spindle angle error and improves the direction estimation accuracy by an average of 36.9%, meeting the requirements for precision gripping.

[0049] This invention utilizes a "3D projection-SAM cue" feedback mechanism to correct large errors in single-frame image segmentation by leveraging geometric consistency in the time domain. Even if a frame is missed, the projection of the 3D model can still serve as a cue word for SAM, resulting in the correct segmentation and improving recall to 100%.

[0050] Specifically, such as Figure 2 , Figure 3 and Figure 4 As shown, where, Figure 2 This is the recognition result image after processing the image of the vessel. Figure 2 The thick rectangular frame outside the middle is the text label annotation box, which indicates the recognition result and the confidence level of the recognition. Figure 2 The thin box inside the image represents the bounding box of the recognition result. The jagged edges that appear are due to the small size of the target object in the image and the insufficient resolution of the segmentation network model.

[0051] Figure 3 This is a screenshot showing the results of vessel detection using the QuadricSLAM algorithm. Figure 4 This is an image showing the vessel detection results using the algorithm of this invention. Red dots represent currently active map points, primarily used for tracking and localization in the current frame. Black dots represent ordinary map points, used to construct the background map. Both red and black dots are background points. Blue lines and blue rectangles represent the poses of historical keyframes. Green lines and green rectangles represent the pose of the current frame. The three different colored ellipsoids represent different objects of interest.

[0052] This invention ensures the stability of the size and shape of the object model in the video sequence through multi-frame consistency, preventing the model from fluctuating in size or shaking violently.

[0053] The algorithm of this invention relies on geometrically constrained self-supervised optimization, and can operate with only a 2D segmentation model, thus solving the problem of the extreme difficulty in annotating 3D data of transparent objects.

[0054] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0055] While this disclosure has been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the specific embodiments disclosed. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback, characterized in that, Includes the following steps: S1: Receiver receives continuous images, acquires the camera's movement trajectory and attitude in three dimensions, and obtains the dual quadratic curve; S2: Obtain a binary mask by performing mask extraction based on the continuous image of the vessel; S3: Calculate residual constraints based on the binary mask and the dual quadratic curve to obtain the optimized dual quadratic surface; Step S3 includes: S31: Convert the dual quadratic curve into binary mask coordinates. S32: Calculate the observation moments of the binary mask based on the binary mask coordinates, where the observation moments are the first to fourth order geometric moments of the binary mask; S33: Calculate the analytical moments based on the dual quadratic surface, where the analytical moments are the theoretical first to fourth order geometric moments of the dual quadratic surface; S34: Construct higher-order moment constraints based on the analytical moments and observed moments; S35: Construct bounding box constraints based on the dual quadratic surface and binary mask coordinates; S36: Based on the higher-order moment constraints and bounding box constraints, optimize the dual quadratic curve to obtain the optimized dual quadratic surface; S4: Reproject the dual quadratic surface onto the plane to obtain the optimized binary mask; S5: The optimized binary mask is subjected to residual constraints based on the binary mask and the dual quadratic curve to obtain the bidirectional enhanced dual quadratic surface, and the position detection result of the vessel is obtained based on the dual quadratic surface.

2. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, The calculation formula for step S1 is: in, For dual conic sections, For dual quadric surfaces, For camera projection matrix, This is the matrix transpose.

3. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Step S32 includes: S321: Calculate the first-order geometric moment of the observed moment: in, The first geometric moment of the observation moment. The second first-order geometric moment of the observation moment. Let x be the x-axis of the mask. The vertical coordinate of the mask is... It is a binary mask; S322: Calculate the second-order geometric moments of the observed moments: in, The first and second order geometric moments of the observation moments. The second geometric moment of the observation moment. The third and second order geometric moments of the observation moments; S323: Calculate the third geometric moment of the observed moment: in, The first and third geometric moments of the observation moments. The second and third geometric moments of the observation moments. The third geometric moment of the observation moment. The fourth and third geometric moments of the observation moments; S324: Calculate the fourth geometric moment of the observed moment: in, The first and fourth geometric moments of the observation moments. The second and fourth geometric moments of the observation moments. The third and fourth geometric moments of the observation moments, The fourth geometric moment of the observation moment. It is the fifth and fourth geometric moments of the observation moments.

4. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Step S33 includes: S331: Calculate the first-order geometric moment of the analytic moment: in, The first geometric moment of the analytic moment. Let x be the x-coordinate of the center of the dual conic section. The moment at the origin, The second first-order geometric moment is the analytic moment. The ordinate of the center of the dual conic section; S332: Calculate the second-order geometric moments of the analytic moments: in, These are the first and second order geometric moments of the analytic moments. The second second-order geometric moment is the analytic moment. The third and second order geometric moments of the analytic moments. The semi-major axis length of the dual conic section. Let be the semi-minor axis length of the dual conic section. The rotation angle of the dual conic section; S333: Calculate the third geometric moment of the analytic moment: in, These are the first and third geometric moments of the analytic moments. These are the second and third order geometric moments of the analytic moments. The third and third geometric moments are the analytic moments. The fourth and third geometric moments are the analytic moments; S334: Calculate the fourth geometric moment of the analytic moment: in, The first and fourth geometric moments are the analytic moments. The second and fourth geometric moments are the analytic moments. These are the third and fourth order geometric moments of the analytic moments. The fourth and fourth geometric moments of the analytic moments. It is the fifth and fourth order geometric moment of the analytic moment.

5. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Higher-order moment constraints in step S34 The calculation formula is: in, Let n be the order of the moment. For the first Weighting coefficients of the step moment, For the first The geometric moment of the observed moment of the order moment. For the first The analytic moment and the geometric moment of the first moment. Represents norm operations.

6. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Boundary box constraints in step S35 The calculation formula is: in, The observation bounding box extracted by the mask. Based on ellipsoid parameters With camera pose The calculated theoretical projected boundary, Let covariance matrix be the variance matrix. For projection function, ellipsoid parameters The corresponding dual quadratic surface, Represents norm operations, For camera projection matrix, This is the matrix transpose.

7. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Step S4 is: S41: Project the optimized dual quadratic surface according to the current camera projection matrix to obtain the predicted projection region. : in, The optimized dual quadratic surface, For the camera projection matrix at that time, This is the matrix transpose. S42: Input the predicted projection region into the projection function to obtain the optimized binary mask.

8. The method for enhancing vessel detection and modeling based on higher-order moment constraints and projection feedback according to claim 1, characterized in that, Optimization function of vessel detection and modeling enhancement method for: in, The reprojection error constraint is used to ensure the consistency between the camera pose and the map point position. It is a higher-order moment constraint used to constrain the geometric consistency between the 3D ellipsoid model and the observation semantic mask; This is a bounding box constraint used to ensure consistency between the bounding rectangle of the 3D ellipsoid's projection onto the image and the 2D object detection box.

Images