A guidance assistance device and a control system thereof

Abstract

The application discloses a guiding auxiliary device and a control system thereof, and belongs to the technical field of feature registration. A target object surface model is constructed through multi-model access and three-dimensional point cloud reconstruction. The registration problem caused by the difference between global and local scales is solved by using curvature feature matching and potential area screening. A closed operation area is formed through edge point mapping, and precise spatial alignment of the design model and the actual scene is realized by combining centroid deviation adjustment. Key points of equipment positioning are planned based on a rigid transformation matrix, ensuring geometric alignment of the operation end and the functional feature points, and a collision-free path is generated through safety verification. With the aid of light source visualization guidance and real-time deviation monitoring, dynamic feedback of the operation process is realized. After operation, effect evaluation is performed through comparison of the geometric features of the functional points, and global reset is avoided through local correction. The device operation precision and safety in the precise medical implantation scene are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of feature registration, and more particularly to a guiding auxiliary device and a control system thereof. BACKGROUND

[0002] In the field of precision manufacturing, high-end equipment installation and complex component positioning, etc., which requires high precision for object surface processing and component adaptation, the existing technical means face a series of problems to be solved. The traditional method relies on manual repeated measurement or offline scanning according to the mold, which is difficult to obtain complete three-dimensional information of the object surface in real time. For processing scenes with complex curved surfaces or dynamic changes, it is difficult to accurately capture surface features, resulting in that the processing precision is greatly affected by human factors, and it is difficult to meet the high-precision adaptation requirements.

[0003] At the same time, the processing path planning is often based on fixed models or standard parameters, which deviates from the actual object surface geometry. The operator needs to frequently intervene manually to compare the adaptation degree in the process. In the complex surface processing of multi-curvature and multi-feature, this problem leads to low process efficiency and significantly increased correction cost. In addition, in the processing flow involving multi-device cooperation, lighting, mapping, and execution are usually run independently, lacking real-time linkage mechanism. When the object surface material and morphology change, it is difficult to automatically adjust the processing strategy. More importantly, the geometric parameters of different target components differ greatly. For example, when the target component is a cochlear implant, different cochlear molds have size and shape differences. The traditional method needs to design a special cochlear mold for each type of cochlea, which lacks universality. When adapting to new components, the preparation period is longer, and the monitoring means for sensitive areas on the object surface during the processing process is limited, and the control of the safety boundary depends on manual judgment, which is easy to cause misjudgment due to limited operation visual angle, fatigue, etc., causing irreversible damage.

[0004] The above problems make it difficult to achieve the ideal overall qualification rate of precision component installation. With the continuous growth of personalized customization demand, higher requirements are put forward for the universality, real-time performance and intelligent level of the processing device. Therefore, in order to overcome these limitations, the present application proposes a guiding auxiliary device and a control system thereof. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a guiding auxiliary device and a control system thereof, which acquires multi-dimensional data of a target object surface by accessing multiple models, extracts key feature points of a design model, and screens potential areas of the target object surface based on curvature matching, establishes an edge point mapping relationship between the design model and an actual operation area to form a closed contour, and realizes spatial alignment in combination with centroid deviation adjustment; ensures geometric alignment by a rigid transformation matrix when planning device positioning key points, and generates a collision-free path in combination with safety verification; acquires data in real time during operation to perform deviation and collision warning, and realizes matching degree determination and local correction by comparing geometric features of functional feature points after operation, so as to effectively handle global and local scale differences, solve the problem of inaccurate positioning caused by occlusion, and realize high-precision device operation guidance and closed-loop control in the scene of precise medical implantation.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] A guiding auxiliary device control system, comprising:

[0008] Accessing a target component model, a design model of a target installation area, and a target operation device model, and acquiring multi-dimensional data of a target object surface to construct a target object surface model;

[0009] Extracting key feature points in the design model, dividing candidate areas of the target object surface model based on the size of the design model bounding box, identifying target candidate areas and extracting curvature feature points, screening effective curvature feature points by configuring a matching threshold, dividing an extended verification range, and selecting a potential area by counting the matching density in the extended verification range;

[0010] Mapping the edge points of the design model to the potential area of the target object surface model based on the curvature feature similarity degree and the spatial distance constraint to form a closed contour as an actual operation area, performing spatial alignment of the target object surface model and the design model according to the deviation of the geometric centroid of the actual operation area and the global centroid of the target object surface model, and determining whether to trigger intelligent planning operation;

[0011] When the intelligent planning operation is triggered, generating guiding instructions by planning the positioning key point position of the target operation device on the target object surface model based on the position of the functional feature points of the design model mapped to the actual operation area in combination with the positioning key points and the geometric shape of the target operation device model;

[0012] Providing target operation device operation guidance by a light source emitting assembly, acquiring positioning key point data in real time to perform deviation and collision warning, and determining whether the target component model and the actual operation area match by the geometric features of the local area where the functional feature points are located after operation, and generating a local correction instruction if they do not match.

[0013] Specifically, the step of spatially aligning the target object surface model with the design model comprises:

[0014] Performing coordinate normalization on the target installation area design model, constructing an axis-aligned bounding box of the design model and marking the geometric center of the design model, determining the principal inertia axis of the design model by principal component analysis with the geometric center as the origin, and constructing a local coordinate system;

[0015] Extracting key feature points in the design model, the key feature points including functional feature points, edge feature points and curvature feature points;

[0016] Based on the bounding box size of the design model, by grid division of the target object surface model, curvature feature point matching, identification of effective curvature feature points and division of an extended verification range, a potential region containing the design model is selected in the target object surface model;

[0017] In the potential region, based on curvature feature similarity and spatial distance constraint, edge points of the design model are searched for mapping based on the matched effective curvature feature points as the reference, to generate an actual operation region, and the functional feature points of the design model are mapped into the actual operation region of the potential region.

[0018] Specifically, the step of spatially aligning the target object surface model with the design model further comprises:

[0019] Configuring a deviation threshold, calculating the geometric centroid of the actual operation region as the target installation center, and calculating the global centroid of the target object surface model as the guide auxiliary device center;

[0020] Calculating the spatial position deviation of the target installation center and the guide auxiliary device center, if the spatial position deviation is greater than the deviation threshold, triggering the device position adjustment mechanism; otherwise, triggering the intelligent planning operation;

[0021] The device position adjustment mechanism comprises:

[0022] According to the spatial position deviation, the adjustment direction and movement distance of the guide auxiliary device are calculated, the light source emission assembly is used to indicate the movement direction and movement distance of the guide auxiliary device, so that the guide auxiliary device center moves to the target installation center;

[0023] After the guide auxiliary device moves, the target object surface model is reacquired, and the actual operation region and the key feature points are repositioned based on the adjustment direction and the movement distance.

[0024] Specifically, the step of selecting a potential region containing the design model in the target object surface model comprises:

[0025] Grid division is performed on the target object surface model to set an initial search range based on the size of the model bounding box, and the target object surface model is divided into a plurality of candidate regions;

[0026] The candidate regions are traversed, the surface point curvature values are calculated through a local surface fitting algorithm, and the target candidate region is identified;

[0027] The curvature feature points of the target candidate region are extracted, the similarity of the curvature feature points is calculated, and the curvature feature points of the design model are matched to identify effective curvature feature points;

[0028] An extension verification range is demarcated with the effective curvature feature points as the center and the size of the design model bounding box as the radius, all curvature feature points in the extension verification range are extracted and screened for effective curvature feature points, the number of effective curvature feature points is counted to form a matching density, and the target candidate region is selected as a potential region according to the matching density of the target candidate region.

[0029] Specifically, the step of generating the actual operation region includes:

[0030] A similarity threshold and a spatial search threshold are set, the edge point set of the design model and its local geometric feature are obtained, the point cloud data of the potential region and its three-dimensional coordinates and local geometric feature are obtained;

[0031] For each edge point of the design model, the target region point located within the spatial search threshold range is screened out with the edge point as the center in the potential region;

[0032] The similarity of the local geometric feature of the target region point and the edge point of the design model is calculated, and if it is greater than the similarity threshold, the candidate edge point is marked, and the mapping edge point of the design model edge point is selected according to the similarity of the candidate edge point;

[0033] The design model edge point and the screened potential region mapping edge point are established in a corresponding relationship, the mapping edge points are connected in the order of the edge of the design model to form a closed actual operation region contour.

[0034] Specifically, the step of planning the positioning key point position of the target operation device on the target object surface model includes:

[0035] The coordinates of the functional feature points in the actual operation region are received, the target operation device model is accessed, and its three-dimensional geometric shape parameters and positioning key points are obtained;

[0036] The functional feature points, device positioning key points, and target object surface model are uniformly mapped to the same spatial coordinate system;

[0037] establish a rigid spatial mapping relationship between the functional feature points and the device positioning key points, define a fixed offset and a posture constraint of the device positioning key points relative to the functional feature points based on a translation vector and a rotation matrix, form a geometric alignment benchmark model, and determine the operation posture of the target operation device;

[0038] Specifically, the step of planning the position of the positioning key point of the target operation device on the target object surface model further comprises:

[0039] Based on the three-dimensional coordinates of the functional feature points, the operation posture and the rigid spatial mapping relationship, the target position of the device positioning key point in the coordinate system of the target object surface model is calculated through a rigid transformation matrix, so that the device operation end is aligned with the spatial position and posture of the functional feature points;

[0040] In combination with the geometric features of the target object surface model, the safety of the target position corresponding to the positioning key point is verified, and whether the target position is qualified is judged;

[0041] When the target position is qualified, the operation sequence and the collision-free motion path of the functional feature points are planned by using a path planning algorithm according to the operation priority of the functional feature points, the kinematics constraint of the device and the geometric features of the target object surface.

[0042] Specifically, the step of collecting the positioning key point data for deviation and collision warning comprises:

[0043] The target position corresponding to the positioning key point, the operation sequence and the collision-free motion path are received, and the target position is converted into a motion instruction executable by the target operation device;

[0044] Based on the motion instruction, the operation position, the moving direction and the operation progress of the target operation device are indicated by the light source emitting component;

[0045] The target object surface model is reconstructed in real time by the light source receiving component, and the actual position and posture data of the positioning key point are collected in real time by the built-in sensor of the target operation device;

[0046] A precision threshold is configured, the spatial deviation between the actual position and the target position is calculated and compared with the precision threshold, if the spatial deviation is greater than the precision threshold, a deviation warning is performed by the light source emitting component, otherwise no operation is performed;

[0047] A collision threshold is configured, based on the bounding box of the target operation device model and the target object surface model, the distance between the high curvature region of the target operation device model and the target object surface model and the boundary of the anatomical structure is calculated in real time by the bounding box hierarchy tree algorithm, if the distance is less than the collision threshold, a collision warning is performed by the light source emitting component, otherwise no operation is performed.

[0048] Specifically, the step of judging whether the target component model matches the actual operation area comprises:

[0049] A distance deviation threshold is configured, after the target operating device completes the operation, the local region data where the functional feature points of the actual operation region are divided is extracted, the geometric features are calculated, the distance deviation of the geometric features of the local region where the functional feature points corresponding to the target component model are located is calculated, if it is greater than the distance deviation threshold, it is determined that it is not matched, otherwise it is determined that it is matched;

[0050] If it is determined that it is not matched, the functional feature points causing the mismatch are identified, the operation path is re-planned, and local correction instructions are generated.

[0051] A guiding auxiliary device, comprising: a support structure, a light source emission receiving structure, a data transmission structure and an external processing structure;

[0052] The support structure serves as the physical basis of the guiding auxiliary device, and defines the collection range of the target object surface model through adjustable geometric shapes and degrees of freedom;

[0053] The light source emission receiving structure is used to realize the construction of the target object surface model, synchronously acquire the multi-dimensional features of the target object surface model, and provide visual operation guidance, indicating the target position corresponding to the positioning key point of the target operating device, deviation warning and operation progress;

[0054] The data transmission structure is used to realize the mechanical fixation of the guiding auxiliary device, the calibration of the spatial coordinate system and the data interaction;

[0055] The external processing structure is used to process three-dimensional point cloud data in real time, and execute dynamic registration, intelligent planning and safety control.

[0056] The beneficial effects of the present application are:

[0057] The present application realizes accurate registration of cross-scale models by accessing multiple models and collecting multi-dimensional data of the target object surface, and uses curvature feature matching and potential region screening to realize accurate registration of cross-scale models, solves the registration problem caused by the difference between global and local scales; based on the rigid spatial mapping relationship, the positioning key points of the device are planned, the operation end and the geometric alignment of the functional feature points are ensured in the occlusion scene, and the limitation of relying on direct visibility in visual guidance is broken through; through real-time deviation and collision warning, comparison of geometric features of functional feature points after operation and local correction, dynamic closed-loop control is formed, and the operation precision and safety are significantly improved; support complex curved surface environment and medical implantation, effectively exclude high curvature risk area, realize efficient adaptation, provide high-precision, high-robustness solution for precision operation. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The structure diagram of the guiding auxiliary device of the present application is shown in the figure;

[0059] Figure 2A schematic diagram of the structural principle of a guiding auxiliary device control system according to the present application;

[0060] Figure 3 A flowchart of the present application for spatial alignment of the target object surface model and the design model;

[0061] Figure 4 A flowchart of the present application for selecting a potential region containing the design model in the target object surface model;

[0062] Figure 5 A flowchart of the present application for generating an actual operation region;

[0063] Figure 6 A flowchart of the present application for planning the positioning key point position of the target operation device on the target object surface model;

[0064] Figure 7 A flowchart of the present application for collecting positioning key point data for deviation and collision warning. DETAILED DESCRIPTION

[0065] Embodiment 1

[0066] Please refer to Figure 1 and Figure 2 , this embodiment introduces a guiding auxiliary device control system, which includes a data acquisition module, a dynamic registration module, an intelligent planning module and an execution control module.

[0067] The data acquisition module is used to access the target component model, the target installation area design model and the target operation device model, and to complete initialization and coordinate system calibration through a device-compatible light source emitting assembly and a light source receiving assembly, thereby establishing a unified data acquisition reference. Specifically, the light source emitting assembly projects a structured light pattern onto the target object surface, and the light source receiving assembly synchronously acquires a deformed light image modulated by the surface topography. Based on a stereo vision algorithm, the parallax is calculated and converted into a three-dimensional point cloud, while multi-dimensional features such as surface texture, curvature and reflectivity are extracted. The point cloud local fitting is used to calculate the normal vector and curvature value of each point, the texture information is obtained through the image color channel, and the bone surface material is distinguished through the reflectivity parameter. The discrete point cloud collected is processed by filtering and grid reconstruction, and a continuous target object surface model is generated in real time, thereby providing a high-precision data basis for subsequent registration and planning.

[0068] In the present embodiment, the guiding auxiliary device is used in cochlear implant surgery. In the cochlear implant surgery scene, the target component is a cochlear implant, and its functional feature points include implant positioning points, cochlear electrode tunnel positioning points, and curved surface fitting feature points. The curved surface fitting feature points reflect the edge sampling points of the curvature matching between the bottom surface of the implant and the skull surface. The target installation area design model is constructed based on preoperative CT data, and marks the cochlear implant safe area on the skull surface to avoid important blood vessels such as sigmoid sinus and transverse sinus. The edge is composed of uniformly sampled closed contour points, and the inside is extracted by medical image analysis to define the allowable curvature range. Intraoperative point cloud of the skull surface is collected by a structured light projector to accurately capture subtle structures such as bone grooves and bone ridges. After the target operation device model is connected, its coordinate system, end effector size and positioning key points such as drill bit tip and pressing surface center are uniformly converted to the surgical space coordinate system with the skull surface model, and the preliminary alignment of the geometric center of the design model and the pre-set implant area of the skull is realized through the preoperative calibration matrix, providing an initial reference for dynamic registration. The light source emitting assembly, as the core perception front end of the guiding auxiliary device, integrates a multi-modal optical projection unit that is designed to fully adapt to the high-precision imaging needs and biological safety standards of cochlear implant surgery. The multi-modal optical projection unit adopts a modular light source design, including a cold light source illuminator, a structured light projector, and a near-infrared auxiliary light source. The three work together to provide clear lighting and accurate three-dimensional data acquisition for the surgery, helping to ensure safety and efficiency.

[0069] The dynamic registration module is used for spatial alignment and deviation analysis of the real-time collected target object surface model and the design model of the target installation area. In precise installation and medical implantation scenes, the target object surface model usually covers a larger range than the target installation area design model, forming a global and local scale difference. To ensure that the target installation area design model is completely contained in the target object surface model and is in the geometric center position of the target object surface model, the dynamic registration module adopts a hierarchical registration strategy under regional semantic constraints, and through geometric containment verification, center alignment optimization and boundary constraint algorithm, it realizes high-precision spatial mapping of cross-scale models. First, the feature points are extracted to convert the two types of models to a unified coordinate system, establishing an initial spatial mapping relationship to provide a reference for accurate registration; a hierarchical registration strategy is adopted, first narrowing the position deviation range through a global search algorithm, and then using an iterative closest point or non-rigid deformation algorithm for precise registration, and iteratively optimizing until the surface fitting error is controlled within the accuracy range; in the cochlear implant surgery scene, the dynamic registration module integrates preoperative feature pre-calibration and intraoperative dynamic semantic constraint technology to realize an efficient operation mode of one-time registration and whole-process locking, thereby avoiding the infection risk and precision loss caused by traditional repeated measurement.

[0070] Preferably, the specific steps of spatial alignment and deviation analysis of the real-time collected target object surface model and the design model of the target installation area include:

[0071] Referring to Figure 3 , the design model of the target installation area is subjected to coordinate normalization, including translation to the origin and scaling to the unit scale, so as to eliminate the influence of model size and position difference on registration; an axis alignment bounding box is constructed through extreme point search to determine the spatial range, and the geometric center is marked as a registration reference point, thereby providing a basis for subsequent main inertia axis calculation and local coordinate system construction.

[0072] Taking the geometric center of the design model as the origin, the main inertia axis of the design model is determined through principal component analysis, a local coordinate system is constructed, and the long axis and short axis of the design model are ensured to be consistent with the key directions in actual application. The model posture is consistent with the geometric trend of the target object surface, and registration errors caused by posture deviation are avoided.

[0073] Key feature points in the design model are extracted, including functional feature points, edge feature points and curvature feature points; when the guide auxiliary device is applied to the cochlear implant surgery scene, the functional feature points include cochlear implant positioning points, cochlear electrode tunnel positioning points and curved surface fitting feature points, which are one-to-one corresponding to the functional feature points of the target component model; the edge feature points are uniformly sampled along the boundary of the design model to form a closed contour point set; the curvature feature points are extracted through curvature analysis to extract local extreme points, which are used for geometric shape matching. These feature points serve as core constraint elements for registration, ensuring the priority of the accuracy of the core functional area in the subsequent registration process.

[0074] The collected target object surface model is subjected to downsampling and outlier removal; the normal vector of each point of the target object surface model is calculated by fitting a plane through adjacent points, thereby providing surface direction information for subsequent registration;

[0075] The target object surface model is usually much larger than the design model, and the search range needs to be reduced through grid division and curvature screening, focusing on flat and regular areas with stable curvature, and excluding concave and convex areas with high curvature, so as to ensure that the potential area contains sufficient geometric features and is suitable for precise operation. Based on the bounding box size of the design model, the potential area containing the design model is screened in the target object surface model, and the potential area containing sufficient large and curvature matched potential area is searched in the target object surface model, thereby ensuring that the target area contains sufficient geometric features for subsequent registration.

[0076] Referring to Figure 4 , specifically, the specific steps of screening the potential area containing the design model in the target object surface model include:

[0077] Grid division is performed on the target object surface model to set an initial search range based on the size of the design model bounding box, and the target object surface model is divided into a plurality of candidate regions, for example, with twice the size of the design model bounding box as the initial search radius.

[0078] The candidate regions are sequentially traversed, the curvature values of the points on the surface of the candidate regions are calculated through a local surface fitting algorithm such as a nearby point polynomial fitting, the candidate regions with stable curvature are identified as target candidate regions such as flat bone surfaces or regular curved surfaces, and the concave-convex regions with high curvature such as bone grooves and bone ridges are excluded.

[0079] The curvature feature points of the target candidate regions are extracted and matched with the curvature feature points of the design model, and further precise matching verification is performed on the curvature features to ensure high conformity with the surface shape of the target part, i.e.

[0080] According to the design requirements of the target part, such as the ideal curvature parameters of the fitted surface, a matching degree threshold is configured, a curvature feature point of the design model is randomly selected as a reference point for matching verification, and the curvature features of the reference point are compared with all the curvature feature points in the target candidate region one by one to calculate their similarity.

[0081] If there is a curvature feature point in the target candidate region whose similarity with the reference point of the design model is greater than the matching degree threshold, the point is marked as an effective curvature feature point, indicating that the surface shape at this point has good matching with the fitted surface of the target part.

[0082] Taking the effective curvature feature point as the center and the bounding box size of the design model as the radius, an expanded verification range is divided, and more curvature feature points are extracted in the expanded verification range for further matching verification.

[0083] In the expanded verification range, all curvature feature points are extracted and compared with the curvature feature points of the design model one by one to calculate their similarity, and effective curvature feature points are selected.

[0084] The number of effective curvature feature points in the expanded verification range is counted as the matching density, which is used to evaluate the curvature matching consistency of the target region and the design model in a larger range, avoiding ignoring the overall shape difference due to the success of local single-point matching.

[0085] According to the matching density of the target candidate region, a potential region is selected, for example, the target candidate region with the highest matching density is selected as the potential region.

[0086] Please refer to Figure 5 In the potential region, the matched effective curvature feature points are taken as the reference to locate the corresponding edge feature points, and the actual operation region is generated.

[0087] According to the local coordinate system of the design model, the attitude of the design model is adjusted so that the principal inertia axis is consistent with the overall trend of the potential region, and the curved surface bending directions of the two are matched. Exemplarily, the long axis of the bottom surface of the implant is consistent with the front-to-back arc of the skull surface.

[0088] With the edge point of the design model as the center, the edge point of the target component in the target installation region of the target object surface model is identified based on the curvature feature in the potential region. Specifically, the edge point of the design model is searched in the potential region under double constraints: the target region point with a high similarity in curvature feature to the edge point of the design model is screened to ensure that the local geometric shapes of the two are consistent and the edge fitting deviation caused by the difference in curved surface is avoided; and the search range is limited to a reasonable region around the edge point of the design model to ensure that the mapped edge point is close to the boundary of the design model and is located in the effective geometric range of the potential region.

[0089] Preferably, the specific steps of searching and mapping the edge point of the design model in the potential region under double constraints include:

[0090] According to the accuracy requirement of the target component, the similarity threshold of the curvature feature is set, and the spatial search threshold with the edge point of the design model as the center is determined.

[0091] The edge point set of the design model is obtained, and each edge point carries its local geometric shape feature. Meanwhile, the point cloud data of the potential region is obtained, and each point contains three-dimensional coordinates and corresponding local geometric shape features.

[0092] For each edge point of the design model, the target region point within the spatial search threshold range is screened out by searching around the point as the center in the potential region, the range of subsequent feature matching is narrowed, and it is ensured that the mapped edge point is in the effective geometric range of the potential region.

[0093] The similarity of the local geometric shape feature of each of the above screened target region points to the edge point of the design model is calculated one by one. When the similarity of the local geometric shape of the target region point to the edge point of the design model is greater than the similarity threshold, it is marked as a candidate edge point, so that the curved surface shapes of the two are consistent and the edge fitting deviation caused by the difference in local geometry is avoided.

[0094] If the candidate edge point is not unique, the candidate edge point with the largest similarity is selected as the mapped edge point of the edge point of the design model. After the mapping is completed, it is checked whether the curved surface direction of the region where the mapped point is located matches the curved surface direction of the corresponding edge point of the design model, and it is confirmed that the mapped point is located in the effective range of the potential region, so as to ensure the feasibility of subsequent operations.

[0095] If a certain design model edge point cannot find a target region point that meets the geometric shape requirements within the preset spatial range, a virtual edge point is generated based on the position and shape characteristics of adjacent edge points through an interpolation method to ensure the continuity of the edge contour. If multiple consecutive design model edge points cannot find effective mapping points, it is determined that the current potential region and the edge shape of the design model do not match well, and an abnormality warning is given;

[0096] A corresponding relationship is established between the design model edge point and the mapped edge point of the screened potential region, all mapped edge points are connected in the order of the edge of the design model, a closed actual operation region contour is formed, and an actual operation region is generated. If there is a gap in the contour, an edge point is supplemented through interpolation to ensure the completeness of the contour, and a complete boundary basis is provided for the mapping of the functional feature point and the demarcation of the operation region.

[0097] The functional feature points of the design model are mapped to the actual operation region of the potential region through a rigid transformation matrix, so that these key functional points are located within the installation range defined by the edge contour.

[0098] A deviation threshold is configured, the geometric centroid of the region surrounded by the closed edge contour, i.e., the actual operation region, is calculated as the target installation center of the target component, the global centroid of the entire target object surface model is calculated to reflect the center of the guiding auxiliary device, and the spatial position deviation between the geometric centroid of the actual operation region and the global centroid of the target object surface model is calculated. If it is greater than the deviation threshold, it indicates that the target installation center of the target component deviates from the center position of the guiding auxiliary device, which is not conducive to actual operation, and triggers the device position adjustment mechanism; otherwise, an intelligent planning operation is triggered.

[0099] If the device position adjustment mechanism is triggered, the adjustment direction and movement distance of the guiding auxiliary device are calculated according to the spatial position deviation between the geometric centroid of the actual operation region and the global centroid of the target object surface model, and the guiding device sends a guiding instruction to the device through the light source emission component. The light source emission component uses light beams or light spots of different colors and flicker frequencies to intuitively indicate the movement direction of the device, for example, a green light beam points to the direction in which the device needs to move, and the light beam intensity corresponds to the movement distance. The auxiliary device quickly and accurately moves towards the target installation center to ensure that the reference center of the device after adjustment completely coincides with the ideal installation center of the target component.

[0100] When the guiding auxiliary device moves, the target object surface model is reacquired, and the actual operation area of the target object surface model is repositioned according to the adjustment direction and the moving distance. The spatial position deviation of the geometric centroid of the actual operation area and the global centroid of the target object surface model is calculated again, and it is judged whether it is greater than the deviation threshold. If it is greater, the device position adjustment mechanism is triggered again, otherwise the intelligent planning operation is triggered, and through curvature matching, edge contour closure checking, functional point dissection safety verification and device posture consistency detection, the global and local scale differences are effectively processed and the safety of the functional area positioning is guaranteed.

[0101] The intelligent planning module is used to plan the positioning key point position of the target operation device on the target object surface model based on the functional feature point position of the design model mapped to the actual operation area, in combination with the positioning key points and geometric shapes of the target operation device model when the intelligent planning operation is triggered. The positioning key points and the functional feature points are indirectly ensured to accurately operate the functional feature points through the preset spatial relationship between the positioning key points and the functional feature points, so as to avoid the operation deviation caused by the device shielding the functional feature points.

[0102] Please refer to Figure 6 , preferably, the specific steps of planning the positioning key point position of the target operation device on the target object surface model include:

[0103] The functional feature points output by the dynamic registration module are the core targets of the operation, and the positioning key points of the target operation device model are the reference for the device control. The two types of data and the target object surface model need to be unified to the same coordinate system to eliminate the coordinate deviation of different models and ensure the consistency of the spatial position calculation.

[0104] The functional feature point coordinates mapped in the actual operation area output by the dynamic registration module are received; the target operation device model is accessed to obtain its three-dimensional geometric shape parameters and positioning key points, including the end effector center point and the visual positioning marker point. The functional feature points, the device positioning key points and the target object surface model are uniformly mapped to the same spatial coordinate system to establish a unified reference for multi-source data and ensure the consistency of the spatial relationship calculation.

[0105] The functional feature points and the device positioning key points need to meet strict geometric relationships. A rigid spatial mapping relationship between the functional feature points and the target operation device positioning key points of the design model mapped to the actual operation area is established, the fixed offset and the posture constraint of the device positioning key points relative to the functional feature points are defined based on the translation vector and the rotation matrix, and a geometric alignment reference model of the device operation end and the functional feature points is formed. The geometric alignment reference model reflects the preset spatial relationship between the core part of the device operation end and the functional feature points, and serves as a reference for positioning planning.

[0106] According to the local geometric features of the region where the functional feature points are located, including the surface normal vector of the target object, the principal inertia axis of the design model, the operating posture of the target operating device is determined. For example, for the positioning point of the cochlear implant, the cochlear electrode tunnel positioning, in order to ensure that the implant, cochlear electrode and bone groove and tunnel are completely matched, the device positioning key point needs to be aligned along the normal vector direction of the point, so that the device operating end axis is consistent with the interface direction, and the posture requirements of anatomy or engineering assembly are met.

[0107] Based on the three-dimensional coordinates of the functional feature points and the operating posture, combined with the rigid space mapping relationship, the target position of the positioning key point of the target operating device in the coordinate system of the target object surface model is calculated through the rigid transformation matrix. The target position meets: the core part of the device operating end is strictly aligned with the functional feature point in space position and posture, even if the functional feature point is blocked by the device, the core operating part still accurately acts on the target point.

[0108] Combined with the geometric features of the target object surface model, including high curvature regions, anatomical structure boundaries, the safety of the target position corresponding to the positioning key point is verified: the minimum distance between the device model and the high curvature region of the target object surface, the structure boundary is calculated, if the distance is greater than the preset safety threshold determined by the risk level of the operating scene, the target position is qualified; otherwise, the position adjustment operation is triggered, under the premise of maintaining the geometric alignment relationship, the positioning key point position of the device is fine-tuned until the safety distance requirement is met. Avoid physical collision risk in the operation process, ensure the safety of the device motion path, especially suitable for medical implantation and other scenes with extremely high safety requirements.

[0109] When the target position is qualified, if the functional feature point is not unique, according to its operation priority, device kinematics constraint and geometric features of the target object surface, the operation sequence and collision-free motion path of the target operating device are planned by using the path planning algorithm. The operation priority is determined by the importance of the functional point and the operation process, the device kinematics constraint includes the joint activity range of the target operating device, the accessibility of the target operating device, the geometric features of the target object surface include the curvature distribution, the anatomical safety area, and the path planning algorithm includes A* algorithm, Dijkstra algorithm. Reduce the time-consuming of device repeated positioning, improve the operation efficiency while guarantee the kinematics feasibility of the device, suitable for multi-step precise operation scene.

[0110] The execution control module is used for operation guidance, deviation monitoring and effect evaluation of the target operation device. The operation sequence and collision-free motion path of the target operation device generated by the intelligent planning module are received to generate guidance instructions to provide target operation device operation guidance through the light source emitting assembly. Real-time actual position and attitude data of the positioning key points are collected to calculate the spatial deviation from the target position. The geometric safety of the device and the target object surface model is verified in real time to avoid collision risk and kinematic singularity. After the device operation is completed, the actual operation area is compared with the functional feature point position of the design model to verify the functional point operation accuracy. If the functional point operation accuracy does not meet the requirements, a correction instruction or a re-planning is triggered to realize the full-process closed-loop control of the device operation guidance, deviation monitoring and effect evaluation.

[0111] Preferably, the specific steps of operation guidance, deviation monitoring and effect evaluation of the target operation device include:

[0112] Referring to Figure 7 , the target position, operation sequence and collision-free motion path corresponding to the positioning key points output by the intelligent planning module are received. The target position is converted into a motion instruction executable by the target operation device. The format difference between the planning data and the device control interface is eliminated to provide a basis for subsequent visual guidance and accurate motion control, and to avoid operation failure caused by incompatible instructions.

[0113] Based on the motion instruction, the target operation device operation position, movement direction and operation progress are indicated by the light source emitting assembly through different color light beams, flashing frequencies and intensities to solve the problem of difficult manual judgment or insufficient device autonomous navigation accuracy in a complex three-dimensional space.

[0114] The target object surface model is reconstructed in real time through the light source receiving assembly, and the actual position and attitude data of the positioning key points are collected in real time through the built-in sensors of the target operation device, including the visual positioning system and the inertial measurement unit. Real-time data support is provided for deviation calculation and collision detection to avoid control delay caused by data lag.

[0115] The accuracy threshold is configured, including position accuracy and angle accuracy. The spatial deviation of the actual position from the target position is calculated, including position offset and angle deviation. The accuracy threshold is compared. If the accuracy threshold is greater than the accuracy threshold, the deviation warning is performed through the light source emitting assembly. The light source emitting assembly flashes with yellow light beam as an example, and the terminal interface displays the deviation value. Otherwise, no processing is performed.

[0116] The collision threshold is configured according to the risk area of the target object surface. Based on the bounding box of the target operation device model and the target object surface model, the distance between the high curvature area of the target operation device model and the target object surface model and the boundary of the anatomical structure is calculated in real time through the bounding box hierarchical tree algorithm. If the distance is less than the collision threshold, a collision warning is performed. Otherwise, no operation is performed.

[0117] A distance deviation threshold is configured, after the target operating device completes the operation, based on the point cloud data of the real-time target object surface model, the local region data where the functional feature points are located in the actual operation region is segmented by edge contour point cloud clustering, and the corresponding geometric features are extracted, the geometric features include normal vector, curvature distribution, the distance deviation of the geometric features of the local region data where the functional feature points of the target component model are located is calculated, if greater than the distance deviation threshold, it is determined that the target component model and the actual operation region are not matched, otherwise it is determined that the target component model and the actual operation region are matched;

[0118] If the target component model and the actual operation region are not matched, the functional feature points causing the target component model and the actual operation region to be not matched are identified, and the intelligent planning module is triggered to re-plan the operation path of the functional feature points not matched to generate local correction instructions.

[0119] Embodiment 2

[0120] This embodiment introduces a guide auxiliary device for realizing high-precision registration of cross-scale models and device operation guidance, solving the operation deviation problem in the global and local scale difference and occlusion scene, Figure 1 The structure diagram of the guide auxiliary device disclosed by the present application is shown in the figure, the device mainly consists of a support structure, a light source emitting and receiving structure, a data transmission structure and an external processing structure, each structure cooperates with each other to realize the construction of the target object surface model, operation guidance and data processing and other functions, and provides support for precise operation. In the precise medical implantation scene, high-precision and high-reliability device operation guidance and closed-loop control are realized.

[0121] The support structure serves as the physical basis of the guiding assistance device, defining the collection range of the target object surface model through adjustable geometry and degrees of freedom, adapting to target objects of different sizes and curved surface shapes. In the cochlear implant surgery scenario, this structure exhibits significant clinical value, reducing the risk of damage to surrounding tissues during surgery and optimizing the allocation of surgical manpower. In traditional cochlear implant surgery, to fully expose the surgical field, a retractor is often used to pull the nearby tissue cortex for a long time, which can easily cause complications such as tissue ischemia and swelling. The support structure can accurately position the light source and image acquisition device above the surgical area through flexible form adjustment, forming a non-contact vision acquisition scheme. At the same time, its lightweight design and soft scratch-resistant coating surface avoid pressure and scratches on the patient's skin, effectively reducing the risk of tissue damage and shortening the patient's postoperative recovery period. In previous surgeries, a dedicated person was responsible for holding and adjusting the image acquisition device to ensure a clear view. The support structure, with its highly automated adjustment function and stable fixing effect, can replace manual operation. The surgical team does not need to arrange additional personnel to hold the device, and a single person can complete the position adjustment of the device, reducing the number of auxiliary operators, optimizing the allocation of surgical room manpower, and making team collaboration more efficient and smooth.

[0122] The light source emission-receiving structure is used to realize the construction of the target object surface model, synchronously acquire multi-dimensional features of the target object surface model, and provide visual operation guidance, indicating the target position corresponding to the positioning key point of the target operation device, deviation warning, and operation progress, including a light source emission component, a light source receiving component, and a guidance signal component;

[0123] Through the light source emission component, such as an infrared structured light projector, a coded light pattern of Gray code, sinusoidal stripe, etc. is projected onto the target object surface. The light source receiving component collects the deformed light image modulated by the surface topography. Based on the stereo vision algorithm, three-dimensional point cloud data is calculated in real time, and multi-dimensional features such as surface texture, curvature, and reflectivity are synchronously acquired. Through a detachable calibration board or a self-calibration algorithm, the spatial conversion relationship between the light source emission component, the light source receiving component, and the target operation device is established, ensuring that the design model, the target object surface model, and the device model are uniformly mapped to the same spatial coordinate system. The guidance component provides visual operation guidance, pointing to the target position corresponding to the positioning key point of the device;

[0124] The data transmission structure is used to realize the mechanical fixation of the guiding assistance device, the calibration of the spatial coordinate system, and the data interaction;

[0125] The external processing structure is used for real-time processing of three-dimensional point cloud data, performing dynamic registration, intelligent planning and safety control, performing dynamic registration through feature point extraction, potential area screening and hierarchical registration, performing intelligent planning through rigid transformation matrix calculation and path planning, and performing deviation monitoring and collision warning.

[0126] Working principle and effects:

[0127] The application accesses target components, designs installation areas and operation equipment models, uses structured light vision technology to collect multi-dimensional data of target object surfaces to construct high-precision surface models, provides data basis for cross-scale registration, extracts key features such as functional points, edge points and curvature points in the design model, divides the target object surface based on the size grid of the design model, screens potential areas with stable curvature through curvature matching and matching density statistics, ensures the geometric form of the core area to fit, and solves the registration problem caused by the difference between global and local scales.

[0128] In the potential area, the edge points of the design model are mapped to form a closed operation area through curvature similarity and spatial distance double constraints, the device position is adjusted combined with the centroid deviation analysis to realize accurate spatial alignment of the design model and the actual scene, and the registration accuracy is improved. After triggering the planning, the rigid spatial mapping relationship between the functional feature points and the device positioning key points is established, the device target position is calculated through the rigid transformation matrix, and the safety is verified, the collision-free path is generated combined with the path planning algorithm, the device operation end and the functional point space position and attitude are strictly aligned, and the limitation of relying on direct visibility in the visual guidance in the occluded scene is broken through.

[0129] During operation, the device position and progress are indicated through light source visualization, the positioning data are collected in real time for deviation and collision warning, and the operation risk is reduced; after operation, the local area of the functional point is segmented to extract geometric features, and the matching degree is determined by comparing with the design model, when the matching degree is not matched, only the abnormal point path is re-planned, a dynamic closed loop is formed, global reset is avoided, the operation accuracy and safety in the complex curved surface environment are significantly improved, and the precise medical implantation scene is effectively adapted.

[0130] The above only describes the preferred embodiments of the application, and the protection scope of the application is not limited to the above-mentioned embodiments, and any technical scheme falling within the idea of the application belongs to the protection scope of the application. It should be noted that for ordinary skilled persons in the art, some improvements and decorations without departing from the principle of the application can also be considered as the protection scope of the application.

Claims

1. A guidance assistance device control system characterized by comprising: The method comprises the following steps: accessing a target component model, a design model of a target installation area, and a target operating device model, and collecting multi-dimensional data of a target object surface to construct a target object surface model; extracting key feature points in the design model, dividing a candidate area of the target object surface model based on a bounding box size of the design model, identifying a target candidate area and extracting curvature feature points, screening effective curvature feature points by configuring a matching degree threshold, dividing an extended verification range, and selecting a potential area by counting the matching density in the extended verification range; mapping edge points of the design model in the potential area of the target object surface model based on curvature feature similarity and spatial distance constraints to form a closed contour as an actual operating area, performing spatial alignment of the target object surface model and the design model according to the deviation of the geometric centroid of the actual operating area from the global centroid of the target object surface model, and determining whether to trigger intelligent planning operation; when the intelligent planning operation is triggered, generating a guide instruction by planning the position of a positioning key point of the target operating device on the target object surface model based on the position of a functional feature point of the design model mapped to the actual operating area and in combination with the positioning key point and the geometric shape of the target operating device model; providing target operating device operation guidance through a light source emitting assembly, performing deviation and collision warning by collecting positioning key point data in real time, and determining whether the target component model matches the actual operating area by the geometric features of the local area where the functional feature points are located after the operation is completed, and generating a local correction instruction if the target component model does not match the actual operating area.

2. A guidance assistance device control system according to claim 1, wherein The step of performing spatial alignment of the target object surface model and the design model comprises: performing coordinate normalization on the target installation area design model, constructing an axis-aligned bounding box of the design model and marking the geometric center of the design model, taking the geometric center as the origin, determining the principal inertia axis of the design model through principal component analysis, and constructing a local coordinate system; extracting key feature points in the design model, wherein the key feature points include functional feature points, edge feature points, and curvature feature points; based on the bounding box size of the design model, identifying effective curvature feature points and dividing an extended verification range by performing grid division and curvature feature point matching on the target object surface model to select a potential area containing the design model in the target object surface model; in the potential area, searching and mapping edge points of the design model based on curvature feature similarity and spatial distance constraints with the matched effective curvature feature points as the reference to generate an actual operating area, and mapping functional feature points of the design model to the actual operating area in the potential area.

3. A guidance assistance device control system according to claim 2, wherein The step of performing spatial alignment of the target object surface model and the design model further comprises: configuring a deviation threshold, calculating the geometric centroid of the actual operating area as a target installation center, and calculating the global centroid of the target object surface model as a guide auxiliary device center; calculating the spatial position deviation of the target installation center and the guide auxiliary device center, and if the spatial position deviation is greater than the deviation threshold, triggering a device position adjustment mechanism; otherwise, triggering intelligent planning operation; The device position adjustment mechanism comprises: The adjustment direction and movement distance of the guiding auxiliary device are calculated according to the spatial position deviation, the movement direction and distance of the guiding auxiliary device are indicated by the light source emission assembly, and the center of the guiding auxiliary device is moved to the target installation center; After the guiding auxiliary device is moved, the target object surface model is reacquired, and the actual operation area and the key feature points are repositioned based on the adjustment direction and the movement distance.

4. A guidance assistance device control system according to claim 2, wherein The step of selecting the potential region containing the design model in the target object surface model comprises: Grid division is performed on the target object surface model, an initial search range is set based on the size of the bounding box of the design model, and the target object surface model is divided into a plurality of candidate regions; The candidate regions are traversed, the surface point curvature values are calculated by using a local surface fitting algorithm, and the target candidate region is identified; The curvature feature points of the target candidate region are extracted, the similarity of the curvature feature points is calculated, the curvature feature points of the design model are matched, and the effective curvature feature points are identified; An extension verification range is demarcated with the effective curvature feature points as the center and the size of the bounding box of the design model as the radius, all the curvature feature points in the extension verification range are extracted and the effective curvature feature points are screened, the number of the effective curvature feature points is counted to form a matching density, and the target candidate region is selected as the potential region according to the matching density of the target candidate region.

5. A guidance assistance device control system according to claim 2, wherein The step of generating the actual operation area comprises: Similarity threshold and spatial search threshold are set, edge point set and local geometric feature of the design model are acquired, point cloud data of the potential region and three-dimensional coordinates and local geometric feature thereof are acquired; For each edge point of the design model, a target region point located within the spatial search threshold range is screened out with the edge point as the center in the potential region; The local geometric feature of the target region point and the similarity of the edge point of the design model are calculated, if the similarity is greater than the similarity threshold, the target region point is marked as a candidate edge point, and a mapping edge point of the design model edge point is selected according to the similarity of the candidate edge point; The design model edge point and the screened mapping edge point of the potential region are corresponded, the mapping edge points are connected in the order of the design model edge, and a closed actual operation area contour is formed.

6. A guidance assistance device control system according to claim 1, wherein The step of planning the position of the key point of the target operation device on the target object surface model comprises: The coordinates of the functional feature points in the actual operation area are received, the target operation device model is accessed, and the three-dimensional geometric shape parameters and the key point position are acquired; The functional feature points, the device key point and the target object surface model are mapped to the same spatial coordinate system; A rigid spatial mapping relationship between the functional feature points and the device key point is established, a fixed offset and a posture constraint of the device key point relative to the functional feature points are defined based on a translation vector and a rotation matrix, a geometric alignment reference model is formed, and the operation posture of the target operation device is determined.

7. A guidance assistance device control system according to claim 6, wherein The step of planning the position of the key point of the target operation device on the target object surface model further comprises: Based on the three-dimensional coordinates of the functional feature points, the operation posture and the rigid space mapping relationship, the target position of the positioning key point in the coordinate system of the target object surface model is calculated through the rigid transformation matrix, so that the operation end of the device is aligned with the spatial position and posture of the functional feature points; Combined with the geometric features of the target object surface model, the safety of the target position corresponding to the positioning key point is verified to determine whether the target position is qualified; When the target position is qualified, the operation sequence and collision-free motion path of the functional feature points are planned by using a path planning algorithm according to the operation priority of the functional feature points, the kinematic constraints of the device and the geometric features of the target object surface.

8. A guidance assistance device control system according to claim 7, wherein, The step of collecting the positioning key point data for deviation and collision warning includes: Receiving the target position corresponding to the positioning key point, the operation sequence and the collision-free motion path, and converting the target position into a motion instruction executable by the target operation device; Based on the motion instruction, the operation position, moving direction and operation progress of the target operation device are indicated by the light source emitting component; The target object surface model is reconstructed in real time by the light source receiving component, and the actual position and posture data of the positioning key point are collected in real time by the built-in sensor of the target operation device; A precision threshold is configured, the spatial deviation between the actual position and the target position is calculated and compared with the precision threshold, if it is greater than the precision threshold, a deviation warning is given by the light source emitting component, otherwise no operation is performed; A collision threshold is configured, based on the bounding box of the target operation device model and the target object surface model, the distance between the high curvature region of the target operation device model and the target object surface model and the boundary of the anatomical structure is calculated in real time by the bounding box hierarchical tree algorithm, if it is less than the collision threshold, a collision warning is given by the light source emitting component, otherwise no operation is performed.

9. A guidance assistance device control system according to claim 1, wherein, The step of determining whether the target component model matches the actual operation area includes: A distance deviation threshold is configured, after the target operation device completes the operation, the local area data where the functional feature points are located is segmented and the geometric features are extracted, the distance deviation between the geometric features of the local area where the functional feature points corresponding to the target component model are located is calculated, if it is greater than the distance deviation threshold, it is determined that they do not match, otherwise it is determined that they match; If it is determined that they do not match, the functional feature points that cause the mismatch are identified, the operation path is re-planned and local correction instructions are generated.

10. A guidance assistance device comprising a guidance assistance device control system according to any one of claims 1 to 9, characterized in that, It also includes a support structure, a light source emitting and receiving structure, a data transmission structure and an external processing structure; The support structure serves as the physical basis of the guiding auxiliary device, and defines the collection range of the target object surface model through adjustable geometric shapes and degrees of freedom; The light source emitting and receiving structure is used to realize the construction of the target object surface model, to synchronously acquire the multi-dimensional features of the target object surface model, and to provide visual operation guidance to indicate the target position corresponding to the positioning key point of the target operation device, deviation warning and operation progress; The data transmission structure is used to realize the mechanical fixation, space coordinate system calibration and data interaction of the guiding auxiliary device; The external processing structure is used to process three-dimensional point cloud data in real time, to perform dynamic registration, intelligent planning and safety control.

Images