Guiding auxiliary device and control system thereof

Abstract

The invention discloses a guiding auxiliary device and a control system thereof, and belongs to the technical field of feature registration, and the method comprises the steps: building a target object surface model through multi-model access and three-dimensional point cloud reconstruction; curvature feature matching and potential region screening are utilized to solve the registration problem caused by global and local scale differences; a closed operation area is formed through edge point mapping, and precise space alignment of a design model and an actual scene is achieved by combining centroid deviation adjustment; planning equipment positioning key points based on a rigid transformation matrix, ensuring that an operation end is geometrically aligned with the functional feature points, and generating a collision-free path through safety verification; dynamic feedback of the operation process is achieved by means of light source visual guidance and real-time deviation monitoring, effect evaluation is conducted through functional point geometric feature comparison after operation, and global reset is avoided through local correction; and the equipment operation precision and safety in a precise medical implantation scene are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of feature registration, and more particularly to a guidance assisting device and a control system thereof. Background Art

[0002] In areas such as precision manufacturing, high-end equipment installation, and complex component positioning, which require extremely high precision in surface treatment and component fitting, existing technologies face a series of pressing challenges. Traditional methods rely on manual, repeated measurement of molds or offline scanning, making it difficult to obtain complete three-dimensional information about the object surface in real time. In processing scenarios with complex curves or dynamic changes, the surface features cannot be accurately captured, resulting in processing accuracy being significantly affected by human factors, making it difficult to meet high-precision fitting requirements.

[0003] At the same time, processing path planning is often based on fixed models or standard parameters, which deviate from the actual surface geometry of the object. Operators need to frequently intervene manually during the process to compare the degree of fit. In the processing of complex surfaces with multiple curvatures and features, this problem leads to low process efficiency and significantly increased correction costs. In addition, in processing processes involving the collaboration of multiple devices, lighting, mapping, execution, etc. usually operate independently and lack a real-time linkage mechanism. When the material or shape of the object surface changes, it is difficult to automatically adjust the processing strategy. More importantly, the geometric parameters of different target components vary greatly. For example, when the target component is a cochlear implant, different cochlear molds have size and shape differences. Traditional methods require the design of a dedicated cochlear mold for each type of cochlea, which lacks versatility and has a long preparation cycle when adapting new components. In addition, the monitoring methods for sensitive areas of the object surface during processing are limited. The control of safety boundaries relies on manual judgment, which is prone to misjudgment due to factors such as limited operating perspective and fatigue, causing irreversible damage.

[0004] The above-mentioned problems make it difficult for the overall qualification rate of precision component installation to reach an ideal level. With the continuous growth of personalized customization demand, higher requirements are placed on the versatility, real-time performance and intelligence of processing devices. Therefore, in order to overcome these limitations, the present invention proposes a guidance auxiliary device and its control system. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a guidance assistance device and its control system, which connects to multiple models and collects multi-dimensional data of the target object surface, extracts key feature points of the design model and screens potential areas of the target object surface based on curvature matching, establishes an edge point mapping relationship between the design model and the actual operation area to form a closed contour, and realizes spatial alignment in combination with the center of mass deviation adjustment; ensures geometric alignment through a rigid transformation matrix when planning the key points of equipment positioning, and generates a collision-free path in combination with safety verification; collects data in real time during operation for deviation and collision warnings, and realizes matching degree judgment and local correction through comparison of the geometric features of functional feature points after the operation, thereby effectively handling global and local scale differences in precision medical implant scenarios, solving the problem of inaccurate positioning caused by occlusion, and realizing high-precision equipment operation guidance and closed-loop control.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A guidance assist device control system, comprising:

[0008] Access the target component model, the design model of the target installation area, and the target operating equipment model, collect multi-dimensional data on the target object surface, and construct a target object surface model;

[0009] Extract key feature points from the design model, divide the target object surface model into candidate regions based on the design model bounding box size, identify the target candidate regions and extract curvature feature points, filter valid curvature feature points by configuring the matching threshold, divide the extended verification range, and select potential regions by counting the matching density within the extended verification range;

[0010] The edge points of the design model are mapped within the potential area of ​​the target object surface model based on the similarity of curvature features and spatial distance constraints to form a closed contour as the actual operation area. Based on the deviation between the geometric center of mass of the actual operation area and the global center of mass of the target object surface model, the target object surface model and the design model are spatially aligned, and a decision is made as to whether to trigger an intelligent planning operation.

[0011] When triggering the intelligent planning operation, based on the functional feature point positions mapped from the design model to the actual operation area, combined with the positioning key points and geometric shape of the target operation device model, the positioning key points of the target operation device on the target object surface model are planned to generate guidance instructions;

[0012] The light source emission component provides guidance for target operating equipment operation, and real-time data of key positioning points is collected for deviation and collision warning. After the operation is completed, the geometric features of the local area where the functional feature points are located are used to determine whether the target component model matches the actual operating area. If they do not match, local correction instructions are generated.

[0013] Specifically, the steps of spatially aligning the target object surface model with the design model include:

[0014] Perform coordinate normalization on the design model of the target installation area, construct an axis-aligned bounding box of the design model and mark the geometric center of the design model. Using the geometric center as the origin, determine the principal inertia axes of the design model through principal component analysis and construct a local coordinate system.

[0015] Extract key feature points from the design model, including functional feature points, edge feature points, and curvature feature points;

[0016] Based on the size of the design model's bounding box, the target object's surface model is meshed and matched with curvature feature points to identify valid curvature feature points and divide the extended verification range to select potential areas containing the design model in the target object's surface model.

[0017] In the potential area, based on the matched effective curvature feature points and the similarity of curvature features and spatial distance constraints, the edge points of the mapped design model are searched to generate the actual operation area, and the functional feature points of the design model are mapped to the actual operation area of ​​the potential area.

[0018] Specifically, the step of spatially aligning the target object surface model with the design model further includes:

[0019] Configure the deviation threshold, calculate the geometric center of mass of the actual operation area as the target installation center, and calculate the global center of mass of the target object surface model as the center of the guidance auxiliary device;

[0020] Calculate the spatial position deviation between the target installation center and the center of the guidance auxiliary device. If the spatial position deviation is greater than the deviation threshold, trigger the device position adjustment mechanism; otherwise, trigger the intelligent planning operation;

[0021] The device position adjustment mechanism includes:

[0022] Calculate the adjustment direction and movement distance of the guidance auxiliary device based on the spatial position deviation, and indicate the movement direction and movement distance of the guidance auxiliary device through the light source emission component, so that the center of the guidance auxiliary device moves to the target installation center;

[0023] After guiding the auxiliary device to move, the surface model of the target object is reacquired, and the actual operation area and key feature points are relocated based on the adjustment direction and movement distance.

[0024] Specifically, the step of selecting a potential region containing a design model in the target object surface model includes:

[0025] Perform meshing on the surface model of the target object, set the initial search range based on the size of the design model bounding box, and divide the surface model of the target object into multiple candidate areas;

[0026] Traverse the candidate area, calculate the surface point curvature value through the local surface fitting algorithm, and identify the target candidate area;

[0027] Extract the curvature feature points of the target candidate area, calculate the similarity of the curvature feature points, match them with the curvature feature points of the design model, and identify the valid curvature feature points;

[0028] The extended verification range is defined with the effective curvature feature point as the center and the design model bounding box size as the radius. All curvature feature points within the extended verification range are extracted and the effective curvature feature points are screened. The number of effective curvature feature points is counted to form the matching density. According to the matching density of the target candidate area, the target candidate area is selected as the potential area.

[0029] Specifically, the steps of generating the actual operation area include:

[0030] Set the similarity threshold and spatial search threshold to obtain the edge point set of the design model and its local geometric features, obtain the point cloud data of the potential area, and its three-dimensional coordinates and local geometric features;

[0031] For each edge point of the design model, the target area points within the spatial search threshold range are screened out with the edge point as the center in the potential area;

[0032] Calculate the similarity between the local geometric features of the target area points and the edge points of the design model. If the similarity is greater than the similarity threshold, mark it as a candidate edge point, and select the mapping edge point of the design model edge point based on the similarity of the candidate edge point;

[0033] Establish a correspondence between the edge points of the design model and the mapped edge points of the screened potential area, and connect the mapped edge points according to the edge order of the design model to form a closed outline of the actual operation area.

[0034] Specifically, the steps of planning the positioning key points of the target operating device on the surface model of the target object include:

[0035] Receive the coordinates of the functional feature points in the actual operation area, access the target operation device model and obtain its three-dimensional geometric shape parameters and positioning key points;

[0036] Map functional feature points, equipment positioning key points and target object surface models to the same spatial coordinate system;

[0037] Establish a rigid spatial mapping relationship between functional feature points and device positioning key points, define the fixed offset and posture constraints of the device positioning key points relative to the functional feature points based on the translation vector and rotation matrix, form a geometric alignment reference model, and determine the operating posture of the target operating device;

[0038] Specifically, the step of planning the positioning key point positions of the target operating device on the target object surface model further includes:

[0039] Based on the three-dimensional coordinates of the functional feature points, the operating posture and the rigid space mapping relationship, the target position of the device positioning key point in the target object surface model coordinate system is calculated through the rigid transformation matrix, so that the device operating end is aligned with the spatial position and posture of the functional feature points;

[0040] Combined with the geometric features of the target object surface model, verify the safety of the target position corresponding to the positioning key points and determine whether the target position is qualified;

[0041] When the target position is qualified, the path planning algorithm is used to plan the operation sequence of the functional feature points and the collision-free motion path according to the operation priority of the functional feature points, the kinematic constraints of the equipment and the surface geometric characteristics of the target object.

[0042] Specifically, the steps of collecting positioning key point data for deviation and collision warning include:

[0043] Receive the target position, operation sequence and collision-free motion path corresponding to the positioning key point, and convert the target position into motion instructions executable by the target operation device;

[0044] Based on the motion instructions, the light source emission component indicates the operating position, moving direction and operation progress of the target operating device;

[0045] The surface model of the target object is reconstructed in real time through the light source receiving component, and the actual position and posture data of the positioning key points are collected in real time using the built-in sensors of the target operation device;

[0046] Configure the accuracy threshold, calculate the spatial deviation between the actual position and the target position, and compare it with the accuracy threshold. If it is greater than the accuracy threshold, the light source emission component will issue a deviation warning; otherwise, no action will be taken.

[0047] Configure the collision threshold. Based on the bounding box of the target operation device model and the surface model of the target object, the bounding box hierarchical tree algorithm is used to calculate in real time the distance between the target operation device model and the high curvature area and anatomical structure boundary of the target object surface model. If the distance is less than the collision threshold, a collision warning is issued through the light source emission component. Otherwise, no operation is performed.

[0048] Specifically, the step of determining whether the target component model matches the actual operation area includes:

[0049] A distance deviation threshold is configured, after the target operating device completes the operation, the local region data where the functional feature points of the actual operation region are divided is extracted, the geometric features are calculated, the distance deviation of the geometric features of the local region where the functional feature points corresponding to the target component model are located is calculated, if it is greater than the distance deviation threshold, it is determined that it is not matched, otherwise it is determined that it is matched;

[0050] If it is determined that it is not matched, the functional feature points causing the mismatch are identified, the operation path is re-planned, and local correction instructions are generated.

[0051] A guiding auxiliary device, comprising: a support structure, a light source emission receiving structure, a data transmission structure and an external processing structure;

[0052] The support structure serves as the physical basis of the guiding auxiliary device, and defines the collection range of the target object surface model through adjustable geometric shapes and degrees of freedom;

[0053] The light source emission receiving structure is used to realize the construction of the target object surface model, synchronously acquire the multi-dimensional features of the target object surface model, and provide visual operation guidance, indicating the target position corresponding to the positioning key point of the target operating device, deviation warning and operation progress;

[0054] The data transmission structure is used to realize the mechanical fixation of the guiding auxiliary device, the calibration of the spatial coordinate system and the data interaction;

[0055] The external processing structure is used to process three-dimensional point cloud data in real time, and execute dynamic registration, intelligent planning and safety control.

[0056] The beneficial effects of the present application are:

[0057] The present application realizes accurate registration of cross-scale models by accessing multiple models and collecting multi-dimensional data of the target object surface, and realizes cross-scale model accurate registration by using curvature feature matching and potential area screening, solves the registration problem caused by global and local scale differences; based on the rigid space mapping relationship, the positioning key points of the device are planned, the operation end and the geometric alignment of the functional feature points are ensured in the occlusion scene, and the limitation of relying on direct visibility in visual guidance is broken through; through real-time deviation and collision warning, comparison of geometric features of functional feature points after operation and local correction, dynamic closed-loop control is formed, operation precision and safety are significantly improved; support complex curved surface environment and medical implantation, effectively exclude high curvature risk area, realize efficient adaptation, provide high precision, high robustness solution for precision operation. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The structure diagram of the guiding auxiliary device of the present application is shown in the figure;

[0059] Figure 2A schematic diagram of the structural principle of a guiding auxiliary device control system according to the present application;

[0060] Figure 3 A flowchart of the present application for spatial alignment of the target object surface model and the design model;

[0061] Figure 4 A flowchart of the present application for selecting a potential region containing the design model in the target object surface model;

[0062] Figure 5 A flowchart of the present application for generating an actual operation region;

[0063] Figure 6 A flowchart of the present application for planning the positioning key point position of the target operation device on the target object surface model;

[0064] Figure 7 A flowchart of the present application for collecting positioning key point data for deviation and collision warning. DETAILED DESCRIPTION

[0065] Embodiment 1

[0066] Please refer to Figure 1 and Figure 2 , this embodiment introduces a guiding auxiliary device control system, which includes a data acquisition module, a dynamic registration module, an intelligent planning module and an execution control module.

[0067] The data acquisition module is used to access the target component model, the target installation area design model and the target operation device model, and to complete initialization and coordinate system calibration through a device-compatible light source emitting assembly and a light source receiving assembly, thereby establishing a unified data acquisition reference. Specifically, the light source emitting assembly projects a structured light pattern onto the target object surface, and the light source receiving assembly synchronously acquires a deformed light image modulated by the surface topography. Based on a stereo vision algorithm, the parallax is calculated and converted into a three-dimensional point cloud, while multi-dimensional features such as surface texture, curvature and reflectivity are extracted. The point cloud local fitting is used to calculate the normal vector and curvature value of each point, the texture information is obtained through the image color channel, and the bone surface material is distinguished through the reflectivity parameter. The discrete point cloud collected is processed by filtering and grid reconstruction, and a continuous target object surface model is generated in real time, thereby providing a high-precision data basis for subsequent registration and planning.

[0068] In this embodiment, the guidance assist device is used in cochlear implant surgery. In the cochlear implant surgery scenario, the target component is the cochlear implant. Its functional feature points include implant positioning points, cochlear electrode tunnel positioning points, and surface alignment feature points. The surface alignment feature points represent edge sampling points where the curvature of the implant bottom surface matches that of the skull surface. A design model of the target installation area is constructed based on preoperative CT data. It marks a safe area on the skull surface for cochlear implantation, avoiding critical blood vessels such as the sigmoid and transverse sinuses. Its edge is composed of evenly sampled closed contour points. Internally, curvature feature points are extracted through medical image analysis and the permissible curvature range is defined. During surgery, a structured light projector is used to capture a point cloud on the skull surface, accurately capturing microstructures such as bone grooves and ridges. After the target operating device model is integrated, its coordinate system, end-effector dimensions, and key positioning points, such as the drill tip and the center of the pressure surface, are uniformly converted to the surgical space coordinate system along with the skull surface model. A preoperative calibration matrix is ​​used to achieve preliminary alignment of the design model's geometric center with the pre-determined skull implant area, providing an initial benchmark for dynamic registration. The light source emitting component serves as the core sensing front end of the guidance assistance device and integrates a multimodal optical projection unit. Its design fully adapts to the high-precision imaging requirements and biosafety standards of cochlear implant surgery. The multimodal optical projection unit adopts a modular light source design, including: a cold light source illuminator, a structured light projector and a near-infrared auxiliary light source; the three work together to provide clear lighting and accurate three-dimensional data acquisition for surgery, helping the surgery to be carried out safely and efficiently.

[0069] The dynamic registration module is used to spatially align and analyze deviations between the real-time acquired surface model of the target object and the design model of the target installation area. In scenarios such as precision installation and medical implants, the target object surface model typically covers a larger area than the design model of the target installation area, resulting in global and local scale differences. To ensure that the design model of the target installation area is completely contained within the target object surface model and geometrically centered on the target object surface model, the dynamic registration module employs a hierarchical registration strategy under regional semantic constraints. Through geometric inclusion verification, center alignment optimization, and boundary constraint algorithms, high-precision spatial mapping of models across scales is achieved. First, the two types of models are converted into a unified coordinate system through feature point extraction, and the initial spatial mapping relationship is established to provide a benchmark for precise alignment; a hierarchical alignment strategy is adopted, firstly the position deviation range is quickly narrowed by a global search algorithm, and then the iterative nearest point or non-rigid deformation algorithm is used for precise alignment, and the iterative optimization is carried out successively until the surface fitting error is controlled within the accuracy range; in the cochlear implant surgery scenario, the dynamic alignment module meets the precise positioning requirements of the cochlear mold, integrates preoperative feature pre-calibration and intraoperative dynamic semantic constraint technology, and realizes an efficient operation mode of one-time alignment and full locking, thereby avoiding the infection risk and accuracy loss caused by traditional repeated measurements from the root.

[0070] Preferably, the specific steps of performing spatial alignment and deviation analysis on the real-time collected target object surface model and the design model of the target installation area include:

[0071] See also Figure 3 , normalize the coordinates of the design model of the target installation area, including translating to the origin and scaling to the unit scale to eliminate the influence of model size and position differences on the registration; construct an axis-aligned bounding box by searching for extreme points to determine the spatial range, mark the geometric center as the registration reference point, and provide a basis for the subsequent calculation of the main inertia axes and the construction of the local coordinate system.

[0072] Using the geometric center of the design model as the origin, principal component analysis is used to determine the principal axes of inertia of the design model. A local coordinate system is then constructed to ensure that the major and minor axes of the design model are consistent with the key directions in the actual application. This ensures that the model's posture is consistent with the geometric orientation of the target object's surface, avoiding registration errors caused by posture deviations.

[0073] Extract key feature points from the design model, including functional feature points, edge feature points, and curvature feature points; when the guidance assist device is applied to cochlear implant surgery scenarios, the functional feature points include cochlear implant positioning points, cochlear electrode tunnel positioning points, and surface fitting feature points, which correspond one-to-one to the functional feature points of the target component model; edge feature points are evenly sampled along the boundary of the design model to form a closed contour point set; curvature feature points are extracted through curvature analysis to extract local extreme points for geometric shape matching. These feature points serve as the core constraint primitives for registration to ensure that the accuracy of the core functional areas is prioritized in the subsequent registration process.

[0074] The collected surface model of the target object is downsampled and outliers are removed; the plane is fitted by neighboring points, and the normal vector of each point on the surface model of the target object is calculated to provide surface direction information for subsequent registration;

[0075] The target object surface model is often much larger than the design model, so meshing and curvature filtering are required to narrow the search range, focusing on flat, regular areas with stable curvature and excluding concave and convex areas with high curvature. This ensures that the potential areas contain sufficient geometric features and are suitable for precision operations. Based on the size of the design model's bounding box, the target object surface model is screened for potential areas containing the design model. By searching the target object surface model for potential areas that are large enough and have matching curvature, it is ensured that the target area contains sufficient geometric features for subsequent registration.

[0076] See also Figure 4 Specifically, the specific steps of screening the potential area containing the design model in the target object surface model include:

[0077] Performing meshing on the target object surface model, setting an initial search range based on the design model bounding box size, illustratively using twice the design model bounding box size as the initial search radius, and dividing the target object surface model into multiple candidate areas;

[0078] The candidate regions are traversed in sequence. The curvature values ​​of each point on the surface of the candidate region are calculated using a local surface fitting algorithm, such as polynomial fitting of neighboring points. The candidate regions with stable curvature are identified as target candidate regions, such as flat bone surfaces or regular arc surfaces. High curvature concave-convex regions, such as bone grooves and bone ridges, are excluded.

[0079] Extract the curvature feature points of the target candidate area and match them with the curvature feature points of the design model. Further verify the precise matching of the curvature features to ensure a high degree of fit with the surface morphology of the target component, namely:

[0080] According to the design requirements of the target component, such as the ideal curvature parameters of the fitting surface, a matching threshold is configured, and a curvature feature point of the design model is randomly selected as the reference point for matching verification. The curvature feature of the reference point is compared one by one with all the curvature feature points in the target candidate area, and the similarity between the two is calculated.

[0081] If there is a curvature feature point in the target candidate area and its similarity with the design model reference point is greater than the matching threshold, the point is marked as a valid curvature feature point, indicating that the surface morphology here has a good match with the target component fitting surface.

[0082] Taking the effective curvature feature point as the center, the model bounding box size is designed as the radius to divide the extended verification range. More curvature feature points are further extracted within the extended verification range for deep matching verification.

[0083] Within the extended verification range, all curvature feature points are fully extracted, and similarity calculations are performed one by one with the curvature feature points of the design model to screen valid curvature feature points;

[0084] The number of effective curvature feature points in the extended verification range is counted as the matching density to evaluate the surface matching consistency between the target area and the design model in a larger range, avoiding ignoring the overall morphological differences due to successful local single-point matching.

[0085] Selecting a potential region based on the matching density of the target candidate region, exemplarily selecting the target candidate region with the highest matching density as the potential region;

[0086] See also Figure 5 , in the potential area, based on the matched effective curvature feature points, the corresponding edge feature points are located in the following way to generate the actual operation area:

[0087] Adjust the design model's posture based on its local coordinate system so that its principal inertial axis aligns with the overall orientation of the potential area, ensuring that the curvature of the two surfaces matches. For example, the long axis of the implant's bottom surface aligns with the anteroposterior curvature of the skull surface.

[0088] With the edge points of the design model as the center, within the potential area, spatial distance mapping is used to identify the edge points of the target installation area of ​​the target component in the surface model of the target object based on the curvature characteristics. Specifically, a dual-constraint search is performed within the potential area to map the edge points of the design model: target area points with highly similar curvature characteristics to the edge points of the design model are screened to ensure that the local geometric forms of the two are consistent and to avoid edge fitting deviations caused by surface differences; the search range is limited to a reasonable area around the edge points of the design model to ensure that the mapped edge points are close to the boundary of the design model and within the effective geometric range of the potential area.

[0089] Preferably, the specific steps of performing a dual-constraint search mapping the edge points of the design model in the potential region include:

[0090] According to the accuracy requirements of the target component, the similarity threshold of the curvature feature is set, and the spatial search threshold centered on the edge point of the design model is determined;

[0091] Obtain a set of edge points of the design model, each of which carries its local geometric features; at the same time, obtain point cloud data of the potential area, each of which contains three-dimensional coordinates and corresponding local geometric features.

[0092] For each edge point of the design model, a search is performed within the potential area with the point as the center and within the spatial search threshold range. The target area points within the spatial search threshold range are screened out, and the range of subsequent feature matching is narrowed to ensure that the mapped edge points are within the valid geometric range of the potential area.

[0093] For each of the selected target area points, the degree of similarity between their local geometric features and the edge points of the design model is calculated. If the degree of similarity between the local geometric features of the target area point and the edge points of the design model exceeds the similarity threshold, the point is marked as a candidate edge point to ensure that the surface morphology of the two points is consistent and to avoid edge alignment deviations caused by local geometric differences.

[0094] If the candidate edge point is not unique, the candidate edge point with the greatest similarity is selected as the mapping edge point for the design model edge point. After mapping is completed, check whether the surface orientation of the area where the mapping point is located matches the surface orientation of the corresponding edge point in the design model, and confirm that the mapping point is within the valid range of the potential area to ensure the feasibility of subsequent operations.

[0095] If a design model edge point cannot find a target area point that meets the geometric requirements within the preset spatial range, a virtual edge point is generated based on the position and morphological characteristics of adjacent edge points through interpolation to ensure the continuity of the edge contour. If multiple consecutive design model edge points cannot find a valid mapping point, it is determined that the current potential area does not match the edge morphology of the design model, and an abnormality warning is issued;

[0096] Establish a correspondence between the edge points of the design model and the edge points of the potential area mapping screened out. Connect all mapped edge points according to the edge order of the design model to form a closed outline of the actual operation area to generate the actual operation area. If there are gaps in the outline, interpolation is used to supplement the edge points to ensure the outline is complete, providing a complete boundary foundation for the subsequent mapping of functional feature points and delineation of the operation area.

[0097] The functional feature points of the design model are mapped to the actual operation area of ​​the potential area through a rigid transformation matrix, ensuring that these key functional points are located within the installation range defined by the edge contour.

[0098] Configure a deviation threshold, calculate the geometric centroid of the area enclosed by the closed edge contour, that is, the actual operation area, as the target installation center of the target component; calculate the global centroid of the entire target object surface model, which reflects the center of the guidance assistance device; calculate the spatial position deviation between the geometric centroid of the actual operation area and the global centroid of the target object surface model. If it is greater than the deviation threshold, it indicates that the target installation center of the target component deviates from the center position of the guidance assistance device, which is not conducive to actual operation, and the device position adjustment mechanism is triggered; otherwise, the intelligent planning operation is triggered;

[0099] If the device's position adjustment mechanism is triggered, the adjustment direction and movement distance of the guidance assist device are calculated based on the spatial position deviation between the geometric center of mass of the actual operating area and the global center of mass of the target object's surface model. Guidance instructions are then sent to the device via the light emitting assembly. The light emitting assembly uses beams or light spots of varying colors and flashing frequencies to visually indicate the device's movement direction. For example, a green beam points to the desired direction of movement, and the beam intensity corresponds to the movement distance. The assist device quickly and precisely moves toward the target installation center, ensuring that the adjusted device's reference center completely aligns with the target component's ideal installation center.

[0100] After the guidance assist device moves, it reacquires the target object's surface model and repositions the actual operating area and key feature points on the target object's surface model based on the adjustment direction and movement distance. The spatial position deviation between the geometric center of mass of the actual operating area and the global center of mass of the target object's surface model is recalculated to determine whether it exceeds the deviation threshold. If so, the device's position adjustment mechanism is triggered again; otherwise, intelligent planning is triggered. Through curvature matching, edge contour closure checks, functional point anatomical safety verification, and device posture consistency detection, the system effectively addresses global and local scale differences and ensures the safety of functional area positioning.

[0101] The intelligent planning module is used to plan the positions of the key positioning points of the target operating device on the surface model of the target object based on the functional feature point positions mapped to the actual operating area based on the design model when triggering the intelligent planning operation, combined with the positioning key points and geometric shape of the target operating device model. Through the preset spatial relationship between the positioning key points and the functional feature points, it indirectly ensures the accurate operation of the functional feature points by the target operating device, and avoids operation deviation caused by the device blocking the functional feature points.

[0102] See also Figure 6 Preferably, the specific steps of planning the positioning key point positions of the target operating device on the target object surface model include:

[0103] The functional feature points output by the dynamic registration module are the core objectives of the operation, while the key positioning points of the target operation device model are the benchmark for device control. Both types of data and the target object surface model must be unified into the same coordinate system to eliminate coordinate deviations between different models and ensure consistency in spatial position calculations.

[0104] Receive the coordinates of the functional feature points mapped in the actual operating area output by the dynamic registration module; access the target operating device model to obtain its 3D geometric shape parameters and positioning key points, including the end effector center point and visual positioning markers. Map the functional feature points, device positioning key points, and target object surface model to the same spatial coordinate system, establishing a unified benchmark for multi-source data and ensuring consistency in spatial relationship calculations.

[0105] Functional feature points and key points of device positioning must meet strict geometric relationships. A rigid spatial mapping relationship is established between the functional feature points of the design model mapped to the actual operating area and the key points of the target operating device positioning. Fixed offsets and posture constraints of the device positioning key points relative to the functional feature points are defined based on translation vectors and rotation matrices, forming a geometric alignment reference model between the device operating end and the functional feature points. This geometric alignment reference model reflects the preset spatial relationship between the core parts of the device operating end and the functional feature points, serving as the basis for positioning planning.

[0106] The operating posture of the target operating device is determined based on the local geometric features of the area where the functional feature points are located, including the surface normal vector of the target object and the principal inertial axes of the design model. For example, for the positioning of a cochlear implant and the cochlear electrode tunnel, to ensure that the implant, cochlear electrode, bone groove, and tunnel are fully aligned, the key positioning points of the device must be aligned along the direction of the normal vector of the point, so that the axis of the device operating end is consistent with the direction of the interface, meeting the posture requirements of anatomical or engineering assembly.

[0107] Based on the three-dimensional coordinates and operating posture of the functional feature points, combined with the rigid space mapping relationship, the target position of the key positioning point of the target operating device in the target object surface model coordinate system is calculated using a rigid transformation matrix. This target position satisfies the following requirements: the core part of the device operating end is strictly aligned with the functional feature points in spatial position and posture. Even if the functional feature points are obscured by the device, the core operating part still accurately acts on the target point.

[0108] The safety of the target position corresponding to the positioning key points is verified by combining the geometric features of the target object's surface model, including areas of high curvature and anatomical structure boundaries. The minimum distance between the device model and the high curvature areas and structural boundaries of the target object's surface is calculated. If this distance is greater than a preset safety threshold determined by the risk level of the operation scenario, the target position is deemed qualified. Otherwise, a position adjustment operation is triggered, and the position of the device's positioning key points is fine-tuned while maintaining the geometric alignment until the safety distance requirement is met. This avoids the risk of physical collision during operation and ensures the safety of the device's motion path. It is particularly suitable for scenarios with extremely high safety requirements, such as medical implants.

[0109] When the target position is qualified, if the functional feature points are not unique, a path planning algorithm is used to plan the operation sequence and collision-free motion path of the target operation device based on its operation priority, the device's kinematic constraints, and the geometric characteristics of the target object's surface. The operation priority is determined by the importance of the functional point and the operation process. The device's kinematic constraints include the range of motion of the target operation device's joints and its accessibility. The geometric characteristics of the target object's surface include curvature distribution and anatomical safety areas. Path planning algorithms include A* and Dijkstra algorithms. This reduces the time spent on repeated device positioning, improves operational efficiency, and ensures the device's kinematic feasibility, making it suitable for multi-step precision operation scenarios.

[0110] The execution control module is used to provide operation guidance, deviation monitoring and effect evaluation for the target operation equipment. By receiving the operation sequence and collision-free motion path of the target operation equipment generated by the intelligent planning module, it generates guidance instructions to provide operation guidance for the target operation equipment through the light source emission component; it collects the actual position and posture data of the positioning key points in real time, calculates the spatial deviation between the position and the target position, and verifies the geometric safety of the equipment and the surface model of the target object in real time to avoid collision risks and kinematic singularities; after the equipment operation is completed, it compares the functional feature point positions of the actual operation area with the design model to verify the operation accuracy of the functional points. If the requirements are not met, it generates correction instructions or triggers re-planning to achieve full-process closed-loop control of equipment operation guidance, deviation monitoring, and effect evaluation.

[0111] Preferably, the specific steps of performing operation guidance, deviation monitoring, and effect evaluation on the target operation device include:

[0112] See also Figure 7 , receive the target position, operation sequence and collision-free motion path corresponding to the positioning key points output by the intelligent planning module; convert the target position into motion instructions executable by the target operation device; eliminate the format differences between the planning data and the device control interface, provide a basis for subsequent visual guidance and precise motion control, and avoid operation failures due to incompatible instructions.

[0113] Based on motion instructions, the light source emission component indicates the target operating device's operating position, moving direction, and operation progress with light beams of different colors, flashing frequencies, and intensities; solving the problem of difficult manual judgment or insufficient accuracy of autonomous navigation of equipment in complex three-dimensional space.

[0114] The surface model of the target object is reconstructed in real time through the light source receiving component, and the actual position and posture data of the key positioning points are collected in real time through the built-in sensors of the target operating device, including the visual positioning system and inertial measurement unit; providing real-time data support for deviation calculation and collision detection, avoiding control delays caused by data lag.

[0115] Configure the accuracy threshold, including position accuracy and angle accuracy, calculate the spatial deviation between the actual position and the target position, including position offset and angle deviation, and compare them with the accuracy threshold. If the deviation is greater than the accuracy threshold, the light source emission component will issue a deviation warning. For example, the light source emission component will flash a yellow light beam as a warning, and the terminal interface will display the deviation value. Otherwise, no processing will be performed.

[0116] Configure a collision threshold based on the target object's surface risk area. Based on the bounding box of the target operating device model and the target object's surface model, use the bounding box hierarchical tree algorithm to calculate the distance between the target operating device model and the high curvature area and anatomical structure boundary of the target object's surface model in real time. If the distance is less than the collision threshold, a collision warning is issued; otherwise, no action is taken.

[0117] Configure the distance deviation threshold. After the target operation device completes the operation, the local area data where the functional feature points in the actual operation area are located is segmented based on the point cloud data of the real-time target object surface model through edge contour point cloud clustering, and the corresponding geometric features are extracted. The geometric features include normal vectors and curvature distribution. The distance deviation between the geometric features of the local area data where the functional feature points of the target component model are located is calculated. If the distance deviation is greater than the distance deviation threshold, it is determined that the target component model does not match the actual operation area. Otherwise, it is determined that the target component model matches the actual operation area.

[0118] If the target component model does not match the actual operation area, the functional feature points that cause the target component model to not match the actual operation area are identified, and the intelligent planning module is triggered to replan the operation path of the mismatched functional feature points to generate local correction instructions.

[0119] Example 2

[0120] This embodiment introduces a guidance auxiliary device for achieving high-precision cross-scale model registration and device operation guidance, solving the problem of operation offset in global and local scale differences and occlusion scenarios. Figure 1 This is a schematic diagram of the structure of a guidance assistance device disclosed in the present invention. The device primarily comprises a support structure, a light source transmitting and receiving structure, a data transmission structure, and an external processing structure. These structures work together to implement functions such as target object surface model construction, operation guidance, and data processing, providing support for precision operations. This enables high-precision, high-reliability equipment operation guidance and closed-loop control in precision medical implant scenarios.

[0121] The support structure, the physical foundation of the guidance assist device, defines the acquisition range of the target object's surface model through adjustable geometry and degrees of freedom, adapting to targets of varying sizes and curved shapes. This structure has demonstrated significant clinical value in cochlear implant surgery, reducing the risk of damage to surrounding tissue and optimizing surgical manpower. Traditional cochlear implant surgery often requires prolonged traction of nearby cortical tissue to fully expose the surgical field, which can lead to complications such as tissue ischemia and swelling. This support structure's flexible shape adjustment allows precise positioning of the light source and image acquisition equipment above the surgical field, creating a contactless visual field acquisition solution. Furthermore, its lightweight design and soft, scratch-resistant coating prevent pressure and scratches on the patient's skin, effectively reducing the risk of tissue damage and shortening the patient's postoperative recovery period. Previously, dedicated personnel were required to hold and adjust the image acquisition equipment to ensure a clear field of view. This support structure, with its highly automated adjustment capabilities and stable fixation, can replace manual operation. The surgical team does not need to arrange additional assistants to hold the equipment. One person can complete the position adjustment of the device, which reduces the number of auxiliary operators, optimizes the allocation of human resources in the operating room, and makes team collaboration more efficient and smooth.

[0122] The light source transmitting and receiving structure is used to realize the construction of the target object surface model, synchronously obtain the multi-dimensional features of the target object surface model, and provide visual operation guidance, indicating the target position, deviation warning and operation progress corresponding to the positioning key points of the target operation equipment. It includes a light source transmitting component, a light source receiving component and a guidance signal component;

[0123] Through the light emitting component, such as an infrared structured light projector, coded light patterns such as Gray code and sinusoidal stripes are projected onto the surface of the target object. The light receiving component is used to collect deformed light images modulated by the surface topography. Based on the stereo vision algorithm, three-dimensional point cloud data is calculated in real time, and multi-dimensional features such as surface texture, curvature, and reflectivity are simultaneously acquired. And through a detachable calibration plate or self-calibration algorithm, a spatial conversion relationship is established between the light emitting component, the light receiving component and the target operating device to ensure that the design model, the target object surface model, and the device model are uniformly mapped to the same spatial coordinate system. The guidance component provides visual operation guidance, pointing to the target position corresponding to the key positioning point of the device;

[0124] The data transmission structure is used to realize the mechanical fixation of the guidance auxiliary device, the calibration of the spatial coordinate system and the data interaction;

[0125] The external processing structure is used for real-time processing of three-dimensional point cloud data, performing dynamic registration, intelligent planning and safety control, performing dynamic registration through feature point extraction, potential area screening and hierarchical registration, performing intelligent planning through rigid transformation matrix calculation and path planning, and performing deviation monitoring and collision warning.

[0126] Working principle and effects:

[0127] The application accesses target components, designs installation areas and operation equipment models, uses structured light vision technology to collect multi-dimensional data of target object surfaces to construct high-precision surface models, provides data basis for cross-scale registration, extracts key features such as functional points, edge points and curvature points in the design model, divides the target object surface based on the size grid of the design model, screens potential areas with stable curvature through curvature matching and matching density statistics, ensures the geometric form of the core area to fit, and solves the registration problem caused by the difference between global and local scales.

[0128] In the potential area, the edge points of the design model are mapped to form a closed operation area through curvature similarity and spatial distance double constraints, the device position is adjusted combined with the centroid deviation analysis to realize accurate spatial alignment of the design model and the actual scene, and the registration accuracy is improved. After triggering the planning, the rigid spatial mapping relationship between the functional feature points and the device positioning key points is established, the device target position is calculated through the rigid transformation matrix, and the safety is verified, the collision-free path is generated combined with the path planning algorithm, the device operation end and the functional point space position and attitude are strictly aligned, and the limitation of relying on direct visibility in the visual guidance in the occluded scene is broken through.

[0129] During operation, the device position and progress are indicated through light source visualization, the positioning data are collected in real time for deviation and collision warning, and the operation risk is reduced; after operation, the local area of the functional point is segmented to extract geometric features, and the matching degree is determined by comparing with the design model, when the matching degree is not matched, only the abnormal point path is re-planned, a dynamic closed loop is formed, global reset is avoided, the operation accuracy and safety in the complex curved surface environment are significantly improved, and the precise medical implantation scene is effectively adapted.

[0130] The above only describes the preferred embodiments of the application, and the protection scope of the application is not limited to the above-mentioned embodiments, and any technical scheme falling within the idea of the application belongs to the protection scope of the application. It should be noted that for ordinary skilled persons in the art, some improvements and decorations without departing from the principle of the application can also be considered as the protection scope of the application.

Claims

1. A guidance assist device control system, characterized in that: include: Access the target component model, the design model of the target installation area, and the target operating equipment model, collect multi-dimensional data on the target object surface, and construct a target object surface model; Extracting key feature points from the design model, dividing the target object surface model into candidate regions based on the size of the design model bounding box, identifying the target candidate regions and extracting curvature feature points, screening valid curvature feature points by configuring a matching threshold, dividing an extended verification range, and selecting potential regions by counting the matching density within the extended verification range; Mapping the edge points of the design model within the potential area of ​​the target object surface model based on the similarity of curvature features and spatial distance constraints to form a closed contour as the actual operation area; spatially aligning the target object surface model with the design model based on the deviation between the geometric center of mass of the actual operation area and the global center of mass of the target object surface model, and determining whether to trigger an intelligent planning operation; When triggering the intelligent planning operation, based on the functional feature point positions mapped from the design model to the actual operation area, combined with the positioning key points and geometric shape of the target operation device model, the positioning key points of the target operation device on the target object surface model are planned to generate guidance instructions; The light source emission component provides guidance for target operating equipment operation, and real-time data of key positioning points is collected for deviation and collision warning. After the operation is completed, the geometric features of the local area where the functional feature points are located are used to determine whether the target component model matches the actual operating area. If they do not match, local correction instructions are generated.

2. A guidance assistance device control system according to claim 1, characterized in that: The step of spatially aligning the target object surface model with the design model comprises: Performing coordinate normalization on the target installation area design model, constructing an axis-aligned bounding box of the design model and marking the geometric center of the design model, determining the principal inertia axes of the design model through principal component analysis with the geometric center as the origin, and constructing a local coordinate system; Extracting key feature points from the design model, wherein the key feature points include functional feature points, edge feature points, and curvature feature points; Based on the size of the design model's bounding box, the target object's surface model is meshed and matched with curvature feature points to identify valid curvature feature points and divide the extended verification range to select potential areas containing the design model in the target object's surface model. In the potential area, based on the matched effective curvature feature points and the similarity of curvature features and spatial distance constraints, the edge points of the mapped design model are searched to generate the actual operation area, and the functional feature points of the design model are mapped to the actual operation area of ​​the potential area.

3. A guidance assistance device control system according to claim 2, characterized in that: The step of spatially aligning the target object surface model with the design model further comprises: Configure the deviation threshold, calculate the geometric center of mass of the actual operation area as the target installation center, and calculate the global center of mass of the target object surface model as the center of the guidance auxiliary device; Calculating the spatial position deviation between the target installation center and the center of the guidance auxiliary device. If the spatial position deviation is greater than the deviation threshold, triggering the device position adjustment mechanism; otherwise, triggering the intelligent planning operation; The device position adjustment mechanism includes: Calculate the adjustment direction and movement distance of the guidance auxiliary device based on the spatial position deviation, and indicate the movement direction and movement distance of the guidance auxiliary device through the light source emission component, so that the center of the guidance auxiliary device moves to the target installation center; After guiding the auxiliary device to move, the surface model of the target object is reacquired, and the actual operation area and key feature points are relocated based on the adjustment direction and movement distance.

4. A guidance assistance device control system according to claim 2, characterized in that: The step of selecting a potential area containing a design model in the target object surface model comprises: Perform meshing on the surface model of the target object, set the initial search range based on the size of the design model bounding box, and divide the surface model of the target object into multiple candidate areas; Traverse the candidate area, calculate the surface point curvature value through the local surface fitting algorithm, and identify the target candidate area; Extract the curvature feature points of the target candidate area, calculate the similarity of the curvature feature points, match them with the curvature feature points of the design model, and identify the valid curvature feature points; An extended verification range is defined with the effective curvature feature point as the center and the design model bounding box size as the radius. All curvature feature points within the extended verification range are extracted and effective curvature feature points are screened. The number of effective curvature feature points is counted to form a matching density. According to the matching density of the target candidate area, the target candidate area is selected as the potential area.

5. A guidance assistance device control system according to claim 2, characterized in that: The step of generating the actual operation area includes: Set the similarity threshold and spatial search threshold to obtain the edge point set of the design model and its local geometric features, obtain the point cloud data of the potential area, and its three-dimensional coordinates and local geometric features; For each edge point of the design model, the target area points within the spatial search threshold range are screened out with the edge point as the center in the potential area; Calculate the similarity between the local geometric features of the target area points and the edge points of the design model. If the similarity is greater than the similarity threshold, mark it as a candidate edge point, and select the mapping edge point of the design model edge point based on the similarity of the candidate edge point; Establish a correspondence between the edge points of the design model and the mapped edge points of the screened potential area, and connect the mapped edge points according to the edge order of the design model to form a closed outline of the actual operation area.

6. A guidance assistance device control system according to claim 1, characterized in that: The step of planning the positioning key point positions of the target operating device on the target object surface model includes: Receive the coordinates of the functional feature points in the actual operation area, access the target operation device model and obtain its three-dimensional geometric shape parameters and positioning key points; Map functional feature points, equipment positioning key points and target object surface models to the same spatial coordinate system; A rigid spatial mapping relationship between functional feature points and device positioning key points is established. Based on the translation vector and rotation matrix, the fixed offset and posture constraints of the device positioning key points relative to the functional feature points are defined to form a geometric alignment reference model and determine the operating posture of the target operating device.

7. A guidance assistance device control system according to claim 6, characterized in that: The step of planning the positioning key point positions of the target operation device on the target object surface model also includes: Based on the three-dimensional coordinates of the functional feature points, the operating posture and the rigid space mapping relationship, the target position of the device positioning key point in the target object surface model coordinate system is calculated through the rigid transformation matrix, so that the device operating end is aligned with the spatial position and posture of the functional feature points; Combined with the geometric features of the target object surface model, verify the safety of the target position corresponding to the positioning key points and determine whether the target position is qualified; When the target position is qualified, the path planning algorithm is used to plan the operation sequence of the functional feature points and the collision-free motion path according to the operation priority of the functional feature points, the kinematic constraints of the equipment and the surface geometric characteristics of the target object.

8. A guidance assistance device control system according to claim 7, characterized in that: The step of collecting and positioning key point data for deviation and collision warning includes: Receive the target position, operation sequence and collision-free motion path corresponding to the positioning key point, and convert the target position into motion instructions executable by the target operation device; Based on the motion instruction, the light source emission component indicates the operation position, movement direction and operation progress of the target operation device; The surface model of the target object is reconstructed in real time through the light source receiving component, and the actual position and posture data of the positioning key points are collected in real time using the built-in sensors of the target operation device; Configure the accuracy threshold, calculate the spatial deviation between the actual position and the target position, and compare it with the accuracy threshold. If it is greater than the accuracy threshold, the light source emission component will issue a deviation warning; otherwise, no action will be taken. Configure the collision threshold. Based on the bounding box of the target operation device model and the surface model of the target object, the bounding box hierarchical tree algorithm is used to calculate in real time the distance between the target operation device model and the high curvature area and anatomical structure boundary of the target object surface model. If the distance is less than the collision threshold, a collision warning is issued through the light source emission component. Otherwise, no operation is performed.

9. A guidance assistance device control system according to claim 1, characterized in that: The step of determining whether the target component model matches the actual operation area includes: Configure a distance deviation threshold. After the target operating device completes the operation, segment the local area data where the functional feature points of the actual operating area are located and extract the geometric features. Calculate the distance deviation between the local area data and the geometric features of the functional feature points corresponding to the target component model. If the distance deviation is greater than the distance deviation threshold, it is considered a mismatch; otherwise, it is considered a match. If a mismatch is determined, the functional feature points that cause the mismatch are identified, the operation path is replanned, and local correction instructions are generated.

10. A guidance assistance device, comprising a guidance assistance device control system according to any one of claims 1 to 9, characterized in that: It also includes: a supporting structure, a light source transmitting and receiving structure, a data transmission structure and an external processing structure; The support structure serves as the physical basis of the guidance assist device, defining the acquisition range of the target object surface model through adjustable geometry and degrees of freedom; The light source transmitting and receiving structure is used to realize the construction of the target object surface model, synchronously obtain the multi-dimensional features of the target object surface model, and provide visual operation guidance to indicate the target position, deviation warning and operation progress corresponding to the positioning key points of the target operation device; The data transmission structure is used to achieve mechanical fixation of the guidance auxiliary device, spatial coordinate system calibration and data interaction; The external processing structure is used to process three-dimensional point cloud data in real time, and perform dynamic alignment, intelligent planning and safety control.

Images