Target area operation method and device based on perceptual image, equipment and medium

Abstract

The invention relates to the technical field of medical robots and image guidance, and discloses a target area operation method, device, equipment and medium based on a perceptual image, and the method comprises the steps: obtaining the perceptual image data of a target area, recognizing the boundary of the target area based on the perceptual image data, and planning and generating an operation path according to the boundary of the target area, and controlling the execution component to execute operation along the operation path, acquiring a contact feedback signal of the operation tool in the execution process, adjusting the operation state of the execution component, detecting interference information of the execution component in real time, dynamically correcting the operation path according to a detection result, and realizing autonomous perception and operation control of the target area. According to the method, a closed-loop autonomous control mechanism from sensing to operation is constructed by fusing sensing image recognition, path planning, execution control and real-time feedback correction processes, and accurate recognition and path adaptive adjustment of a target area in a dynamic environment are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical robots and image-guided technology, and in particular to a target region operation method and device based on perception images, equipment and a medium. BACKGROUND

[0002] In current tumor resection surgery, ultrasound guidance and robot-assisted surgery technology has been gradually applied in clinical practice, but there are still significant deficiencies in key links such as target region identification, path planning and execution control. The existing ultrasound guidance system usually relies on the experience of the operator to identify and label the image, and the result is easily affected by the subjective judgment of the operator, resulting in insufficient accuracy and consistency of tumor boundary positioning. Especially in soft tissue organs such as liver or lung, due to the influence of tissue deformation, blood flow interference and image noise, there is often a deviation between real-time images and actual anatomical structures, so that the boundary determination based on a single ultrasound image cannot meet the requirements of high-precision resection.

[0003] In addition, although the existing robot-assisted surgery has high motion control accuracy, it generally lacks autonomous perception and decision-making capabilities for the surgical area. In actual operation, the path executed by the robot usually relies on pre-set models or manual planning, and lacks a dynamic updating mechanism based on real-time image data. When the tumor position or tissue morphology changes during the operation, the robot cannot automatically adjust the operation path according to the new image, thereby affecting the integrity and safety of the resection.

[0004] At the same time, the existing force feedback and path correction mechanism is a separate control loop, which cannot form a closed-loop control structure of perception-decision-execution. During the execution of the operation, the robot often cannot integrate the contact feedback signal of the operation tool and the environmental interference information, resulting in a lag in response to abnormal contact, changes in tissue resistance or obstacle interference, and prone to problems such as improper operation force or path deviation. SUMMARY

[0005] The main purpose of the present application is to provide a target region operation method, device, equipment and storage medium based on perception images, aiming to solve the technical problem that the existing technology lacks automatic identification of the target region boundary and autonomous correction of the operation path based on real-time perception images, and cannot realize integrated autonomous control of dynamic perception, feedback adjustment and path update during execution.

[0006] To achieve the above-mentioned purpose, the present application provides a target region operation method based on perception images, comprising: acquiring perception image data of a target region; identifying a target region boundary of the target region based on the perception image data; generating an operation path of the target region based on the target region boundary planning; controlling an execution component to perform an operation along the operation path; acquiring a contact feedback signal of an operation tool during the execution component performing the operation, and adjusting an operation state of the execution component according to the contact feedback signal; detecting interference information of the execution component in real time during the execution component performing the operation, and dynamically correcting the operation path according to the interference information.

[0007] Further, to achieve the above object, the present application provides a target region operation device based on perception image, comprising: a perception image acquisition module, configured to acquire perception image data of a target region; a region identification and segmentation module, configured to identify a target region boundary of the target region based on the perception image data; a path planning module, configured to plan and generate an operation path of the target region based on the target region boundary; an execution control module, configured to control an execution component to perform an operation along the operation path; a force feedback adjustment module, configured to acquire a contact feedback signal of an operation tool during the execution component performing the operation, and adjust an operation state of the execution component according to the contact feedback signal; an interference detection and path correction module, configured to detect interference information of the execution component in real time during the execution component performing the operation, and dynamically correct the operation path according to the interference information.

[0008] Further, to achieve the above object, the present application further provides a computer device, comprising a memory, a processor, and a target region operation program based on perception image stored in the memory and executable on the processor, wherein the target region operation program based on perception image, when executed by the processor, implements the steps of the target region operation method based on perception image.

[0009] Further, to achieve the above object, the present application further provides a computer readable storage medium, wherein the storage medium stores a target region operation program based on perception image, and the target region operation program based on perception image, when executed by a processor, implements the steps of the target region operation method based on perception image.

[0010] Beneficial effects: The application relates to the field of medical robots and image guidance technology, and discloses a target region operation method and device based on perception images, equipment and a medium, which comprises the following steps: acquiring perception image data of a target region, identifying the boundary of the target region based on the perception image data, generating an operation path according to the target region boundary, controlling an execution component to execute an operation along the operation path, acquiring a contact feedback signal of an operation tool during the execution process and adjusting the operation state of the execution component, simultaneously detecting interference information of the execution component in real time and dynamically correcting the operation path according to the detection result, and realizing autonomous perception and operation control of the target region. The application fuses the processes of perception image identification, path planning, execution control and real-time feedback correction, constructs a closed-loop autonomous control mechanism from perception to operation, realizes accurate identification of the target region and adaptive adjustment of the path in a dynamic environment. By acquiring the contact feedback signal and the interference information in real time, the system can automatically optimize the operation state and the path trajectory during the execution process, thereby improving the operation precision and safety, reducing errors caused by manual intervention, and improving the reliability and stability of the overall autonomous operation. BRIEF DESCRIPTION OF DRAWINGS

[0011] The application will be further described below in combination with the drawings and embodiments, wherein: Figure 1 It is an application environment schematic diagram of the target region operation method based on perception images in an embodiment of the application. Figure 2 It is a flowchart of the target region operation method based on perception images in an embodiment of the application. Figure 3 It is a functional module schematic diagram of the target region operation device based on perception images in a preferred embodiment of the application. Figure 4 It is a structural schematic diagram of a computer device in an embodiment of the application. Figure 5 It is another structural schematic diagram of a computer device in an embodiment of the application. DETAILED DESCRIPTION

[0012] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0013] The target region operation method based on perception images provided by the embodiments of the application can be applied to, for example, Figure 1In an application environment of the application, the client communicates with the server through a network. The server can obtain perception image data of a target region through the client, identify a target region boundary based on the perception image data, plan an operation path according to the target region boundary, control an execution component to execute an operation along the operation path, obtain a contact feedback signal of an operation tool during the execution process and adjust an operation state of the execution component, simultaneously, detect interference information of the execution component in real time and dynamically correct the operation path according to the detection result, so as to realize autonomous perception and operation control of the target region. The application fuses the processes of perception image recognition, path planning, execution control and real-time feedback correction, constructs a closed-loop autonomous control mechanism from perception to operation, and realizes accurate identification and adaptive path adjustment of the target region in a dynamic environment. By obtaining the contact feedback signal and the interference information in real time, the system can automatically optimize the operation state and the path trajectory during the execution process, so as to improve the operation precision and safety, reduce errors caused by manual intervention, and improve the reliability and stability of the overall autonomous operation. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be realized by an independent server or a server cluster composed of multiple servers. The application will be described in detail through specific embodiments.

[0014] Please refer to Figure 2 , Figure 2 The flowchart of an embodiment of the target region operation method based on perception image provided by the application is shown. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0015] As Figure 2 shown, the target region operation method based on perception image provided by the application includes the following steps: S10, obtaining perception image data of a target region; In this embodiment, obtaining perception image data of a target region is the starting link of information acquisition and environment modeling in the entire autonomous operation system, and the fundamental purpose is to provide high-quality image information with spatial and temporal continuity for subsequent target recognition and path planning. The acquisition of perception image data depends on the cooperative work of the ultrasonic imaging device and the data acquisition unit, and a comprehensive perception of the target region is formed through multi-angle and multi-direction scanning actions of the probe. During the scanning process, the imaging probe emits high-frequency sound waves and receives the echo signals reflected by the tissue, and an initial image matrix is formed through the calculation of wave speed, amplitude and phase difference. In order to reduce the influence of tissue motion, probe jitter and signal reflection interference on data stability, the acquisition unit records the time stamp of each frame of image at a millisecond level of precision through the built-in clock synchronization mechanism, so as to realize time sequence reconstruction of the subsequent image sequence.

[0016] The initial perception image data is collected and then enters the preprocessing procedure. The noise filtering process suppresses high-frequency random noise and low-frequency background drift by combining band-pass filtering with spatial convolution. The spatial filtering module uses Gaussian kernel smoothing in the two-dimensional imaging layer to eliminate edge artifacts caused by tissue interfaces; the time filtering module introduces sliding window averaging in the inter-frame sequence to eliminate the inter-frame flicker effect caused by probe micro-motion. The filtered image data is transmitted to a distributed computing unit, which is usually composed of multiple parallel GPU nodes interconnected by high-speed buses to realize fast distribution and parallel computing of image data.

[0017] The spatial coordinate registration process is based on the spatial pose parameters output by the probe position sensor or the mechanical arm end encoder. By calculating the affine transformation matrix between the image coordinates and the world coordinates, the spatial alignment between different scanning frames is realized. The time synchronization process is based on a unified system time reference, which interpolates and matches the image frames and sensor data to form a time-continuous data stream. The image sequence after spatial registration and time synchronization is further used to establish a spatial mapping model of the target region. This model corresponds the ultrasound image intensity distribution to the three-dimensional structure of the target anatomical region through a mapping function, thereby generating calibrated perception image data.

[0018] The quality assessment of the calibrated image adopts a multi-index joint judgment mechanism. Structural similarity (SSIM) is used to evaluate image clarity and contrast, signal-to-noise ratio (SNR) reflects signal stability, and edge preservation index (EPI) measures structural continuity. When any index is lower than the system set threshold, the corresponding image frame is automatically excluded. The remaining frames are reconstructed by inter-frame interpolation to generate continuous, stable, and final perception image data that can be used for target identification.

[0019] In specific applications, different types of ultrasound probes and acquisition methods can be selected. Linear array probes can be used for local high-resolution scanning, and convex array probes can be used for wide-area detection of deeper tissues. In dynamic tissue environments, the probe pose can be automatically adjusted in combination with a mechanical arm control module. The pose control algorithm maintains the relative constancy of the sound beam incidence angle and the target surface normal to reduce sound reflection loss.

[0020] The implementation of the distributed computing unit can rely on a hybrid architecture of field programmable logic array (FPGA) and graphics processing unit (GPU), where FPGA is used for data preprocessing and transmission scheduling, and GPU is used for image enhancement and registration calculation. The spatial registration method can use an affine model based on feature point matching or a dense transformation model based on optical flow field estimation. The choice of different algorithms depends on the image texture features and tissue motion characteristics. The implementation of time synchronization can be based on the system bus time signal or external high-precision synchronization pulse signal to ensure the synchronization consistency between multiple data sources.

[0021] The embodiment forms high-quality image data with spatial continuity and temporal consistency through the above acquisition process of the perception image data, and provides stable and accurate input for subsequent target recognition, boundary extraction and path planning. The process eliminates the influence of artificial detection errors and environmental noise, enhances the usability and reliability of the image data, and thus lays a foundation for the autonomous operating system to realize accurate identification and dynamic control of the target region.

[0022] S20, identifying a target region boundary of the target region based on the perception image data; In the embodiment, the process of identifying the target region boundary based on the perception image data is a core link for the autonomous operating system to realize accurate spatial understanding and boundary definition of the target region. The target is to automatically extract the difference features of the target region and the surrounding tissues from the multi-dimensional image data, and form a geometric expression that can be used for subsequent path planning at the calculation level. The input of the image data is usually a perception image sequence processed by spatial registration and time synchronization, wherein each image corresponds to a specific physical coordinate and time stamp. The identification process first needs to load a trained image segmentation model. The model learns the gray distribution, texture features and spatial continuity rules of different tissues through a large amount of labeled data.

[0023] In the image recognition stage, the calculation unit inputs the perception image data into the segmentation model to perform forward inference, and outputs a probability matrix of each pixel point belonging to different tissue categories. Through thresholding operation, the probability matrix is converted into a preliminary binary segmentation result, wherein the foreground region corresponds to the target region candidate area. In order to eliminate segmentation noise and broken edges, the system uses morphological operation for connectivity repair in the spatial domain, including the alternative application of erosion and dilation processing to strengthen the continuity of the region boundary. In the time domain, for the small differences between multiple images, the boundary temporal consistency is realized through the inter-frame fusion mechanism to avoid boundary jumping caused by transient noise.

[0024] On the basis of the binary segmentation result, the system extracts the pixel set of the target region through a contour extraction algorithm, and generates a boundary point set according to the pixel connected region. The boundary point set is resampled by interpolation to maintain the uniformity of the spatial distribution, so as to facilitate the use of the subsequent path generation module. In a complex tissue environment, in order to avoid over-fitting or serration of the boundary, the calculation unit introduces a smoothing optimization strategy, which usually uses a curvature-constrained Bezier curve or B-spline function fitting to convert the discrete boundary points into a smooth and continuous boundary curve.

[0025] If there are multiple targets or blurred areas in the image, a region confidence mechanism can be introduced to calculate the boundary confidence distribution by statistical region gray mean, texture complexity and shape compactness, and to re-segment or reconstruct the boundary segments with low confidence. The final output of the target region boundary is stored in the form of a multi-dimensional array, containing spatial coordinate sequences and geometric parameters, providing accurate input for the path planning module.

[0026] Under different image types, the identification process can be adjusted accordingly. In ultrasound images, a convolutional network based on the U-Net architecture can be used to achieve pixel-level segmentation, and attention mechanisms can be used to enhance the response to low-contrast areas. In CT or MRI image environments, a three-dimensional convolution-based segmentation model can be used to utilize the context information between spatial voxels. For real-time scenarios, lightweight network structures and GPU-accelerated inference modules can be used to ensure that the boundary identification delay is controlled within milliseconds.

[0027] This embodiment realizes high-precision positioning of the target region without human intervention through automatic identification and boundary extraction of perception image data. This process effectively solves the problems of boundary ambiguity and unstable identification in traditional manual annotation, enabling the operating system to perform autonomous path planning based on reliable spatial information. Automatic boundary generation not only improves identification accuracy but also significantly improves overall processing speed, providing a stable data foundation for subsequent path generation and dynamic control.

[0028] S30, planning an operation path of the target region based on the target region boundary; In this embodiment, the process of planning an operation path based on the target region boundary is the core conversion link from spatial understanding to motion decision. The fundamental purpose of this process is to convert the geometric information of the target region boundary into trajectory data that can be directly utilized by the executing components, thereby realizing path control for autonomous operation. The input of the planning process is the spatial coordinate point set of the target region boundary, and the output is a path sequence with continuity, executability and safety. First, the boundary point set is discretized to ensure uniform point spacing in spatial distribution. The significance of discretization is to provide a discrete state space for path search algorithms, enabling the system to evaluate the feasibility and merits of different path schemes within a computable range.

[0029] The path planning module calculates the start and end positions based on the boundary point set. The start point usually corresponds to the starting point at the entrance of the operating area or the safety boundary, while the end point is determined by the closure of the target area boundary and the operating requirements. At this time, the system generates multiple candidate paths between the start and end points of the path through a heuristic search algorithm such as A or D algorithm. The heuristic search considers path length, boundary distance, and spatial obstacle constraints when evaluating path cost, thereby ensuring the shortest path while avoiding potential collision risks. After generating the candidate paths, the system evaluates the length, curvature, and spatial safety distance of each path, and calculates the safety index and execution cost function of the path.

[0030] The evaluation results form a path advantage and disadvantage score table through a weighted comprehensive method. The path with the highest score is selected as the preliminary operation path. The preliminary operation path is smoothed and optimized to eliminate angle mutations between nodes. Least squares fitting or spline interpolation-based curve generation methods are often used for smoothing optimization to obtain a continuous curve trajectory that meets the robot execution constraints.

[0031] In this process, the path planning also needs to consider the kinematic model of the execution component and the environmental constraints. The system adjusts the path curvature and node distance based on the working radius, joint angular velocity, and operating force limit of the execution component to ensure that the path does not exceed the mechanical limit during dynamic execution. To improve path adaptability, a gradient descent-based trajectory optimization mechanism can be introduced to fine-tune the path in local areas, allowing it to maintain safe execution even in the presence of spatial obstacle changes or tissue deformation. The final output operation path consists of a sequence of path nodes and corresponding velocity and attitude parameters, providing real-time driving instructions for the execution control module.

[0032] In different application scenarios, the path planning module can use different algorithm systems. Global planning methods based on graph search, such as A*, D, or RRT, can be used for path generation in structured spaces; local planning methods based on cost function optimization, such as dynamic window method (DWA) or gradient field method, can be used for real-time dynamic correction. For soft tissue operation tasks, a tissue mechanics model can be introduced during the planning phase to associate path cost with local tissue deformation to reduce the damage of execution force to the tissue.

[0033] In ultrasound-guided operation scenarios, the boundary point set is derived from image segmentation results, and the planning module maps it to the mechanical coordinate system through coordinate transformation. Different execution tasks can use different path structures, such as ring or spiral trajectories for uniform cutting in tissue cutting, and radial paths for energy concentration in injection or ablation operations. During path optimization, the smoothing coefficient can be adjusted according to the dynamic response parameters of the execution component to ensure that the trajectory meets the path continuity and ensures stable dynamic response.

[0034] For complex target regions, a multi-layer planning mechanism can be adopted. The first layer performs rough global planning to determine a safe approach path, and the second layer performs fine path adjustment based on local boundary features. This hierarchical strategy allows the system to balance safety and accuracy in complex structures.

[0035] This embodiment realizes the closed-loop connection from spatial information at the perception layer to motion control at the execution layer through the path planning process based on the target region boundary. This path generation mechanism enables the operation system to automatically plan executable trajectories based on target boundaries, significantly improving operation accuracy and autonomy. By combining heuristic search with smoothing optimization, the path has optimality and executability, reducing the deviation caused by manual path design and improving the stability and efficiency of autonomous control.

[0036] S40, controlling the execution component to execute the operation along the operation path; In this embodiment, controlling the execution component to execute the operation along the operation path is a key link from static planning to dynamic execution, and its task is to convert the operation path generated by planning into a motion instruction sequence that can be driven by the execution mechanism, to realize accurate operation on the target region. The input of the execution control module is the node sequence of the operation path and its corresponding pose and speed parameters, and the output is the control signal for driving the execution component. The execution control process first converts the spatial coordinates of the path nodes into motion reference points in the execution component coordinate system through the coordinate transformation module, to ensure accurate alignment of the trajectory in the execution space. The execution component usually includes a mechanical arm or a flexible execution unit with multiple degrees of freedom, whose position and attitude are jointly controlled by angle encoders, position sensors and joint drives.

[0037] During the trajectory execution phase, the control module generates interpolation points according to the time sequence distribution of the path nodes, calculates the speed and acceleration distribution of each interpolation point through the motion planner, and forms a time-continuous trajectory curve. Trajectory interpolation often uses quintic polynomial or spline interpolation to ensure the continuity of the trajectory in speed and acceleration, thereby avoiding vibration or overshoot of the execution component at path turning points. The execution control system reads feedback signals in real time, including position feedback, speed feedback and operation force feedback of the execution component, and differentiates them with the instruction value to obtain the motion deviation.

[0038] The deviation data is input into a closed-loop controller for correction. The controller adopts a proportional-integral-derivative (PID) or model predictive control (MPC) structure to correct the execution trajectory by adjusting the output driving signal, so that the actual motion trajectory is consistent with the target path. During execution, the system performs feedback updates at a millisecond level, achieving real-time correction and error compensation of the motion state. To prevent sudden disturbances in the dynamic environment, such as tissue surface deformation or mechanical arm end slip, the system introduces an adaptive gain adjustment mechanism to dynamically adjust the control gain based on the current error rate, thereby maintaining the stability and precision of the motion.

[0039] The present embodiment realizes continuous closed-loop from path planning to physical operation through precise control of the execution component along the operation path. The control mechanism combines real-time feedback correction and dynamic gain adjustment to maintain trajectory accuracy and motion stability in complex environments. This process significantly improves the repeatability and safety of the execution action, enabling the autonomous operation system to perform tasks stably in a multi-interference environment and effectively reducing the need for human intervention.

[0040] S50, during the execution of the operation by the execution component, obtaining a contact feedback signal of the operation tool, and adjusting an operation state of the execution component according to the contact feedback signal; In the present embodiment, during the execution of the operation by the execution component, the contact feedback signal of the operation tool is obtained and the operation state of the execution component is adjusted based on the signal, which is an important mechanism for realizing real-time interaction and dynamic control of the intelligent autonomous operation system. The core is to realize adaptive adjustment of operation force, motion speed and execution direction through perception and analysis of contact behavior, so that the operation process can remain stable and safe according to the changes of the target environment.

[0041] The acquisition of the contact feedback signal depends on a high-sensitivity force sensor array installed at the end of the operation tool or the execution component. These sensors can collect force, pressure distribution and shear components in three-dimensional directions at the moment of contact. The raw signals output by the sensors are filtered and amplified through a signal conditioning circuit, and then transmitted to a data acquisition module. The acquisition module performs synchronous sampling and digital filtering on the signals to eliminate interference caused by environmental noise and mechanical vibration, thereby obtaining high-fidelity contact feedback signals.

[0042] In the calculation and processing stage of the feedback signal, the system first extracts the modulus and direction components of the contact force, and calculates the change rate of the contact force based on time series analysis to judge the physical properties of the target surface and the contact stability. When the contact force is above or below the preset safety threshold range, the system generates a deviation value. The deviation value reflects the difference between the current operation state and the ideal operation force, and its calculation usually uses a real-time sliding window mechanism to reduce the impact of transient fluctuations on the results.

[0043] After receiving the deviation value, the control module calculates the output power adjustment amount of the execution component according to a preset power adjustment model. The power adjustment model comprehensively considers the current force feedback, the driving characteristics of the execution component, and the operation task type. For example, in a cutting or probing task, when an excessively high contact force is detected, the system automatically reduces the driving force or slows down the movement speed; when an insufficient contact force is detected, the system increases the output power to ensure effective contact. The adjusted operation parameters are applied to the execution component in real time through the driving control interface, so that the movement state of the execution component is dynamically corrected.

[0044] At the same time, the system continuously records the time sequence data of the adjustment parameters and the corresponding feedback signals to form an operation history data set. This data set is used for subsequent adaptive optimization of control parameters, that is, by learning the relationship between power adjustment and contact response in historical operations, the control model is continuously updated to make the response of future operations smoother and more efficient.

[0045] Different types of operation tasks can use different contact feedback mechanisms. In the flexible tissue operation scene, a piezoelectric film-based force sensor can be used to achieve high-sensitivity detection of small force changes; in the rigid material processing scene, a strain gauge or fiber Bragg grating sensor can be used to obtain high-intensity contact force data. The signal processing link can select a filtering algorithm according to the task characteristics, for example, Kalman filtering is used in low-frequency contact fluctuations, and band-pass filtering or adaptive filtering methods are used in high-frequency disturbance environments.

[0046] For high-dynamic-response tasks such as real-time minimally invasive resection, a fast-response controller can be embedded in the contact feedback loop to achieve millisecond-level force compensation adjustment. The controller dynamically adjusts the driving current by comparing the contact force deviation with the threshold difference in real time, and realizes immediate correction of the operation state. In a multi-dimensional operation scene, a six-dimensional moment sensor structure can be used to capture the joint distribution of force and torque, so that the system can adjust the operation direction and contact depth simultaneously.

[0047] In the case of non-uniform contact characteristics between the execution component and the target surface, a contact model can be established to predict the force response of different regions. The system adaptively adjusts the contact gain according to the model to keep the operation state consistent on different material surfaces. In addition, if the task involves continuous multi-point contact operation, a force-constrained trajectory tracking mechanism can be introduced in the path execution to dynamically match the contact force curve of the operation process with the planned value.

[0048] The embodiment can realize dynamic power adjustment and motion state correction during operation by real-time acquisition and analysis of contact feedback signals, significantly improving the adaptability and safety of the execution process. This mechanism makes the operation system no longer rely on fixed force threshold or manual judgment, but can make flexible adjustment based on real-time perception data, thereby avoiding failure caused by excessive operation or insufficient force. By continuously recording and learning feedback signals, the response behavior of the system is gradually optimized, realizing long-term operation self-learning and adaptive control.

[0049] S60, during the execution of the operation process by the execution component, the interference information of the execution component is detected in real time, and the operation path is dynamically corrected according to the interference information.

[0050] In the embodiment, during the execution of the operation by the execution component, the interference information of the execution component is detected in real time and the operation path is dynamically corrected according to the detection result. Through the cooperative operation of the sensor array, the spatial positioning unit and the path correction module, the execution system can automatically identify potential obstacles and path deviations in a complex and dynamic working environment, and maintain the continuity and safety of the operation process.

[0051] When the execution component is working, its surrounding space is monitored in real time by a plurality of distance sensors and a depth perception module. The distance sensor array can include an ultrasonic ranging unit, a laser ranging module or a depth camera based on structured light, for collecting distance data between the execution component and obstacles in the environment. The data of the sensor is transmitted to the data fusion module at a high frequency, and the module performs time synchronization and coordinate alignment processing on the spatial data from different sources to ensure that all ranging information is in the same reference system.

[0052] The fused obstacle distance data and the motion state data (such as attitude angle, speed, position) of the execution component are jointly calculated to obtain the relative angle between the obstacle and the motion direction of the execution component and the dynamic distance change rate. When the distance of the obstacle is detected to be lower than the safety threshold or the relative motion direction tends to collide, the system automatically calculates the interference level. The interference level is comprehensively evaluated according to the distance, relative angle and speed vector change of the obstacle, and is usually divided into three levels of mild, moderate and severe interference.

[0053] The path correction module generates a path correction vector according to the interference level. The path correction vector reflects the spatial direction and correction amplitude that should be offset at the current position, and its generation can be based on gradient field modeling or dynamic obstacle avoidance algorithm. For example, the system can use the artificial potential field method to construct the repulsive potential field of the obstacle, and calculate the minimum cost path direction, so as to guide the execution component to avoid collision while maintaining geometric consistency with the original path as much as possible. After the generated correction vector is processed by path interpolation, a local adjustment trajectory is formed. The trajectory adjustment is injected into the operation path in real time by the controller to dynamically update the trajectory parameters, realizing seamless correction of the operation path.

[0054] At the same time, the system will continuously record the interference event data, including the type of obstacle, relative position and correction amplitude, for subsequent path learning and risk prediction. If the same area triggers interference correction multiple times, the system will mark this area as a high-risk area and increase the obstacle avoidance weight in subsequent path planning, thereby realizing adaptive optimization of the environment model.

[0055] For different application environments, different interference detection and path correction mechanisms can be used. In precise medical operations, a high-precision depth perception module based on structured light can be used to realize millimeter-level obstacle identification; in industrial operation scenarios, laser radar or ultrasonic ranging array can be used to realize large-range space monitoring. For dynamic obstacle environment, the motion trend of the obstacle can be calculated in combination with inertial measurement unit data (IMU), and the obstacle trajectory can be predicted using extended Kalman filter (EKF), so as to plan the path offset in advance.

[0056] In the implementation of the path correction algorithm, dynamic obstacle avoidance algorithm based on local planning (such as DWA) or path adjustment strategy based on reinforcement learning can be used. The dynamic obstacle avoidance algorithm updates the speed and direction instructions in real time, so that the execution component avoids obstacles while maintaining trajectory continuity; the reinforcement learning strategy learns the optimal correction behavior in the interference scene, realizes more efficient adaptive path generation. For multi-sensor systems, data fusion weight adjustment can be used to cope with the differences in sensing accuracy, ensuring the robustness of interference information.

[0057] In the execution control link, the path correction vector can be superimposed into the motion control instruction in real time to realize soft transition trajectory adjustment. If the interference level is high, the controller temporarily reduces the execution speed or suspends the operation until the path is recalibrated and the execution is resumed. For scenarios involving flexible execution components, a force-position hybrid control structure can be used to consider spatial obstacle avoidance and contact force stability during the correction process, ensuring that the operation process is both safe and smooth.

[0058] The embodiment realizes the fusion control of environment self-sensing and path self-adjustment by detecting the interference information of the execution component in real time and dynamically correcting the operation path based on the interference level. The mechanism enables the execution component to maintain safe operation in unstructured or dynamically changing space, significantly reducing the need for external intervention. The path correction process has continuity and real-time nature, avoiding the misoperation or shutdown phenomenon caused by obstacle interference in traditional fixed path execution, thereby improving the stability and execution precision of the operation process.

[0059] The application relates to the field of medical robots and image guidance technology, and discloses a target region operation method and device based on sensing image data, equipment and medium, which comprises the following steps: acquiring sensing image data of a target region, identifying the boundary of the target region based on the sensing image data, planning and generating an operation path according to the target region boundary, controlling an execution component to execute an operation along the operation path, acquiring contact feedback signals of an operation tool and adjusting the operation state of the execution component in the execution process, and simultaneously detecting interference information of the execution component in real time and dynamically correcting the operation path according to the detection result, so as to realize autonomous sensing and operation control of the target region. The application constructs a closed-loop autonomous control mechanism from sensing to operation by fusing the sensing image recognition, path planning, execution control and real-time feedback correction process, realizes accurate identification and path adaptive adjustment of the target region in a dynamic environment, and realizes the automatic optimization of the operation state and path trajectory in the execution process by acquiring the contact feedback signals and interference information in real time, so as to improve the operation precision and safety, reduce the error caused by manual intervention, and improve the reliability and stability of the autonomous operation.

[0060] In one embodiment, the above step S10 comprises: S101, scanning a target region through an ultrasonic probe to generate initial sensing image data; S102, performing noise filtering processing on the initial sensing image data to obtain preprocessed sensing image data; S103, transmitting the preprocessed sensing image data to a distributed computing unit; S104, performing spatial coordinate registration and time synchronization processing on the preprocessed sensing image data in the distributed computing unit to generate spatially registered and time-synchronized sensing image data; S105, establishing a mapping relationship with the anatomical structure of the target region based on the spatially registered and time-synchronized sensing image data to generate calibrated sensing image data; S106, performing quality evaluation on the calibrated sensing image data, removing unqualified data frames, and generating final sensing image data.

[0061] In the embodiment, the perception image data of the target region is acquired to support subsequent identification and autonomous operation, and a continuous processing chain from the ultrasound probe to the distributed computing unit to the calibration and quality control needs to be constructed. The target region is scanned by the ultrasound probe to generate initial perception image data. The scanning process includes three links of transmission sequence configuration, echo reception and beam forming. The transmission sequence configuration determines the pulse center frequency, duty cycle, transmission aperture and focal depth to match the tissue acoustic properties of the target region. The echo reception completes preamplification and bandpass filtering at the channel level, and acquires the original radio frequency signal at a fixed sampling rate. The beam forming aligns and superimposes the multi-channel echo according to the dynamic focusing delay to obtain the radio frequency map, and then outputs the gray scale frame through quadrature demodulation and envelope detection. The set of gray scale frames constitutes the initial perception image data. To reduce the spatial distortion caused by probe motion, the probe pose and timestamp can be recorded synchronously for subsequent spatial coordinate registration and time synchronization.

[0062] The initial perception image data is subjected to noise filtering processing to obtain pre-processed perception image data. For ultrasound speckle and high-frequency point noise, a multi-stage denoising link can be used. The first layer performs adaptive bandpass and time gain compensation in the radio frequency domain to equalize deep echo energy. The second layer performs anisotropic diffusion or joint bilateral filtering in the envelope domain to suppress speckle while maintaining edge intensity. The third layer performs motion estimation driven time domain denoising between frames, aligns multiple frames through small displacement optical flow or block matching, and then performs time domain median or low rank constraint filtering, thereby reducing random noise without sacrificing edges. The filtering output should maintain the traceability of bit depth and dynamic range, and be accompanied by processing parameters for quality control and backtracking.

[0063] The pre-processed perception image data is transmitted to the distributed computing unit. To meet real-time and reliability, the transmission link can use zero-copy buffering and ring queue to manage frame flow, support frame-level timestamp and sequence number to detect out-of-order and frame loss. On the network side, Ethernet high-throughput channels can be selected and congestion-aware queues can be enabled. If necessary, lightweight compression is enabled to reduce bandwidth occupancy while ensuring lossless or controllable distortion. The receiving end of the distributed computing unit maintains frame index and metadata index to ensure that the computing task can access the image and pose information of the same frame according to the time and spatial context.

[0064] The pre-processed perception image data is subjected to spatial coordinate registration and time synchronization processing in a distributed computing unit to generate spatially registered and time-synchronized perception image data. The spatial coordinate registration is accomplished through two-stage calibration to map from image coordinates to world coordinates. The first stage is an internal probe calibration to establish the geometric relationship between the probe coordinates and the image coordinates, with input including the array element layout, sound speed model, and field of view parameters, and output being a transformation from image voxels to probe coordinates. The second stage is an external calibration to establish the transformation from probe coordinates to world coordinates, which can be based on optical or electromagnetic tracking to obtain the probe pose, and then through hand-eye calibration to solve the rigid body transformation matrix. The time synchronization uses a unified time base to align the image acquisition clock, pose tracking clock, and network clock. The alignment strategy combines hardware timestamps and software interpolation to output image frames and corresponding poses at the same reference time, eliminating sub-frame level trigger deviations, and forming the spatially registered and time-synchronized perception image data.

[0065] Based on the spatially registered and time-synchronized perception image data, a mapping relationship with the anatomical structure of the target region is established to generate calibrated perception image data. The mapping relationship refers to the correspondence between the image space and the anatomical structure space, which can be constructed through two paths. The first path uses geometric constraints based on anatomical priors to align the known anatomical coordinate system with the world coordinates, and then maps the image voxels to the anatomical coordinate system through the obtained transformation, thereby assigning anatomical position labels on the image frames. The second path uses feature-based registration to measure the similarity between identifiable structures such as blood vessel courses and organ boundaries in the ultrasound image and corresponding structures in the anatomical model, to obtain a sparse or dense correspondence field, and then to re-label the coordinates of the image based on the correspondence field. The two paths can be used jointly to improve robustness. After mapping, the anatomical labels, coordinate conversion matrices, and registration confidence are recorded for each image frame to form the calibrated perception image data.

[0066] The calibrated perception image data is subjected to quality assessment, unqualified data frames are removed, and the final perception image data is generated. The quality assessment gives quantitative indicators from three dimensions of signal integrity, imaging clarity, and geometric consistency. The signal integrity measures the energy and tissue contrast through frame-level signal-to-noise ratio and contrast-to-noise ratio. The imaging clarity evaluates the structure resolution through edge strength, texture consistency, and speckle index. The geometric consistency evaluates the inter-frame alignment reliability by combining spatial coordinate registration residuals, time synchronization jitter, and pose drift. Based on threshold and multi-index weighted scoring, each frame is judged for eligibility, unqualified data frames are removed, and a backfill strategy is triggered to maintain the continuity of the time sequence. The final perception image data retains the qualified frames and their complete metadata, ensuring that subsequent boundary identification and path planning can be run on stable inputs.

[0067] The embodiment constructs a closed-loop acquisition chain from ultrasound probe scanning, noise filtering, distributed transmission, spatial coordinate registration and time synchronization, anatomical mapping to quality assessment, and perceives image data to be systematically improved in spatio-temporal consistency and geometric availability. Noise filtering suppresses speckle and preserves edge details, distributed transmission and time synchronization ensure low latency and alignment accuracy of frame stream, spatial coordinate registration and anatomical mapping embed the image into a computable three-dimensional reference system, and quality assessment filters low-confidence frames to reduce error propagation in subsequent boundary recognition.

[0068] In one embodiment, the above step S20 comprises: S201, loading a pre-trained neural network model to a computing unit; S202, inputting the perception image data into the neural network model to generate an initial segmentation result; S203, performing binaryzation processing on the initial segmentation result to obtain a binaryzation segmentation image; S204, extracting a target region pixel set from the binaryzation segmentation image; S205, generating a target region boundary contour based on the target region pixel set; S206, performing smoothing optimization processing on the target region boundary contour to obtain a final target region boundary.

[0069] In the embodiment, the step of loading a pre-trained neural network model to a computing unit requires model file reading, weight integrity verification, computation graph construction and execution environment initialization. The pre-trained neural network model can be a two-dimensional segmentation network or a three-dimensional segmentation network, and common structures include encoder-decoder morphology and hollow convolution stacking morphology. Model loading binds inference operator library, tensor precision configuration and memory allocation strategy at the same time to ensure that perception image data completes batch inference within a unit of time. To ensure consistency across batches, the normalization mean and standard deviation, interpolation method, boundary padding rule need to be fixed, and the convolution kernel size, receptive field, down-sampling ratio and up-sampling method need to be recorded in the model version metadata. If a lightweight model obtained by quantization or distillation is used, the quantization scale parameter and the dequantization operator need to be registered in the same execution environment to avoid numerical drift.

[0070] When inputting perceived image data into a pre-trained neural network model to generate initial segmentation results, grayscale frames or volume data need to be rearranged according to the tensor shape desired by the model. Two-dimensional networks accept single-frame or multi-frame stacked tensors, while three-dimensional networks accept voxel blocks. To balance field of view and memory usage, a sliding window convergence strategy can be used; overlapping window regions are eliminated by weighted fusion to remove stitching edge artifacts. Input normalization performs linear stretching or logarithmic compression in the intensity domain to match the dynamic range with the training distribution; size integration maintains the spatial aspect ratio through proportional scaling or zero padding. The model's forward inference output is a pixel-wise probability map or a voxel-wise probability map, with output channels containing the probability distributions of foreground and background, and an uncertainty map if necessary, which can be obtained through Monte Carlo discarding or multi-view ensemble. The initial segmentation results are saved in floating-point probability form, and the inference timestamp and model version identifier used are recorded to ensure traceability of subsequent processing.

[0071] The initial segmentation result is binarized to obtain a binarized segmented image. A foreground threshold needs to be determined in the probability domain. Three thresholding strategies can be used: First, a global threshold, which directly uses a fixed threshold or a percentile threshold determined based on training set statistics, suitable for scenes with stable tissue contrast. Second, an adaptive threshold, which calculates a local threshold based on the intensity distribution of the local window, mitigating regional biases caused by uneven gain. Third, a hysteresis threshold, which first uses a high threshold to lock high-confidence pixels, then uses a low threshold to expand the edge bands connected to the high-confidence pixels, reducing breaks. After binarization, small connected component removal and hole filling are performed. Connected component determination uses four-neighbor or eight-neighbor domains. The threshold and structuring element radius are written into the processing metadata for consistency checks.

[0072] Extracting the target region pixel set from a binarized segmented image requires master instance selection and structural correction among candidate connected components. Master instance selection can be based on weighted ranking of indicators such as area, perimeter-to-area ratio, shape compactness, and consistency with established anatomical mapping positions, with the instance having the highest weighted score selected as the target region pixel set. To eliminate jagged edges and isolated spikes, morphological operations can be performed on the edge neighborhood, using opening operations with small structuring elements to remove isolated noise, and then closing operations to close narrow gaps. For cases with multiple lesions, multiple instances can be retained without compromising the consistency of individual instances, and each instance in the set can be labeled with a number and geometric parameters for downstream path planning to process on an instance-by-instance basis.

[0073] When generating the target region boundary contour based on the target region pixel set, contour tracing is performed first, and then sub-pixel positioning is performed. The contour tracing adopts an ordered boundary tracking algorithm to generate a closed point sequence, ensuring that the point sequence is topologically consistent and has no self-intersection. The sub-pixel positioning refines the contour position through edge gradient interpolation, projects the integer grid boundary into a continuous coordinate system, and improves the accuracy of subsequent geometric operations. For three-dimensional data, the contour can be generated in the slice plane by using the contour line algorithm, and then the curved surface segment is formed by correlating between slices. If necessary, the contour of the three-dimensional boundary is reconstructed by using the contour surface method. The contour point sequence is resampled according to the arc length to obtain a uniformly sampled point set, so as to facilitate subsequent smoothing and curvature calculation.

[0074] When the target region boundary contour is subjected to smoothing optimization processing to obtain the final target region boundary, it is necessary to eliminate high-frequency jitter under the constraints of keeping the topology unchanged and the area deviation controllable. Three types of smoothing strategies can be used. The first type is parameterized curve fitting, which fits the contour points into a spline curve, controls the number of nodes and the regularity strength to balance the smoothing degree and shape fidelity. The second type is curvature-based evolution, which performs curvature flow iteration at a small step, limits the maximum displacement and sets an edge stop term to avoid excessive shrinkage. The third type is variational optimization, which minimizes an energy function composed of a boundary length term and a shape prior term. The shape prior can be obtained by statistical training data. After smoothing, the area and centroid variation are evaluated. If the threshold is exceeded, the parameters are rolled back and the process is repeated to ensure that the final target region boundary is geometrically consistent with the target region pixel set and has good differentiability. The final point set, normal estimation, curvature distribution and quality score are written into the boundary data structure for direct consumption by the path generation.

[0075] In this embodiment, the perception image data is converted into the final target region boundary through a probability-geometry dual-channel processing chain. The segmentation probability map provides pixel-level confidence to reduce noise sensitivity. The binarization and set screening compress the uncertain region to the level of verifiable connected domain. The contour tracing and sub-pixel positioning convert the pixel grid constraint into a continuous geometric representation. The smoothing optimization improves the boundary continuity and differentiability within a controlled deviation. This derivation path enables the downstream path planning to directly use a continuous, stable and quality-scored boundary input, reduces the missegmentation propagation caused by speckle, uneven gain and weak contrast, improves the boundary fitting degree and geometric usability, thereby reducing the probability of subsequent operation path length fluctuation and trajectory oscillation, and achieving higher stability and consistency under real-time processing constraints.

[0076] In one embodiment, the above step S30 comprises: S301, discretizing the target region boundary to generate a boundary point set; S302, calculating a path start point position and a path end point position based on the boundary point set; S303, generating multiple candidate paths between the path start point and the path end point using a heuristic search algorithm; S304, evaluating the length and safety indicators of each candidate path to obtain an evaluation result; S305, selecting an optimal path as a preliminary operation path according to the evaluation result; S306, smoothing and optimizing the preliminary operation path to generate a final operation path.

[0077] In this embodiment, the target region boundary is discretized to a computable representation for subsequent search and evaluation, converting the continuous boundary into a boundary point set. The generation of the boundary point set uses a combination of equal arc length sampling and angle-sensitive sampling, first extracting an initial point column according to a uniform arc length interval, and then adaptively inserting supplementary points at curvature mutations to ensure that the geometric features of sharp turns and narrow necks are accurately preserved. During the sampling process, the point sequence is maintained consistent in a clockwise or counterclockwise direction, and the tangent direction, normal direction, and local curvature of each point are recorded, forming a boundary point set with geometric attributes. To connect three-dimensional reconstruction or multi-slice fusion, inter-layer correspondence can be established between boundary point sets of different slices, preserving the section number and spatial coordinate mapping to ensure the stability of the adjacency constraints for subsequent cross-layer path continuity calculation. After the completion of the boundary point set, a distance field and a feasible region mask are generated simultaneously, the distance field gives the minimum signed distance from any point to the target region boundary, and the feasible region mask marks the pixel or voxel positions that allow the path to pass through, providing cost and constraint data for subsequent heuristic search algorithms.

[0078] The determination of the path start point position and the path end point position follows the joint constraints of tool posture, entry direction, feasible region, and safety margin. The path start point position is selected from the accessible subset of the boundary point set, and the shortest collision-free trajectory from the tool tip to the path start point position can be used as the access condition; if there is a predefined set of entry points, the candidate with the lowest cost at the intersection of this set and the boundary point set is selected as the path start point position. The path end point position is determined based on the operation range, exit direction, and avoidance channel, and the most matched point pair in connectivity and normal consistency is selected from the boundary point set, and if necessary, an equidistant inner and outer band is formed by offsetting a small distance in the normal direction inside or outside the boundary to provide a stable path end point position in severe geometric configurations. Once the path start point position and the path end point position are determined, they are written into a key pose structure with posture, which includes position coordinates, tangent direction, allowable curvature upper limit, and minimum clearance threshold, for direct invocation by the heuristic search algorithm.

[0079] A heuristic search algorithm is used to generate multiple candidate paths between the start and end positions of the path. The search space is constructed as a regular grid, a sparse track map or a guide map derived from a set of boundary points according to the feasible region mask, the nodes and edges are defined by the adjacency relationship, and the transition cost is a combination of the path length increment and the safety penalty. The safety penalty comes from the inverse or segmented metric of the distance field, which automatically increases the cost of the candidate path when it approaches the boundary of the target area or the no-entry zone, thereby tending to choose a safer channel. To preserve diversity and robustness, the heuristic search algorithm introduces a random scoring disturbance or a multi-heuristic start mechanism, which is solved under different heuristic functions to form multiple candidate paths; at the same time, curvature constraints and corner penalties are set to prohibit local extreme U-turns. During the search process, the priority queue and closed set marking strategy are used for node expansion to reduce repeated access and loops, and to ensure that a set of available candidate paths is returned within a given time budget.

[0080] Candidate path evaluation focuses on length and safety indicators, and unifies both into the same scoring framework. Length is measured by cumulative arc length, and execution time obtained by adding time parameterization can be used as an alternative indicator. The safety indicator is composed of minimum clearance, clearance distribution quantile, contact risk interval proportion and relative incidence angle with the boundary normal. The minimum clearance guarantees the worst-case scenario for a single point, the quantile and proportion represent the robustness of the entire path, and the relative incidence angle limits the acute angle penetration of weak areas. After dimensionless and weight normalization, the two types of indicators are combined into the evaluation result, which retains the original component values and total score to prevent information loss. To avoid the deviation caused by local roughness in scoring, the curvature statistics of the path trajectory are introduced in the evaluation, limiting the average curvature and maximum curvature within the upper limit, and the curvature peak is written into the evaluation result as an additional penalty term.

[0081] The optimal path selection is sorted and eliminated in the evaluation result, and the candidate with the highest total score that meets the hard threshold is preferred. If there are multiple candidates with similar total scores, the clearance quantile and curvature upper limit are considered with the robustness priority rule, taking into account stability and executability. The selected path is used as the initial operation path for subsequent continuous processing; the initial operation path maintains the original discrete node sequence and segment length information, and is accompanied by local normal estimation to facilitate constraint preservation during smoothing. If all candidates do not meet the hard threshold, return to the heuristic search algorithm phase to automatically relax the soft constraint or expand the search bandwidth, regenerate the candidate and repeat the evaluation and selection to ensure a closed loop.

[0082] The smooth optimization converts the preliminary operation path into a continuous, curvature-limited and parameterizable trajectory, and outputs the final operation path. The optimization adopts a segmented spline or arc hybrid expression, first fitting the discrete nodes with a segmented cubic spline, and then replacing the high-curvature area with a curvature-continuous transition section to eliminate the polyline corner; the clearance keeping term is added to keep the trajectory within the feasible region during iteration, and the negative gradient direction of the distance field is limited to prevent drifting to the high-risk area. To match the dynamic constraints of the execution component, the final operation path is time-parameterized to assign the speed and acceleration curves, so that the speed is automatically reduced on the curve and restored on the straight line, improving the tracking stability. After smoothing, geometric and safety rechecks are performed to calculate the new minimum clearance between the trajectory and the boundary point set, the curvature statistics and the path length difference; if the deviation is out of limit, the weight is adjusted and the iteration is continued until the accuracy and safety thresholds are met, and then the final operation path is output together with the time parameter and quality score for direct use by the execution control.

[0083] In this embodiment, the progressive planning chain composed of the boundary point set, the path start and end positions, the heuristic search algorithm, the evaluation result, the preliminary operation path and the final operation path seamlessly connects the discrete geometry and the continuous trajectory in the same data path, the search stage obtains coverage and diversity, the evaluation stage unifies the length and safety indicators, and the smooth optimization stage converts the feasible solution into an executable solution and explicitly applies the curvature and clearance constraints.

[0084] In one embodiment, the above step S40 comprises: S401, driving the execution component to move to the start position of the operation path; S402, controlling the execution component to move uniformly along the operation path and collecting motion state data of the execution component in real time; S403, calculating the deviation value of the execution component and the operation path based on the motion state data; S404, adjusting the motion parameters of the execution component according to the deviation value to correct the motion trajectory.

[0085] In this embodiment, the movement of the actuator towards the starting position completes the access process with the goal of path consistency and executability. The control unit first reads the parameterized representation and constraint information of the final operation path, parses the spatial coordinates and allowable attitude range of the starting position, and generates an access trajectory consistent with the tangential direction of the operation path. The access trajectory spatially avoids marked restricted areas, temporally aligns with the control cycle, and pre-allocates velocity and acceleration boundaries to meet the driving and braking capabilities of the actuator. To prevent abrupt attitude changes at the end of the access, the final segment of the access trajectory uses a transition segment tangentially continuous with the operation path, and a rapid check of the zero drift and offset of the actuator is performed after access is completed to ensure that subsequent path tracking starts with a consistent coordinate reference.

[0086] Uniform movement along the operating path achieves continuous tracking through time parameterization and sampling synchronization. The control unit reparameterizes the operating path into a monotonic target sequence based on arc length, sets a nominal velocity curve, and automatically reduces the target velocity in sections with high curvature, while restoring the nominal velocity in straight sections. In each control cycle, the motion controller outputs the desired pose and velocity for the next period according to the target sequence and triggers the sampling interface to synchronously acquire motion state data. The motion state data includes basic quantities such as position, attitude, and velocity, which can be provided jointly by the joint encoder, end-effector, and inertial unit. When external positioning is available, the control unit performs time-series alignment and fusion of data from different sources to obtain a single, consistent state estimate, avoiding the accumulation of tracking errors caused by sensor delays or drift. To reduce the impact of sampling jitter on uniformity, the velocity command uses smooth zero-crossing constraints and slope limitations to avoid sudden velocity jumps in high-curvature sections.

[0087] The calculation of deviation values ​​revolves around the geometric projection of the path and the attitude comparison. In the current cycle, the control unit projects motion state data onto the nearest reference point on the operating path, obtaining the orthogonal distance and pointing relationship to the path; simultaneously, it compares the current attitude of the actuator with the path tangent of that reference point to obtain the directional consistency error. Orthogonal distance reflects lateral deviation, and directional consistency reflects pointing deviation; together, they constitute the vectorized representation of the deviation value. To avoid erroneous projections caused by local path self-intersections or dense polylines, geometric projection is only performed within the path interval adjacent to the reference point of the previous cycle, and a monotonicity constraint is applied to the projection results, ensuring that the reference point moves forward along the path without retreating. In high-curvature sections, the weight of the deviation value automatically tilts towards directional consistency to prioritize pointing convergence; in low-curvature sections, more weight is allocated to the orthogonal distance to improve lateral fit. Before entering the adjustment phase, the deviation value undergoes bandpass filtering to remove high-frequency noise and retains low-frequency drift components to support slow-varying compensation.

[0088] The adjustment of motion parameters corrects the motion trajectory with a feedforward plus feedback hierarchical control structure. The feedforward channel directly generates a basic command by adopting the tangential direction of the operation path and the target speed, ensuring the nominal behavior of moving along the path; the feedback channel receives the deviation value and implements a slight correction on the speed, acceleration, and steering amount. The lateral correction aims to reduce the orthogonal distance and gives a steering increment under the premise of maintaining the speed smoothness; the longitudinal correction aims to reduce the direction consistency error and moderately reduces the speed when the curvature and load rise, and restores the speed when the path is flat and the load decreases. To balance the rapid convergence and stability, the control gain adopts an adaptive tuning strategy that changes with the speed and curvature, reducing the gain when the speed is high or the curvature is large, and increasing the gain when the speed is low or the curvature is small, thereby avoiding excessive correction. When the deviation value remains within the set threshold for consecutive multiple periods, the control unit gradually reduces the feedback correction amount to reduce unnecessary micro-vibration and energy consumption. To cope with the hysteresis and dead zone of the execution component, the output command is compensated and protected by limiting the amplitude before entering the driving layer, ensuring that small corrections can be effectively implemented without triggering hardware saturation. During the entire correction process, the control unit continuously records the deviation value, correction amount, and execution feedback for subsequent quality tracking and parameter re-tuning, and triggers a degradation strategy when abnormal jitter or continuous saturation is detected, temporarily reducing the speed or inserting a short stable segment to restore controllability.

[0089] In this embodiment, by starting with consistent pose access at the starting position, uniform speed tracking based on time parameterization, deviation value calculation based on geometric projection and direction comparison, and motion parameter correction with feedforward and adaptive feedback, the actual trajectory of the execution component on the operation path remains highly consistent with the target trajectory, the lateral deviation and pointing deviation are suppressed within a short time, and the speed curve remains continuous under changes in curvature and load, avoiding high-frequency acceleration and deceleration and unnecessary steering oscillation.

[0090] In one embodiment, the above step S50 comprises: S501, during the operation of the execution component, real-time acquisition of contact force data between the operation tool and the target area by the force sensor to generate an original contact feedback signal; S502, noise filtering and signal calibration processing of the original contact feedback signal to obtain a pre-processed contact feedback signal; S503, comparing the pre-processed contact feedback signal with a preset safety threshold to generate a contact force deviation value; S504, calculating the output power adjustment amount of the execution component based on the contact force deviation value; S505, dynamically adjusting the operation state of the execution component according to the output power adjustment amount, and real-time recording of the adjustment parameters.

[0091] In this embodiment, the operating tool and the execution component are in a functionally and structurally tightly coupled relationship, but they have clear division of labor in system level and control logic. The execution component is the main body of driving and control, and its essence is a mechanical or mechatronic assembly that undertakes spatial motion and attitude adjustment, such as a mechanical arm, an end effector or a driven joint module. The operating tool is installed at the end of the execution component and is a unit for energy or force transmission that directly acts on the target area, used to complete specific processing, detection, cutting, pressure application or sensing tasks. The execution component provides precise spatial positioning, attitude control and power input, while the operating tool converts these movements or energies into actual physical actions on the target area.

[0092] At the control level, the execution component realizes closed-loop adjustment of position and attitude through a multi-axis driving system, a motion controller and a feedback loop, and the end flange or interface structure of the execution component provides rigid connection and signal channel for the operating tool. The operating tool acquires real-time action feedback through a sensor or signal acquisition module (such as a force sensor, a temperature sensor or an ultrasonic transducer), and returns it to the control unit of the execution component through a signal processing module. The control unit can thus realize dynamic compensation, path correction or energy adjustment, so that the system forms a complete perception-decision-execution closed loop.

[0093] The core feature of this relationship is that the execution component is responsible for motion accuracy and spatial positioning, and the operating tool is responsible for task energy and contact response. The coupling of the two is reflected not only in the rigid connection at the structural assembly level, but also in the data and energy coupling at the control level. The motion trajectory of the execution component directly determines the action path of the operating tool, while the contact state of the operating tool inversely affects the dynamic control instructions of the execution component, thereby realizing high-precision linkage control and compliant adjustment. This hierarchical division of labor architecture not only guarantees the response speed and control stability of the system, but also makes the operating tool have replaceability and task multiplicity, suitable for automatic fine operation tasks in multiple scenarios.

[0094] The force sensing link works continuously during the motion of the execution component. When the operating tool comes into contact with the target area, the force sensor collects contact force data at a fixed sampling period and generates original contact feedback signals. The sampling period and the motion control period maintain an integer multiple relationship to avoid estimation jitter caused by phase drift. The force sensor is calibrated before work to remove zero drift and update the sensitivity matrix, and during work, zero bias tracking is performed through a short non-carrier band to prevent the accumulation of temperature drift from affecting subsequent threshold judgment. The original contact feedback signal contains not only the instantaneous amplitude, but also the sampling timestamp and channel identifier, which are used to complete time alignment and source verification under multiple sensing channel conditions.

[0095] The original contact feedback signal enters the preprocessing link, and first performs denoising filtering to suppress high-frequency mechanical vibration and electromagnetic interference. The filter is parameterized according to the sampling frequency and the contact frequency band of the operating tool, and the rising edge and falling edge information is retained at the contact mutation edge to maintain the response time. Then, signal calibration is performed, and each component is mapped to a unified force coordinate system using a sensitivity matrix and a cross-axis coupling compensation coefficient to eliminate amplitude and direction deviations caused by structural assembly errors. If there are multiple sensor combinations, a weighted fusion strategy is used, and the weight is related to the confidence of each channel, noise level, and installation position. The fusion result is output as the preprocessed contact feedback signal. To adapt to the asynchronous arrival of sampling frames, the preprocessed contact feedback signal is attached with a timestamp unified to the control time base, ensuring that it is on the same time axis as the motion state estimation for decision-making.

[0096] The preprocessed contact feedback signal enters the threshold comparison link, and a preset safety threshold is used for joint judgment in amplitude and time dimension. The safety threshold is derived from tissue type, operating tool geometric parameters, energy transmission mode and allowable pressure range, and can be mapped to a set of upper and lower limits according to the working condition, which is used to determine under-pressure contact and over-pressure contact. To avoid frequent switching triggered by high-frequency jitter, the threshold comparison introduces a hysteresis band and a retention count. Only when the signal continuously exceeds the threshold for a minimum duration does it generate a contact force deviation value, which includes amplitude deviation and polarity deviation, and records the triggering time and corresponding motion path parameters for subsequent backtracking and statistical setting.

[0097] The contact force deviation value is used to calculate the output power adjustment amount, and the calculation process is bounded by energy density conservation and thermal-force safety constraints. Energy-based operating tools use power or duty cycle as a variable, and the output power adjustment amount is positively correlated with the amplitude deviation and limited by the maximum change rate to avoid instantaneous jump causing tissue carbonization or mechanical feedback oscillation. Mechanical operating tools can map the output power adjustment amount to the increment of drive current or drive pressure. The mapping relationship is obtained during calibration and stored in the control unit. To balance fast convergence and stability, the adjustment amount is added to the feedforward term and the integral suppression term. The feedforward term extrapolates short-term demand based on the trend of the preprocessed contact feedback signal, and the integral suppression term gradually decays when the long-term deviation approaches zero, reducing micro-vibration and heat accumulation. If multiple-axis force component distribution abnormalities are detected, the calculation stage allocates adjustment amounts according to direction, prioritizes normal control of contact depth, and limits lateral slip caused by friction in the tangential direction.

[0098] The output power adjustment is used to update the operation state, which contains two dimensions of energy output parameters and kinematics parameters. The energy output parameters are issued to the end drive in the form of duty cycle, pulse frequency or continuous power, and the drive layer of the execution component executes the soft start and soft stop strategy to ensure the continuous slope of the rising and falling edges. The kinematics parameters focus on maintaining the trace of the path under the premise of slight deceleration or slight retreat, which reduces the instantaneous contact pressure by reducing the nominal speed or introducing a normal micro displacement within the current control period, preventing the over-limit state from continuing. The dual-channel coordination of energy and motion follows a priority strategy, which prioritizes fast response through energy output parameters, and if the over-limit continues, the kinematics parameter adjustment is added to expand the relief space. The updated operation state and execution receipt are written into the record together, and the adjustment parameters in the record include the current contact force deviation value, the output power adjustment, the energy channel setting, the motion channel setting, the timestamp and the path progress index, forming traceable time series data. The control unit uses the record for online self-tuning, automatically fine-tunes the preset safety threshold or adjustment proportion coefficient when detecting that a certain type of tissue frequently triggers over-limit in a specific path segment, and maintains stable energy transmission and contact control within the safety range.

[0099] In this embodiment, the perception, judgment, calculation and execution are connected in series under the same time reference through closed-loop regulation with contact force data as the information source. The preprocessed contact feedback signal improves the measurement reliability, the contact force deviation value provides executable over-limit quantification, and the output power adjustment realizes fast and stable energy regulation at a limited rate, and cooperates with small amplitude kinematic retreat when necessary. As a result, the overpressure and underpressure states are timely suppressed, the contact energy density is stabilized within the safety band, the risk of tissue thermal damage and mechanical pulling is reduced, the energy utilization rate and path tracking stability are improved, and the subsequent working condition self-tuning relies on the recorded adjustment parameters to continuously optimize the threshold and proportion relationship, forming a reproducible contact control quality.

[0100] In one embodiment, the above step S60 comprises: S601, during the execution of the execution component, the distance sensor array is used to collect the obstacle distance data of the environment around the execution component in real time; S602, the relative angle between the obstacle and the motion direction of the execution component is calculated in combination with the inertial measurement unit data; S603, compare the obstacle distance data with the preset safety threshold to generate an obstacle interference level; S604, calculate a path correction vector based on the obstacle interference level and the relative angle; S605, generate an adjustment instruction according to the path correction vector, and dynamically update the trajectory parameters of the operation path based on the adjustment instruction.

[0101] In this embodiment, interference information is defined as a set of environmental factors affecting the execution component's safe progress along the operation path during the whole process of executing the operation. The core observation includes obstacle distance data and the relative angle with the motion direction. The acquisition of interference information relies on the synergy of distance sensor arrays and inertial measurement units. Distance sensor arrays are arranged around the execution component, forming a multi-azimuth distance acquisition structure, which can adopt a ring, sector or end-integrated distribution. Sensor types can include ultrasonic time-of-flight, infrared ranging or structured light depth modules, which can detect obstacles in different distance ranges and reflection characteristics to adapt to the reflection differences of different target surfaces such as soft tissues, medical instruments or surgical supports. Inertial measurement units are used to provide the attitude and angular velocity information of the execution component, and the relative angle between the obstacle direction and the motion direction is calculated in the unified coordinate system by the expected velocity vector output by the motion planner. To realize the fusion of the two types of data in the same time reference, the system completes time alignment through hardware timestamp triggering mechanism or high-precision clock synchronization, and uses calibration matrix to complete coordinate transformation from the end coordinate system to the base coordinate system.

[0102] Obstacle distance data needs to be filtered in time sequence and outliers removed before entering the evaluation stage to suppress the ranging mutations caused by tissue micro-movement or acoustic echo reflection. Time sequence filtering adopts short time window median consistency test method to ensure the stability of local sampling results. The system compares the filtered obstacle distance data with the preset safety threshold to generate obstacle interference levels. The safety threshold is set comprehensively according to the physical outline of the execution component, the allowed approach distance of the tissue and the kinetic braking distance, and can be dynamically adjusted according to the operation stage. The obstacle interference level is not only based on whether the safety threshold is crossed, but also combined with the distance change trend and convergence speed, so as to trigger higher level of interference state in advance when the obstacle approaches quickly, avoiding control response delay.

[0103] The calculation process of path correction vector is based on obstacle interference level and relative angle information. The direction of path correction vector is consistent with the avoidance direction, and its amplitude is determined by interference level, relative angle and current path curvature. To maintain trajectory continuity, the correction vector is injected into the trajectory parameters in incremental manner instead of replacing the original operation path. Trajectory parameters include target position, velocity, acceleration and curvature constraints. The system preferentially adjusts the direction and amplitude of the velocity vector, and adjusts the curvature gently when necessary, to achieve smooth obstacle avoidance. The generated adjustment instruction is used to convert the correction vector into a command structure executable by the controller, including the corrected target pose, speed limit, holding time and path regression parameters. To prevent frequent switching, the adjustment instruction has built-in hysteresis logic, so that the system remains stable when the interference level changes under boundary conditions.

[0104] The trajectory parameters after updating need to be verified for security by a real-time reachability verification module. If the verification is passed, the control command is immediately issued. If the verification fails, the correction amplitude is automatically reduced and the avoidance holding time is extended to prioritize system safety. The entire data flow forms a closed-loop structure of perception, fusion, evaluation, correction, verification, and execution. The synchronous sampling of the distance sensor array and the inertial measurement unit is the input source, and the trajectory parameter update driven by the adjustment command is the output end. The design of the distance sensor array is based on engineering experience from a multi-channel ranging system, and the introduction of the inertial measurement unit ensures accurate fusion of attitude and motion direction information. The obstacle interference level, path correction vector, and adjustment command together constitute the closed-loop obstacle avoidance control logic, achieving adaptive response to dynamic changes in the environment.

[0105] For example, the identification of the tumor boundary is performed by ultrasound image segmentation. Using deep learning methods, especially the U-Net network model, the tumor boundary is automatically extracted by training ultrasound image data.

[0106] The output of the U-Net network is the segmentation result of the ultrasound image, representing the boundary of the tumor region. Given an input image I, the output is a binary segmentation image S, where the tumor region corresponds to a pixel value of 1 and other regions are 0. The formula is as follows:

[0107] By training the network, the model can minimize the loss function, and the cross-entropy loss is usually used to optimize the segmentation accuracy:

[0108] where, is the true label, is the probability value predicted by the model, is the number of pixels.

[0109] After identifying the tumor boundary, the optimal cutting path needs to be generated. To achieve this goal, the A* algorithm is used for path planning.

[0110] The core of the A* algorithm is to generate the shortest path from the starting point to the end point through heuristic search. The value of each node n f(n) is composed of two parts: ① the actual cost g(n) from the starting point to the current node. ② The heuristic estimated cost h(n) from the current node to the target node.

[0111] Therefore, the formula for the value of the A* algorithm is: f(n)=g(n)+h(n) g(n) is the actual path length from the starting point to the current node; h(n) is the estimated cost from the current node to the target node, typically estimated using Euclidean distance or Manhattan distance. The algorithm expands by continuously selecting the node with the minimum cost until the optimal path is found.

[0112] In this system, the contact between the robot and the tissue is monitored by a force feedback sensor, and the intensity of the contact between the electrocautery tool and the tissue is adjusted in real time via the feedback signal. Contact control is achieved by detecting changes in the output power of the electrocautery equipment. The relationship between the contact force F and the output power P can be expressed by the following formula: P=k·F Where k is a constant representing the contact efficiency between the device and the tissue, P is the power, and F is the contact force.

[0113] When the contact force F reaches the set threshold, the system will maintain appropriate cutting intensity by adjusting the robot's motion path or current.

[0114] To handle changes in a dynamic environment, this invention employs local path adjustment to avoid obstacles. In path planning, The robot detects obstacles in real time based on sensor data. If the distance to the detected obstacle is... Less than the set safe distance The system will then dynamically adjust the path. The formula for dynamic path adjustment is:

[0115] in, It is the angle between the obstacle and the robot's current position. This is the distance to the obstacle. The robot will adjust the angle θ of its current path to avoid collisions with the obstacle.

[0116] Example explanation: In the scenario of liver tumor resection surgery, the autonomous operation process of the system is centered on ultrasound perception images, and runs through the complete closed-loop process from data acquisition, tissue identification, path planning to operation control. First, the surgeon scans the local area of the liver through the ultrasound probe installed at the end of the mechanical arm, and the system collects the reflected echo signals in real time and generates initial perception image data. Due to the soft surface of the liver and the easy occurrence of slight displacement due to breathing, the system uses a dynamic time window filtering algorithm to remove noise caused by respiratory motion during the acquisition stage, and compensates for local occlusion through multi-angle scanning. The filtered image data is input into the distributed computing unit, which completes spatial coordinate registration and time synchronization under multi-node parallel processing, ensuring that images at different time frames are accurately aligned in the three-dimensional coordinate system. After that, the system uses the preoperative CT registration model to establish a mapping relationship with the patient's anatomical structure, corrects the image space to a calibrated coordinate system consistent with the position of the real organ, and eliminates ambiguous frames through a quality assessment mechanism to generate high-quality perception image data for subsequent identification.

[0117] In the image recognition stage, the system inputs the calibrated perception image data into the processing unit loaded with a pre-trained neural network model. Based on the medical image segmentation network structure, the model can distinguish between tumor tissue, blood vessel structure and healthy liver tissue. The initial segmentation result output by the model is binarized after thresholding to form a segmentation mask, and the system extracts the pixel set corresponding to the tumor area and performs connected component analysis to exclude artifacts. Then the boundary is smoothly fitted by curvature-constrained splines to eliminate noise and jagged edges, and a continuous and smooth tumor boundary profile is obtained. This boundary is not only used to determine the actual range of the tumor, but also used to provide spatial input for resection operation planning.

[0118] In the path planning stage, the system discretizes the generated tumor boundary to form a boundary point set, and calculates the start and end positions of the resection path in combination with the intraoperative positioning system. The start point is usually located at the safe entry point, which is the optimal entry position that avoids blood vessels and important tissues. The system uses a heuristic search algorithm to generate multiple candidate resection paths, and each path is evaluated considering the cutting length, surgical safety distance and tool accessibility. After double scoring of safety and energy consumption, the system selects the most comprehensive optimal path as the preliminary operation path, and optimizes the final three-dimensional resection trajectory through curve fitting and acceleration constraint. The smoothed path can ensure cutting continuity and mechanical arm motion stability.

[0119] In the operation execution phase, the system controls the robotic arm to drive the execution component to autonomously move along the planned path. The execution component collects its own motion state in real time during the movement, including position, attitude and velocity information. The control unit compares these data with the target path, calculates the deviation of the execution component relative to the operation path. When the deviation exceeds the allowed range, the system immediately adjusts the joint angular velocity of the robotic arm and the end motion parameters to correct the trajectory, so that the mechanical cutting path and the planned trajectory are re-aligned, thereby ensuring the stable and accurate movement of the tool in the tissue.

[0120] During the operation, the system also continuously detects the change of the contact force between the tool and the tissue through the force sensor installed on the cutting tool, generating a contact feedback signal. After filtering and calibration, the signal is compared with the set safety threshold. When the cutting resistance is too large, the system automatically reduces the output power and fine-tunes the pushing rate to prevent the tool from damaging healthy tissue; when the resistance is too small, it may mean that the tumor tissue is not fully contacted, and the system moderately increases the power to ensure complete resection. All adjustment data are recorded in real time for subsequent energy control optimization and model adaptive update.

[0121] At the same time, there are risks of organ movement caused by respiration, deformation of adjacent blood vessels or interference of surgical instruments in the operation environment. The system detects the distance information of the surrounding obstacles in real time through the distance sensor array distributed at the end of the robotic arm, and calculates the relative angle between the obstacles and the motion direction of the robotic arm combined with the data of the inertial measurement unit. If the distance of the obstacles approaches the safety threshold, the system generates an obstacle interference level, and calculates a path correction vector based on it. The path correction vector indicates the avoidance direction and the correction amplitude, and the control module generates adjustment instructions based on it and updates the trajectory parameters in real time to realize obstacle avoidance and path correction. When the interference disappears, the trajectory gradually regresses to the original path to ensure the continuity of the operation.

[0122] The present embodiment quantifies the obstacle distance data and the motion direction angle into a calculable interference index by introducing a real-time fusion detection mechanism of distance sensor array and inertial measurement unit, and feeds back to the control loop in the form of path correction vector, so that the system can dynamically correct the operation path during the execution of the operation. This design enables the execution component to continuously perceive the surrounding obstacles and adjust the motion trajectory in a complex environment, avoiding sudden collisions and path stagnation. The multi-level risk judgment and hysteresis logic based on the interference level suppresses high-frequency oscillation, ensuring the smoothness and continuity of the operation path. The incremental update of the trajectory parameters ensures that the control output is consistent with the dynamics constraints, realizing real-time obstacle avoidance control with high stability, effectively improving the operation precision and environmental adaptability.

[0123] In an embodiment, a target region operation device based on a perception image is provided, which corresponds to the target region operation method based on a perception image in the above-mentioned embodiments. Referring toFigure 3 , Figure 3 Fig. 1 is a schematic diagram of functional modules of a preferred embodiment of the target region operation device based on perception image of the present application. The perception image acquisition module 10, the region identification and segmentation module 20, the path planning module 30, the execution control module 40, the force feedback adjustment module 50 and the interference detection and path correction module 60. The detailed description of each functional module is as follows: The perception image acquisition module 10 is used to acquire the perception image data of the target region; The region identification and segmentation module 20 is used to identify the target region boundary of the target region based on the perception image data; The path planning module 30 is used to plan and generate the operation path of the target region based on the target region boundary; The execution control module 40 is used to control the execution component to execute the operation along the operation path; The force feedback adjustment module 50 is used to acquire the contact feedback signal of the operation tool during the execution component executing the operation, and adjust the operation state of the execution component according to the contact feedback signal; The interference detection and path correction module 60 is used to detect the interference information of the execution component in real time during the execution component executing the operation, and dynamically correct the operation path according to the interference information.

[0124] In one embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes non-volatile and / or volatile storage medium, internal memory. The non-volatile storage medium stores operating system, computer program and database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the function or step of the server side of the target region operation method based on perception image.

[0125] In one embodiment, a computer device is provided, which can be a client, and its internal structure diagram can be as shown in Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of a target area operation method based on perception image client side.

[0126] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the following steps: Obtain perception image data of a target area; Identify the target area boundary of the target area based on the perception image data; Plan an operation path of the target area based on the target area boundary; Control the execution component to execute the operation along the operation path; During the execution of the operation by the execution component, obtain a contact feedback signal of the operation tool, and adjust the operation state of the execution component according to the contact feedback signal; During the execution of the operation by the execution component, real-time detect interference information of the execution component, and dynamically correct the operation path according to the interference information.

[0127] In one embodiment, a computer readable storage medium is provided, which stores a computer program, the computer program is executed by a processor to implement the following steps: Obtain perception image data of a target area; Identify the target area boundary of the target area based on the perception image data; Plan an operation path of the target area based on the target area boundary; Control the execution component to execute the operation along the operation path; During the execution of the operation by the execution component, obtain a contact feedback signal of the operation tool, and adjust the operation state of the execution component according to the contact feedback signal; During the execution of the operation by the execution component, real-time detect interference information of the execution component, and dynamically correct the operation path according to the interference information.

Claims

1. A method for manipulating target regions based on perceived images, characterized in that, Includes the following steps: Acquire sensor image data of the target area; Identify the target region boundary based on the perceived image data; Based on the boundary of the target area, an operation path for the target area is planned and generated. The control execution unit performs operations along the operation path; During the operation of the execution component, a contact feedback signal from the operating tool is acquired, and the operating state of the execution component is adjusted according to the contact feedback signal. During the operation of the execution component, interference information of the execution component is detected in real time, and the operation path is dynamically corrected based on the interference information.

2. The target region manipulation method based on perceived images as described in claim 1, characterized in that, Acquire sensory image data of the target area, including: The target area is scanned by an ultrasonic probe to generate initial sensory image data; The initial sensing image data is subjected to noise filtering to obtain preprocessed sensing image data; The preprocessed sensory image data is transmitted to the distributed computing unit; In the distributed computing unit, the preprocessed sensory image data is processed for spatial coordinate registration and time synchronization to generate spatially registered and time-synchronized sensory image data. Based on the spatially registered and temporally synchronized sensory image data, a mapping relationship with the anatomical structure of the target area is established, and calibrated sensory image data is generated. The calibrated sensory image data is subjected to quality assessment, unqualified data frames are removed, and the final sensory image data is generated.

3. The target region manipulation method based on perceived images as described in claim 1, characterized in that, Identifying the target region boundary based on the perceived image data includes: Load the pre-trained neural network model into the computing unit; The perceived image data is input into the neural network model to generate an initial segmentation result; The initial segmentation result is binarized to obtain a binarized segmented image; Extract the set of pixels of the target region from the binarized segmented image; Generate the boundary contour of the target region based on the pixel set of the target region; The boundary contour of the target region is smoothed and optimized to obtain the final target region boundary.

4. The target region manipulation method based on perceived images as described in claim 1, characterized in that, Based on the boundary of the target area, an operation path for the target area is generated, including: The boundary of the target region is discretized to generate a set of boundary points; Calculate the starting and ending points of the path based on the set of boundary points; A heuristic search algorithm is used to generate multiple candidate paths between the starting point and the ending point of a path; The length and security metrics of each candidate path are evaluated to obtain the evaluation results; Based on the evaluation results, the optimal path is selected as the initial operational path; The initial operation path is smoothed and optimized to generate the final operation path.

5. The target region manipulation method based on perceived images as described in claim 1, characterized in that, The control execution unit performs operations along the operation path, including: The drive unit moves to the starting position of the operation path; The actuator is controlled to move at a constant speed along the operation path, and the motion state data of the actuator is collected in real time. The deviation between the execution component and the operation path is calculated based on the motion state data; The motion parameters of the actuator are adjusted according to the deviation value to correct the motion trajectory.

6. The target region manipulation method based on perceived images as described in claim 1, characterized in that, During the operation of the actuator, a contact feedback signal from the operating tool is acquired, and the operating state of the actuator is adjusted according to the contact feedback signal, including: During the operation of the actuator, the contact force data between the operating tool and the target area is collected in real time by the force sensor to generate the original contact feedback signal. The original contact feedback signal is subjected to noise filtering and signal calibration to obtain a preprocessed contact feedback signal. The preprocessed contact feedback signal is compared with a preset safety threshold to generate a contact force deviation value; The output power adjustment of the actuator is calculated based on the contact force deviation value; The operating state of the actuator is dynamically adjusted according to the output power adjustment amount, and the adjustment parameters are recorded in real time.

7. The target region manipulation method based on perceived images as described in claim 1, characterized in that, During the operation of the execution component, interference information of the execution component is detected in real time, and the operation path is dynamically corrected based on the interference information, including: During the operation of the actuator, distance data of obstacles in the surrounding environment are collected in real time by a distance sensor array; The relative angle between the obstacle and the direction of motion of the actuator is calculated by combining the data from the inertial measurement unit; The obstacle distance data is compared with a preset safety threshold to generate an obstacle interference level; Calculate the path correction vector based on the obstacle interference level and the relative angle; An adjustment instruction is generated based on the path correction vector, and the trajectory parameters of the operation path are dynamically updated based on the adjustment instruction.

8. A target area manipulation device based on perceived images, characterized in that, The target region manipulation device based on the perceived image includes: The sensing image acquisition module is used to acquire sensing image data of the target area; The region identification and segmentation module is used to identify the target region boundary of the target region based on the perceived image data; The path planning module is used to plan and generate an operation path for the target area based on the boundary of the target area; An execution control module is used to control the execution components to perform operations along the operation path; The force feedback adjustment module is used to acquire the contact feedback signal of the operating tool during the operation of the actuator, and adjust the operating state of the actuator according to the contact feedback signal; The interference detection and path correction module is used to detect interference information of the execution component in real time during the operation of the execution component, and dynamically correct the operation path based on the interference information.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a target region manipulation program based on a sensed image stored in the memory and executable on the processor. When executed by the processor, the target region manipulation program based on a sensed image implements the steps of the target region manipulation method based on a sensed image as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a target region manipulation program based on a sensed image, which, when executed by a processor, implements the steps of the target region manipulation method based on a sensed image as described in any one of claims 1-7.

Images