Medical robot safety action generation method and device based on safety perception

Abstract

The invention relates to the technical field of artificial intelligence, and provides a medical robot safety action generation method and device based on safety perception, and the method comprises the steps: obtaining an initial image, extracting visual feature information, combining contact force data and operation force data to obtain multi-modal data, and carrying out the fusion of the multi-modal data to obtain an environment perception matrix, and extracting a security risk feature vector corresponding to the environment perception matrix for analysis and evaluation to obtain a security risk evaluation matrix, and obtaining a security action sequence corresponding to the security risk evaluation matrix and the multi-modal data according to the task target parameter. The method can be applied to medical robot control scenes in the business fields of medical health, old-age care and the like, various safety risks in the operation process can be timely and accurately recognized through multi-modal data fusion perception and accurate safety risk modeling, a safe and reasonable action sequence is planned, and the operation safety is improved. And the safety of operating and controlling the medical robot is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for generating safe actions for medical robots based on safety perception. Background Technology

[0002] Medical robots can be applied in medical settings to replace some manual operations; however, existing control methods struggle to ensure accurate operation in complex scenarios. Furthermore, current control methods for medical robots lack sufficient safety features during motion planning, impacting the safety of medical procedures. Therefore, existing control methods for medical robots suffer from poor safety. Summary of the Invention

[0003] This invention provides a method and apparatus for generating safe actions for medical robots based on safety perception, aiming to solve the problem of poor safety in existing control methods for medical robots.

[0004] In a first aspect, embodiments of the present invention provide a method for generating safe actions of a medical robot based on safety perception. The method is applied to a control terminal, which establishes a communication connection with a visual sensor, a tactile sensor, and a force sensor to transmit data. The tactile sensor is disposed at the tip of a scalpel to sense the pressure contacted by the scalpel tip. The force sensor is disposed on a robotic arm to sense the force applied by the robotic arm to the scalpel. The method includes: The initial image acquired by the vision sensor is obtained and the corresponding visual feature information is extracted. Acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor; The visual feature information, the contact force data, and the operational force data are combined into multimodal data; The multimodal data is fused according to preset data fusion rules to obtain the corresponding environmental perception matrix; The safety risk feature vector corresponding to the environmental perception matrix is ​​extracted by a preset feature extraction model; The security risk feature vector is analyzed and evaluated based on the pre-set security risk model to obtain the corresponding security risk assessment matrix; Based on the preset model predictive control algorithm and the pre-stored task target parameters, obtain the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data.

[0005] Secondly, embodiments of the present invention also provide a safety action generation device for a medical robot based on safety perception. The device is configured on a control terminal, which establishes a communication connection with a visual sensor, a tactile sensor, and a force sensor to transmit data information. The tactile sensor is disposed at the tip of a scalpel to sense the pressure contacted by the scalpel tip. The force sensor is disposed on a robotic arm to sense the force applied by the robotic arm to the scalpel. The device includes: A visual feature extraction unit is used to acquire the initial image collected by the visual sensor and extract the corresponding visual feature information. The data acquisition unit is used to acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor; A multimodal data acquisition unit is used to combine the visual feature information, the contact force data, and the operational force data into multimodal data; An environment perception matrix acquisition unit is used to fuse the multimodal data according to preset data fusion rules to obtain the corresponding environment perception matrix. The safety risk feature vector acquisition unit is used to extract the safety risk feature vector corresponding to the environment perception matrix through a preset feature extraction model. The security risk assessment matrix acquisition unit is used to analyze and evaluate the security risk feature vector according to the preset security risk model to obtain the corresponding security risk assessment matrix. The safety action sequence acquisition unit is used to acquire the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on a preset model predictive control algorithm and pre-stored task target parameters.

[0006] Thirdly, embodiments of the present invention also provide an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0007] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the method described in the first aspect.

[0008] This invention provides a method and apparatus for generating safe actions for medical robots based on safety perception. The method includes: acquiring an initial image and extracting visual feature information; combining contact force data and operational force data to obtain multimodal data and fusing them to obtain an environmental perception matrix; extracting safety risk feature vectors corresponding to the environmental perception matrix for analysis and evaluation to obtain a safety risk assessment matrix; and obtaining a safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on task target parameters. This invention, through multimodal data fusion perception and accurate safety risk modeling, can timely and accurately identify various safety risks during surgery and plan safe and reasonable action sequences, significantly improving the safety of operating and controlling medical robots. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating the method for generating safe actions of a medical robot based on safety awareness, provided in an embodiment of the present invention. Figure 2 A schematic diagram illustrating an application scenario of the safety awareness-based medical robot safety action generation method provided in this embodiment of the invention; Figure 3 A schematic block diagram of a safety-aware medical robot safety action generation device provided in an embodiment of the present invention; Figure 4 A schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] It should also be further understood that the term "and / or" as used in this specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. This invention provides a method and apparatus for generating safety actions for medical robots based on safety perception. For application scenarios of this method for generating safety actions for medical robots based on safety perception, please refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of the safety-aware medical robot safety action generation method provided in this embodiment of the invention. The safety-aware medical robot safety action generation method is applied in, for example... Figure 2 In the application scenario, the control terminal 10 establishes a communication connection with the vision sensor 20, tactile sensor 30, and force sensor 40 via a network to achieve data information transmission. The control terminal can be, but is not limited to, electronic devices such as servers, smartphones, tablets, and desktop computers. The control terminal executes a safety-aware medical robot safety action generation method to obtain initial images collected by the vision sensor, contact force data collected by the tactile sensor, and operational force data collected by the force sensor, and analyzes and plans the action sequence of the medical robot. The tactile sensor is set at the tip of the scalpel to sense the pressure contacted by the tip of the scalpel, and the force sensor is set on the robotic arm to sense the force applied by the robotic arm to the scalpel. The safety-aware medical robot safety action generation method in this embodiment can be applied to medical robot control scenarios in medical and health, elderly care, and other business fields. The invention will be described in detail below through specific embodiments.

[0015] Figure 1 This is a flowchart illustrating a method for generating safe actions for a medical robot based on safety awareness, provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110-S170.

[0016] S110. Acquire the initial image obtained by the vision sensor and extract the corresponding visual feature information.

[0017] The system can acquire initial images from visual sensors. These initial images are surgical scene image data, such as initial images with a resolution of 1920×1080 pixels and a frame rate of 30fps. The initial images contain visual information about surgical instruments and patient tissues and organs. Corresponding visual feature information can be extracted from the acquired initial images.

[0018] In one embodiment, step S110 includes: performing edge detection on the initial image to extract the corresponding edge contour; performing pixel matching on the initial image according to preset standard pixels and the edge contour to identify the corresponding object matching information; and extracting the visual parameters corresponding to the target object as the visual feature information based on the object matching information and the edge contour.

[0019] Specifically, edge detection can be performed on the initial image to obtain the corresponding edge contours. For example, the contrast value of each pixel in the initial image can be calculated. The contrast value is the difference between the current pixel and other surrounding pixels in terms of pixel value. The larger the contrast value, the more significant the difference between the current pixel and other surrounding pixels. Pixels with high contrast values ​​are obtained by pixel dissolution and retained. The retained pixels are then binarized, setting them to "1" and other pixels to "0", thus obtaining the edge contours.

[0020] Further pixel matching is performed on the initial image based on the standard pixels and the edge contours obtained in the above steps. The standard pixels contain the standard pixel values ​​of each object, such as the standard pixel values ​​of "surgical instruments", "patient tissues and organs", and "surgical gowns".

[0021] The initial image is then divided into blocks based on edge contours, resulting in block images corresponding to the edge contours. The outer boundaries of the block images correspond to the contour lines in the edge contours; that is, the initial image is divided into blocks using the contour lines. The pixel values ​​of the block images are then matched with the standard pixel values ​​of each object in the standard pixel set. The object corresponding to the standard pixel value closest to that of the block image is selected as the matching object. This method can be used to obtain matching objects for each block image in the initial image, thus yielding object matching information.

[0022] Visual parameters corresponding to the target object are obtained based on object matching information and edge contour extraction. Object contours corresponding to each object are obtained from the edge contours based on the object matching information, and the object contour of the target object (such as a surgical instrument) is extracted and its position is analyzed to obtain the visual parameters corresponding to the target object. These visual parameters include the precise three-dimensional position and shape parameters of the surgical instrument, and the precise three-dimensional position and shape parameters of the patient's tissues and organs.

[0023] S120: Acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor.

[0024] Acquire contact force data collected by the tactile sensor and operational force data collected by the force sensor; for example, the sampling frequency of the contact force data is 1kHz, and the measurement range is 0-5N; the sampling frequency of the operational force data is 1kHz, and the measurement accuracy is ±0.01N.

[0025] S130. Combine the visual feature information, the contact force data, and the operational force data into multimodal data.

[0026] The obtained force data is combined with visual feature information to obtain multimodal data; the force data includes contact force data and operational force data.

[0027] S140. The multimodal data is fused according to the preset data fusion rules to obtain the corresponding environmental perception matrix.

[0028] Multimodal data is fused according to the set data fusion rules to obtain the corresponding environmental perception matrix.

[0029] In one embodiment, step S140 includes: removing outliers from the multimodal data according to the outlier exclusion interval in the data fusion rules to obtain corresponding valid data; and fusing the valid data according to the fusion equation set in the data fusion rules to obtain the corresponding environmental perception matrix.

[0030] Specifically, the force data in the multimodal data can first be preprocessed, that is, outliers can be excluded from the force data according to the outlier exclusion intervals set in the data fusion rules, thereby obtaining valid data. An outlier exclusion intervals can be set for each value in the force data in the data fusion rules. If a value in the force data falls within this outlier exclusion interval, then that value is identified as an outlier and excluded, and the data after outlier exclusion is obtained as valid data.

[0031] Furthermore, the obtained effective data is fused according to the fusion equation set in the data fusion rules to obtain the corresponding environmental perception matrix; the corresponding fusion equation can be constructed based on Kalman filtering. The fusion equation includes a state equation and an observation equation. Data fusion is performed using Kalman filtering based on the state equation and the observation equation, where the state equation is X. k =A×X k-1 +B×U k-1 +W k-1 , where X kLet A be the state vector of the robot system at time k (extracted from the force data in the valid data), B be the state transition matrix, and C be the control input matrix (obtained by parsing the control input commands of the robot system). Let U be the state vector of the robot system at time k (extracted from the force data in the valid data). k-1 W is the control input at time k-1. k-1 This represents process noise. The observation equation is: Z k =H×X k +V k Z k Let H be the observation vector at time k (extracted from visual feature information in the effective data), and let V be the observation matrix. k To observe noise.

[0032] The environmental state data obtained after fusion includes the precise three-dimensional position and shape parameters of surgical instruments, the precise three-dimensional position and shape parameters of patient tissues and organs, and the contact state between surgical instruments and surrounding objects (magnitude, direction, and point of application of contact force), thus forming the environmental perception matrix E.

[0033] S150. Extract the safety risk feature vector corresponding to the environment perception matrix through a preset feature extraction model.

[0034] The corresponding safety risk feature vector is extracted from the environmental perception matrix using a feature extraction model.

[0035] In one embodiment, step S150 includes: extracting corresponding target features from the environment perception matrix according to the feature extraction network in the feature extraction model; parsing the target features according to the analytical formula in the feature extraction model to obtain corresponding feature parameters; standardizing the feature parameters and combining them to obtain the security risk feature vector.

[0036] First, the environmental perception matrix can be used to extract features through a feature extraction network to obtain target features. This feature extraction network consists of convolutional layers, pooling layers, and fully connected layers. The convolutional layers use 3×3 convolutional kernels to perform convolution operations on the environmental perception matrix and extract local features. The pooling layers use max pooling to reduce the feature dimension. The fully connected layers integrate and filter the extracted features. Finally, the feature extraction network filters and outputs the target features.

[0037] The target features are further analyzed according to the analytical formulas in the data fusion rules to obtain the corresponding feature parameters. For example, for visually related features in the target features, the derivative of the object's trajectory can be calculated to obtain the object's velocity, and the relative distance between the target position and the surgical instruments in the robot system can be calculated using the Euclidean distance formula; for mechanically related features in the target features, the derivative of the fitted curve of the contact force can be calculated to obtain the rate of change of the contact force, and the coordinate points of the force can be determined.

[0038] The feature parameters can be obtained by combining the parameter values ​​obtained from the analysis. Further standardization of the parameter values ​​within the feature parameters is then performed, such as by using a normalization function to normalize each parameter value, thus obtaining the safety risk feature vector F. The range of values ​​for each vector value in the safety risk feature vector is [0,1].

[0039] S160. Analyze and evaluate the security risk feature vector according to the preset security risk model to obtain the corresponding security risk assessment matrix.

[0040] Furthermore, based on the security risk model, the security risk feature vector is analyzed and evaluated to obtain a security risk assessment matrix. This matrix can then be used to perceive and quantify the security status. The security risk model can be obtained based on an improved BP neural network. The security risk model has a three-layer network structure. The number of nodes in the input layer is the same as the dimension of the security risk feature vector F. The number of nodes in the hidden layer is 1.5 times the number of nodes in the input layer, and the number of nodes in the output layer represents the number of possible security risk types. The activation function used is the sigmoid function: f(x) = 1 / (1+e^(-x)). -x The output layer nodes of the security risk model output the corresponding security risk assessment matrix R, where R is a two-dimensional array. ij This represents the probability of the i-th safety risk (such as collision with surgical instruments, excessive compression of tissue, etc.) occurring under the j-th feature, and satisfies ∑ j R ij =1.

[0041] In one embodiment, before step S160, the method further includes: iteratively training the security risk model according to a preset training strategy and pre-stored historical security risk cases to obtain a trained security risk model.

[0042] Before outputting the security risk assessment matrix using the security risk model, the model can be trained using historical security risk cases. Specifically, the training strategy can be set as gradient descent. Then, based on the training strategy and historical security risk cases, the network parameters can be optimized using gradient descent. The loss function used in the parameter optimization process is the cross-entropy loss function. , where y i This refers to the actual risk labels for historical data in historical security risk cases. This is the predicted risk probability obtained by forecasting from historical data. A risk weighting coefficient w is also introduced. j Higher weights are assigned to high-risk types (such as collisions with vital organs of patients) to improve the model's sensitivity to high risks.

[0043] S170. Obtain the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on the preset model prediction control algorithm and the pre-stored task target parameters.

[0044] Based on the set model predictive control algorithm and pre-stored task target parameters, a safety action sequence corresponding to the safety risk assessment matrix and multimodal data is obtained. The task target parameters are the specific objectives of the current surgical operation, such as the location, angle, and range of force of the surgical operation.

[0045] In one embodiment, step S170 includes: constructing a corresponding prediction model based on the visual parameters in the multimodal data, the task target parameters, and the kinematic parameters set in the model predictive control algorithm; iteratively optimizing the output of the prediction model based on the objective function in the model predictive control algorithm and the safety risk assessment matrix, so as to obtain the control input sequence with the minimum value of the objective function as the safety action sequence.

[0046] Model predictive control algorithms are configured with kinematic parameters (such as joint range of motion, maximum speed, etc.). First, a predictive model based on robot kinematics can be constructed based on task target parameters, visual parameters, and kinematic parameters to predict the robot's action sequence over a period of time in the future.

[0047] Based on the objective function and safety risk assessment matrix in the model predictive control algorithm as constraints, the output of the predictive model is iteratively optimized. In each iteration, the predictive model can output a set of control input sequences, thereby obtaining the control input sequence corresponding to the minimum value of the objective function as the safety action sequence. The optimal control input sequence is solved by quadratic programming.

[0048] The objective function can be set as follows: Where N is the prediction time domain, Q is the risk weight matrix, and U k R is the control weight matrix, representing the control input. During motion planning, it's necessary to consider the robot's kinematic parameters, i.e., constrain the robot's actions based on these parameters, such as joint angle ranges and speed limits; simultaneously, surgical operation constraints, such as force limitations, also need to be considered.

[0049] For actions with a high probability of risk, path correction or force adjustment is performed, for example, when the collision risk probability R... 碰撞 When the distance exceeds 0.3, the motion trajectory is automatically adjusted to increase the safety distance Δd=s×R. 碰撞 , where s is the safety factor, which can be set from 1.2 to 1.5 depending on the surgical scenario.

[0050] In robot control, the process of solving the optimal control input sequence through quadratic programming (QP) can be divided into the following steps: (1) Objective function construction, with the quadratic cost function of the control input sequence (such as joint torque / velocity) as the core, including: tracking error term: minimizing the deviation between the actual trajectory and the expected trajectory of the end effector (such as force tracking accuracy); smoothness term: constraining the rate of change of control input (such as jerk) and reducing mechanical vibration; energy efficiency term: reducing the amplitude of control input (such as motor energy consumption). (2) Constraint modeling, transforming physical and task constraints into linear / quadratic inequalities: kinematic constraints: joint angle range (θ min ≤θ≤θ max ), velocity / acceleration boundary (v θ ≤v max Operational constraints: Upper limit of surgical contact force (F≤F) max ), obstacle avoidance distance limit; dynamic constraints: implicit constraints based on robot dynamic equations (such as the Lagrange equation). (3) Numerical solution: using the interior point method or activity set algorithm, the control input sequence that minimizes the cost function is found through iterative optimization under the premise of satisfying all constraints. For example: at each time step, the control output for the next N steps is predicted based on the current state; feasible solutions are calculated in real time through QP solvers (such as OSQP, ACADO), and control instructions are updated in a rolling manner.

[0051] Finally, the control input sequence that satisfies the constraints and minimizes the function value of the objective function is obtained as the safe action sequence A. The robot's safe action sequence A includes parameters such as the motion angle, motion speed, and operating force of each joint. The time interval between two sets of control inputs in the safe action sequence is 0.1s to ensure the continuity and accuracy of the action.

[0052] In one embodiment, after step S170, the method further includes: executing the safety action sequence and acquiring feedback information collected by the visual sensor, the tactile sensor, and the force sensor; updating the environmental perception matrix in real time based on the feedback information; adjusting the safety action sequence based on a preset optimization algorithm and the real-time updated environmental perception matrix; and executing the adjusted and updated safety action sequence to correct the actions.

[0053] Feedback information on the robot's actual actions (such as real-time data from joint angle sensors and force sensors), safety action sequence A, and the updated environmental perception matrix E' (updated in real time by the multimodal data fusion perception module).

[0054] A proportional-integral-derivative (PID) control algorithm is used as the optimization algorithm to perform real-time feedback adjustments on the safety action sequence obtained in the above steps. The deviation between the actual action and the planned action is calculated as e = AA. 实际 Then adjust according to the deviation: K p K is the proportionality coefficient. i K is the integral coefficient. d The coefficients are differential coefficients. Simultaneously, the updated environmental perception matrix E' is compared with the environmental perception matrix E acquired at the previous moment (the environmental perception matrix before the update), and the environmental change ΔE = E' - E is calculated. When ΔE exceeds a set threshold, the safety risk feature extraction and modeling steps are retried, the safety risk assessment matrix R is updated, and the safety action sequence A is adjusted based on the updated safety risk assessment matrix. The adjusted safety action sequence is then executed to correct the robot's operational actions.

[0055] The adjusted robot motion control command U corrects the robot's movements in real time, keeping the deviation between the actual and planned movements within allowable limits (e.g., joint angle deviation less than 0.5°, force deviation less than 0.05N).

[0056] The technical methods employed in this application possess the following superior characteristics: 1. Enhanced safety: Through multimodal data fusion perception and precise safety risk modeling, various safety risks during the surgical process can be identified in a timely and accurate manner. Robotic actions can be planned and adjusted in advance, effectively preventing collisions between the robot and surrounding objects, as well as injuries to patients caused by improper operation, significantly improving the safety of the surgical process. 2. Improved surgical outcomes: Safe action planning ensures that the robot performs surgical tasks more precisely under safe conditions, reducing operational errors caused by safety concerns, thereby improving the precision and quality of the surgery, enhancing surgical outcomes, and helping patients recover faster. 3. Enhanced system adaptability: The real-time feedback and adjustment mechanism enables the robot to flexibly adjust its actions according to dynamic changes during the surgical process, enhancing the adaptability of the medical robot system to complex and variable surgical environments and broadening its application scope. Overall, the above methods, based on advanced safety perception and action generation methods, greatly reduce the risk of medical accidents caused by robot malfunctions or operational errors during surgery, providing more reliable surgical protection for patients and medical staff.

[0057] This application discloses a method for generating safe actions for medical robots based on safety perception. It acquires an initial image and extracts visual feature information, combines contact force data and operational force data to obtain multimodal data, and fuses this data to obtain an environmental perception matrix. Safety risk feature vectors corresponding to the environmental perception matrix are extracted and analyzed to obtain a safety risk assessment matrix. Based on task target parameters, a safe action sequence corresponding to the safety risk assessment matrix and the multimodal data is obtained. This invention, through multimodal data fusion perception and accurate safety risk modeling, can timely and accurately identify various safety risks during surgery and plan safe and reasonable action sequences, significantly improving the safety of operating and controlling medical robots.

[0058] Figure 3 This is a schematic block diagram of a safety-aware-based medical robot safety action generation device provided in an embodiment of the present invention. Figure 3 As shown, corresponding to the above-mentioned method for generating safe actions for medical robots based on safety perception, the present invention also provides a device for generating safe actions for medical robots based on safety perception, the device being configured in, as shown in... Figure 2 In this application scenario, the control terminal 10 establishes a communication connection with the vision sensor 20, the tactile sensor 30, and the force sensor 40 via a network to achieve data transmission. The control terminal can be, but is not limited to, electronic devices such as servers, smartphones, tablets, and desktop computers. For details, please refer to... Figure 3 The safety-aware-based medical robot safety action generation device 700 includes: The visual feature extraction unit 701 is used to acquire the initial image obtained by the visual sensor and extract the corresponding visual feature information.

[0059] The data acquisition unit 702 is used to acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor; The multimodal data acquisition unit 703 is used to combine the visual feature information, the contact force data, and the operational force data into multimodal data.

[0060] The environment perception matrix acquisition unit 704 is used to fuse the multimodal data according to the preset data fusion rules to obtain the corresponding environment perception matrix.

[0061] The safety risk feature vector acquisition unit 705 is used to extract the safety risk feature vector corresponding to the environment perception matrix through a preset feature extraction model.

[0062] The security risk assessment matrix acquisition unit 706 is used to analyze and evaluate the security risk feature vector according to the preset security risk model to obtain the corresponding security risk assessment matrix.

[0063] The safety action sequence acquisition unit 707 is used to acquire the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data according to the preset model prediction control algorithm and the pre-stored task target parameters.

[0064] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned safety perception-based medical robot safety action generation device and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0065] The aforementioned safety-aware medical robot safety action generation device can be implemented as a computer program, which can, for example... Figure 4 It runs on the electronic device shown.

[0066] Please see Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 800 can be a terminal or a server. The terminal can be an electronic device with communication functions. The server can be a standalone server or a server cluster composed of multiple servers.

[0067] See Figure 4 The electronic device 800 includes a processor 802, a memory, and a network interface 805 connected via a system bus 801. The memory may include a non-volatile storage medium 803 and internal memory 804.

[0068] The non-volatile storage medium 803 may store an operating system 8031 ​​and a computer program 8032. The computer program 8032 includes program instructions that, when executed, cause the processor 802 to perform a method for generating safety actions for a safety-aware medical robot.

[0069] The processor 802 provides computing and control capabilities to support the operation of the entire electronic device 800.

[0070] The internal memory 804 provides an environment for the operation of the computer program 8032 in the non-volatile storage medium 803. When the computer program 8032 is executed by the processor 802, the processor 802 can execute a method for generating safety actions of a medical robot based on safety awareness.

[0071] This network interface 805 is used for network communication with other devices. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 800 to which the present invention is applied. The specific electronic device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0072] The processor 802 is used to run the computer program 8032 stored in the memory to implement the steps included in the above-mentioned method for generating safe actions of medical robots based on safety awareness.

[0073] It should be understood that, in this embodiment of the invention, the processor 802 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0074] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0075] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the steps included in the above-described method for generating safety actions for a safety-aware medical robot.

[0076] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0077] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0078] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0079] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0081] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for generating safe actions for a medical robot based on safety perception, characterized in that, The method is applied to a control terminal, which establishes a communication connection with a vision sensor, a tactile sensor, and a force sensor to achieve data transmission. The tactile sensor is disposed at the tip of the scalpel to sense the pressure contacted by the tip of the scalpel. The force sensor is disposed on a robotic arm to sense the force applied by the robotic arm to the scalpel. The method includes: The initial image acquired by the vision sensor is obtained and the corresponding visual feature information is extracted. Acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor; The visual feature information, the contact force data, and the operational force data are combined into multimodal data; The multimodal data is fused according to preset data fusion rules to obtain the corresponding environmental perception matrix; The safety risk feature vector corresponding to the environmental perception matrix is ​​extracted by a preset feature extraction model; The security risk feature vector is analyzed and evaluated based on the pre-set security risk model to obtain the corresponding security risk assessment matrix; Based on the preset model predictive control algorithm and the pre-stored task target parameters, obtain the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data.

2. The method for generating safe actions for medical robots based on safety perception according to claim 1, characterized in that, The step of fusing the multimodal data according to preset data fusion rules to obtain the corresponding environment perception matrix includes: The multimodal data is subjected to outlier exclusion based on the anomaly exclusion interval in the data fusion rules to obtain the corresponding valid data; The effective data is fused according to the fusion equation set in the data fusion rules to obtain the corresponding environmental perception matrix.

3. The method for generating safe actions for medical robots based on safety perception according to claim 1, characterized in that, The step of acquiring the initial image obtained by the visual sensor and extracting the corresponding visual feature information includes: Edge detection is performed on the initial image to extract the corresponding edge contours; The initial image is pixel-matched based on preset standard pixels and the edge contour to identify the corresponding object matching information; The visual parameters corresponding to the target object are extracted based on the object matching information and the edge contour as the visual feature information.

4. The method for generating safe actions for medical robots based on safety perception according to claim 1, characterized in that, The step of extracting the security risk feature vector corresponding to the environmental perception matrix through a preset feature extraction model includes: The corresponding target features are extracted from the environmental perception matrix based on the feature extraction network in the feature extraction model. The target features are analyzed according to the analytical formula in the feature extraction model to obtain the corresponding feature parameters; The feature parameters are standardized and combined to obtain the security risk feature vector.

5. The method for generating safe actions for medical robots based on safety perception according to claim 1, characterized in that, Before analyzing and evaluating the security risk feature vector based on a pre-set security risk model to obtain the corresponding security risk assessment matrix, the method further includes: The security risk model is iteratively trained based on a pre-set training strategy and pre-stored historical security risk cases to obtain the trained security risk model.

6. The method for generating safe actions for medical robots based on safety perception according to claim 1, characterized in that, The step of obtaining the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on the preset model predictive control algorithm and the pre-stored task target parameters includes: Based on the visual parameters in the multimodal data, the task target parameters, and the kinematic parameters set in the model prediction control algorithm, a corresponding prediction model is constructed. Based on the objective function in the model predictive control algorithm and the safety risk assessment matrix, the output of the predictive model is iteratively optimized to obtain the control input sequence with the minimum value of the objective function as the safety action sequence.

7. The method for generating safe actions for medical robots based on safety perception according to claim 1 or 6, characterized in that, After obtaining the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on the preset model predictive control algorithm and the pre-stored task target parameters, the method further includes: The safety action sequence is executed, and feedback information collected by the visual sensor, the tactile sensor, and the force sensor is obtained. The environmental perception matrix is ​​updated in real time based on the feedback information. The safety action sequence is adjusted based on the preset optimization algorithm and the real-time updated environmental perception matrix, and the adjusted and updated safety action sequence is executed to correct the actions.

8. A safety action generation device for medical robots based on safety perception, characterized in that, The device is configured on a control terminal, which establishes a communication connection with a vision sensor, a tactile sensor, and a force sensor to transmit data information. The tactile sensor is disposed on the tip of the scalpel to sense the pressure contacted by the tip of the scalpel. The force sensor is disposed on the robotic arm to sense the force applied by the robotic arm to the scalpel. The device is used to execute the safety action generation method for a medical robot based on safety perception as described in any one of claims 1-7. The device includes: A visual feature extraction unit is used to acquire the initial image collected by the visual sensor and extract the corresponding visual feature information. The data acquisition unit is used to acquire the contact force data collected by the tactile sensor and the operational force data collected by the force sensor; A multimodal data acquisition unit is used to combine the visual feature information, the contact force data, and the operational force data into multimodal data; An environment perception matrix acquisition unit is used to fuse the multimodal data according to preset data fusion rules to obtain the corresponding environment perception matrix. The safety risk feature vector acquisition unit is used to extract the safety risk feature vector corresponding to the environment perception matrix through a preset feature extraction model. The security risk assessment matrix acquisition unit is used to analyze and evaluate the security risk feature vector according to the preset security risk model to obtain the corresponding security risk assessment matrix. The safety action sequence acquisition unit is used to acquire the safety action sequence corresponding to the safety risk assessment matrix and the multimodal data based on the preset model prediction control algorithm and the pre-stored task target parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the safety action generation method for medical robots based on safety awareness as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the safety action generation method for a medical robot based on safety awareness as described in any one of claims 1-7.

Images