A method, device, equipment, and storage medium for motion control of a medical device.

Abstract

This application provides a motion control method, apparatus, device, and storage medium for a medical device. The method includes: acquiring video of a region of the medical device; calculating the depth of pixels in each frame of the image and the rate of change of depth; controlling the device to stop when the rate of change of depth of any pixel exceeds a first threshold; executing the operation when the rate of change of depth of any pixel does not exceed the first threshold; identifying the obstacle type corresponding to each pixel in the current frame of the image, with different weights for different types of pixels; calculating the collision risk between each pixel and the device based on the weight and depth of each pixel; calculating the target speed of the device using the average collision risk of each pixel in a specified area along the device's motion direction and the device's current speed when the average collision risk exceeds a second threshold; and controlling the device to move along the target turning direction at the target speed, using the direction from the device to the center of the area with the smallest average collision risk in the image as the target turning direction.

Description

Technical Field

[0001] This application relates to the field of motion control technology, and in particular to a motion control method, device, equipment and storage medium for a medical device. Background Technology

[0002] In the medical field, mobile C-arms and other medical devices are often required to perform cross-room transfers in complex and dynamic environments such as hospital corridors and operating rooms. The distance of a single movement can reach 100-200 meters. The environment contains various obstacles such as medical staff, medical equipment, and walls. Therefore, high requirements are placed on the safety and flexibility of the device movement.

[0003] Currently, the movement of medical devices in related technologies mainly falls into two categories: manual pushing and automated walking based on SLAM (Simultaneous Localization and Mapping) technology. Manual pushing is not only inefficient but also prone to collisions with obstacles due to human error. SLAM-based automated walking requires first constructing an environmental map by stitching together multiple images, then planning the movement path based on the map. This map construction process is time-consuming, resulting in low initiation efficiency for automated walking. Furthermore, it cannot update the map in real-time for dynamic obstacles such as medical personnel and mobile carts, easily leading to delayed obstacle avoidance responses. Simultaneously, the speed control of medical devices in related technologies lacks deep integration with environmental perception, generally employing a single distance threshold-triggered control mode. Speed ​​reduction only occurs when the distance between the device and an obstacle is less than a preset value, failing to adjust speed based on dynamic information such as the approach trend of obstacles. This makes it difficult to balance safe obstacle avoidance and movement efficiency, severely impacting the safety and convenience of medical devices in clinical settings. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, this application provides a motion control method, device, equipment and storage medium for a medical device.

[0005] According to a first aspect of the embodiments of this application, a motion control method for a medical device is provided, the method comprising: Acquire real-time video of the current area of ​​the medical device; Calculate the depth information of each pixel in each frame of the video, and the rate of change of the depth information within a preset time window; If the rate of change of depth information at any pixel exceeds a first risk threshold, the medical device is controlled to stop moving. If the rate of change of depth information at each pixel does not exceed the first risk threshold, perform the following steps; By utilizing the image features of each pixel in the current frame image, the type of obstacle corresponding to each pixel is identified. Different types of pixels have different weights, and the weights are used to characterize the degree of collision hazard posed by the obstacle corresponding to the pixel to the medical device. Based on the weights of each pixel in the current frame image and the depth information of each pixel, the collision risk between the obstacle corresponding to each pixel and the medical device in motion is calculated. If the average collision risk of each pixel in a designated area within the current direction of movement of the medical device exceeds a second risk threshold, the target speed of the medical device is calculated using the average collision risk and the current speed of the medical device. The target speed is negatively correlated with the average collision risk level. Furthermore, the direction from the medical device to the center pixel of the image region with the lowest average collision risk in the current frame image is taken as the target turning direction, and the medical device is controlled to move along the target turning direction at the target speed.

[0006] According to a second aspect of the embodiments of this application, a motion control device for a medical device is provided, the device comprising: The video acquisition module is used to acquire real-time video of the current area of ​​the medical device; The depth and rate of change calculation module is used to calculate the depth information of the pixels in each frame of the video, and the rate of change of the depth information within a preset time window. The rate of change control module is used to control the medical device to stop moving when the rate of change of the depth information of any pixel exceeds a first risk threshold. The obstacle type recognition module is used to identify the type of obstacle corresponding to each pixel by utilizing the image features of each pixel in the current frame image, provided that the rate of change of the depth information of each pixel does not exceed the first risk threshold. Different types of pixels have different weights, and the weights are used to characterize the degree of collision hazard of the obstacle corresponding to the pixel to the medical device. The collision risk calculation module is used to calculate the collision risk between the obstacle corresponding to each pixel and the medical device in motion, based on the weight of each pixel in the current frame image and the depth information of each pixel. The motion control module is configured to, when the average collision risk of each pixel within a specified area in the current direction of movement of the medical device exceeds a second risk threshold, calculate a target speed for the medical device using the average collision risk and the current speed of the medical device, wherein the target speed is negatively correlated with the average collision risk; and control the medical device to move at the target speed along the target turning direction, using the direction from the medical device toward the center pixel of the image region with the smallest average collision risk in the current frame image as the target turning direction.

[0007] According to a third aspect of the embodiments of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0009] The technical solutions provided in this application embodiment may include the following beneficial effects: This application embodiment acquires real-time video of the current area of ​​the medical device, calculates the depth information and rate of change of pixels in the video frames, and uses the rate of change of depth information to make a preliminary threshold judgment of collision risk. Then, it identifies obstacle types through image features and assigns different collision hazard weights. The collision risk corresponding to each pixel is calculated by fusing the weights and depth information. Finally, the moving speed and turning direction of the medical device are dynamically adjusted according to the distribution characteristics of collision risk, realizing real-time collision detection and adaptive motion control of the medical device without map construction. This solution does not require the pre-construction of an environmental map and can make accurate predictions of collision risks based on real-time environmental perception information. At the same time, it combines the type, distance and approach trend of obstacles to achieve multi-dimensional speed and turning control, effectively improving the obstacle avoidance response speed and control accuracy of the medical device in complex dynamic environments, while taking into account both the safety and efficiency of device movement.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this application, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1This is a schematic flowchart illustrating a motion control method for a medical device according to an exemplary embodiment of this application.

[0013] Figure 2 This is a schematic diagram illustrating the turning of a medical device according to an exemplary embodiment of this application.

[0014] Figure 3 This is a schematic flowchart illustrating another motion control method for a medical device according to an exemplary embodiment of this application.

[0015] Figure 4 This is a schematic diagram of the structure of a motion control device for a medical device according to an exemplary embodiment of this application.

[0016] Figure 5 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment of this application. Detailed Implementation

[0017] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0018] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0020] In the medical field, mobile C-arms and other medical devices are often required to perform cross-room transfers in complex and dynamic environments such as hospital corridors and operating rooms. The distance of a single movement can reach 100-200 meters. The environment contains various obstacles such as medical staff, medical equipment, and walls. Therefore, high requirements are placed on the safety and flexibility of the device movement.

[0021] Currently, the movement of medical devices in related technologies mainly falls into two categories: manual pushing and automated walking based on SLAM (Simultaneous Localization and Mapping) technology. Manual pushing is not only inefficient but also prone to collisions with obstacles due to human error. SLAM-based automated walking requires first constructing an environmental map by stitching together multiple images, then planning the movement path based on the map. This map construction process is time-consuming, resulting in low initiation efficiency for automated walking. Furthermore, it cannot update the map in real-time for dynamic obstacles such as medical personnel and mobile carts, easily leading to delayed obstacle avoidance responses. Simultaneously, the speed control of medical devices in related technologies lacks deep integration with environmental perception, generally employing a single distance threshold-triggered control mode. Speed ​​reduction only occurs when the distance between the device and an obstacle is less than a preset value, failing to adjust speed based on dynamic information such as the approach trend of obstacles. This makes it difficult to balance safe obstacle avoidance and movement efficiency, severely impacting the safety and convenience of medical devices in clinical settings.

[0022] Based on this, and to address the problems existing in related technologies, this application provides a motion control method for medical devices. This method acquires real-time video of the current area of ​​the medical device, calculates the depth information and rate of change of pixels in the video frames, and uses the rate of change of depth information to make a preliminary threshold judgment of collision risk. Then, it identifies obstacle types through image features and assigns different collision hazard weights. The weights and depth information are fused to calculate the collision risk corresponding to each pixel. Finally, based on the distribution characteristics of collision risk, the method dynamically adjusts the moving speed and turning direction of the medical device, achieving real-time collision detection and adaptive motion control of the medical device without map construction. This method does not require pre-construction of an environmental map and can accurately predict collision risks based on real-time environmental perception information. Simultaneously, it combines the type, distance, and approach trend of obstacles to achieve multi-dimensional speed and turning control, effectively improving the obstacle avoidance response speed and control accuracy of medical devices moving in complex dynamic environments, while balancing the safety and efficiency of device movement.

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] The motion control method for medical devices provided in this application can be executed by computer devices capable of data processing, image analysis, and command output. These computer devices include, but are not limited to, physical servers, server clusters, and cloud servers. They can also be embedded computing terminals or industrial control computers integrated into the medical device, and may include terminal devices such as smartphones, tablets, laptops, and desktop computers. These devices can establish connections with the medical device via wired or wireless communication to achieve data interaction and the issuance of motion commands. Specifically, the medical devices described in this application can be various types of equipment in the medical field that require mobility, such as mobile C-arms, mobile DR machines, surgical navigation robots, and medical transport vehicles—equipment that needs to perform mobile operations in complex environments such as hospital corridors and operating rooms.

[0025] Figure 1 This is a schematic flowchart illustrating a motion control method for a medical device according to an exemplary embodiment of this application. Figure 1 As shown, the method includes steps S101 to S107.

[0026] Step S101: Obtain real-time video of the current area of ​​the medical device.

[0027] To achieve real-time perception of the environment surrounding medical equipment, this embodiment can mount a monocular RGB camera at a preset location on the medical equipment. This camera can be positioned at the front or side of the equipment, where the field of view is unobstructed, depending on the equipment's shape and environmental perception requirements. This ensures complete capture of the surrounding environment within a 360° range or a specified angle during the equipment's movement. The camera's frame rate and resolution can be adjusted according to the actual usage scenario to balance real-time image acquisition with clarity. The monocular RGB camera continuously captures environmental images of the area currently occupied by the medical equipment, generating a continuous real-time video stream, which is then transmitted in real-time to the corresponding computer device. This provides the initial visual data foundation for subsequent image analysis and collision risk assessment.

[0028] Step S102: Calculate the depth information of each pixel in the video and the rate of change of the depth information within a preset time window.

[0029] In this embodiment, the depth information of a pixel can characterize the spatial distance between the environmental target corresponding to the pixel and the medical device, which is the core spatial feature for judging collision risk; while the rate of change of the depth information within a preset time window can reflect the movement trend of the corresponding environmental target relative to the medical device, that is, whether the target is approaching, moving away or stationary, which is a key indicator for predicting dynamic collision risk.

[0030] The calculation of pixel depth information can be achieved in various ways, such as LiDAR ranging and stereo matching ranging with binocular cameras. To balance the efficiency and accuracy of depth information calculation, in some embodiments, a pre-trained depth estimation model can be used to estimate the depth of pixels in each frame of the video, thereby obtaining the depth information of each pixel. In specific implementation, the MiDaS depth estimation model can be preferred to complete the calculation of pixel depth information. This model is built on a pre-trained visual Transformer or CNN backbone network (such as DPT-Large). During the training phase, a transfer learning method is used. First, basic training is completed using image depth features of general scenes, and then fine-tuning is performed using image data from medical scenes such as hospital operating rooms and corridors to accelerate the model's adaptability to medical scenes. During model training, the L1 loss between the predicted depth map and the real depth map is used as the optimization objective. The Adam optimizer is used to iteratively adjust the network parameters. At the same time, an early stopping strategy is implemented by monitoring the performance indicators of the validation set to avoid model overfitting and ensure that it can stably identify the depth levels of key targets such as people, equipment, and walls in medical scenes. In the actual prediction process, each frame of RGB image captured by the monocular RGB camera can be preprocessed according to the specifications during model training. After completing operations such as image size adjustment and pixel value normalization, the image is input into the model. The model extracts the depth features of the image through the backbone network, and then the depth estimation head processes the features to finally generate a relative depth map with the same size as the input image. The value of each pixel in the depth map is the depth information of the corresponding position, and this value is positively correlated with the actual distance. That is, the larger the pixel value, the closer the corresponding target is to the medical device.

[0031] After obtaining the depth information of each pixel in each frame of the image, the rate of change of depth information within a preset time window can be calculated. In this embodiment, the preset time window can be flexibly set according to the image acquisition frame rate. For example, 3 or 5 consecutive frames of images can be selected as a time window. First, the depth information value of the same pixel in each frame of the image within the time window is extracted. Then, by calculating the ratio of the change of the value within the time window to the time, the rate of change of the depth information of the pixel is obtained. If the rate of change is positive, it indicates that the corresponding target is approaching the medical device. The larger the value of the rate of change, the faster the target approaches.

[0032] Step S103: Detect whether the rate of change of depth information of any pixel exceeds the first risk threshold. If yes, control the medical device to stop moving. If no, proceed to step S104.

[0033] In this embodiment, whether the rate of change of pixel depth information exceeds a first risk threshold is the core criterion for determining whether there are rapidly approaching dynamic obstacles around the medical device. The first risk threshold can be customized according to the movement scenario and preset movement speed of the medical device. Its value can be calibrated through a large number of medical scenarios and can accurately match the depth information change characteristics when a dynamic obstacle is rapidly approaching. When the rate of change of depth information of any pixel in the video frame image exceeds the first risk threshold, it indicates that the environmental target corresponding to the pixel is rapidly approaching the medical device at a speed exceeding the safety prediction, such as a medical staff walking by quickly or a medical cart being pushed closer. At this time, a high-priority collision warning will be triggered. In order to fundamentally avoid the collision risk, a stop command must be immediately issued to the medical device to control the device to immediately stop all movement actions and avoid safety accidents caused by the rapid movement of dynamic obstacles and the failure to avoid obstacles in time.

[0034] If the detection shows that the rate of change of depth information for each pixel in the image does not exceed the first risk threshold, it indicates that there are no rapidly approaching dynamic obstacles around the medical device, and the device is in a dynamically safe moving environment. In this case, there is no need for an emergency stop, and subsequent steps can be performed to analyze collision risks and plan obstacle avoidance for surrounding static obstacles or slowly moving targets. This preliminary dynamic risk assessment step enables the solution to achieve real-time, rapid identification and emergency handling of dynamic obstacles, significantly improving the safety of medical equipment movement in complex medical scenarios with personnel and equipment flow, and effectively solving the problem of delayed obstacle avoidance for dynamic obstacles in related technologies.

[0035] Step S104: Using the image features of each pixel in the current frame image, identify the type of obstacle corresponding to each pixel. Different types of pixels have different weights, which are used to characterize the degree of collision hazard of the obstacle corresponding to the pixel to the medical device.

[0036] In this embodiment, obstacle type identification based on pixel-level image features can be achieved through various methods such as image feature matching and traditional machine vision classification. To improve the accuracy of obstacle identification in medical scenarios, in some embodiments, a pre-trained deep learning model can be used to segment the current frame image based on the image features such as texture, edge, and color of each pixel, thereby accurately identifying the type of obstacle corresponding to each pixel and achieving refined perception of the surrounding environment of medical equipment. In specific implementation, the UNet semantic segmentation model can be preferred to complete pixel-level identification of obstacle types. The training process of this model can be carried out with medical scenario adaptation as the core. First, targeted preparation of training samples is completed: RGB video streams in various medical scenarios such as hospital operating rooms, corridors, and treatment rooms are collected, and the video is decomposed into single RGB images frame by frame as basic training samples. Then, all samples are labeled at the pixel level, and each pixel is labeled with the corresponding category, including obstacles and background types related to the movement of medical equipment such as medical staff, patients, medical carts, surgical equipment, walls, and ground. To avoid overfitting due to a single scene and to improve the model's generalization ability across different medical scenarios, multi-dimensional data augmentation can be performed on the labeled training samples. For example, the training samples can include several randomly flipped medical equipment environment images, medical equipment environment images with different brightness and / or contrast, medical equipment environment images with different cropping processes, and medical equipment environment images with different shooting angles and / or lighting conditions, simulating the image characteristics under different lighting and shooting perspectives in different medical scenarios. The UNet model can be divided into an encoder and a decoder. The encoder uses feature extraction networks such as ResNet, reducing the image size through multiple downsampling while gradually extracting image features, from initial basic details such as edges and textures to global features such as target contours and the overall scene. The decoder gradually restores the original image size through multiple upsampling. Its core design is "skip connections," which fuse the detailed features extracted by the encoder at the corresponding level with the features at the current level of the decoder, effectively compensating for feature information loss during the upsampling process. This ensures that the final output of the model is a semantic segmentation map with the same size as the input image, achieving accurate classification of each pixel and thus clearly identifying the obstacle type corresponding to each pixel.

[0037] After identifying the obstacle type for each pixel, differentiated weights can be assigned to different types of pixels based on the degree of collision hazard and severity of the consequences for medical equipment. These weights are positively correlated with the degree of collision hazard; the higher the degree of collision hazard, the greater the corresponding weight. In this embodiment, the weight configuration can be flexibly set according to the actual risk characteristics of the medical scenario, and dynamic adjustments are supported for different scenarios. For example, for static background obstacles like walls, relatively small weight values ​​can be configured in normal scenarios, while in high-risk areas such as operating rooms and crowded corridors, where the operating space around walls is limited, the weight value corresponding to walls can be appropriately increased. For medical equipment obstacles such as medical carts and surgical instrument tables, medium to high weight values ​​can be configured, as collisions with these obstacles can easily damage medical equipment. For human obstacles such as medical staff and patients, weight values ​​significantly higher than other types can be configured. Furthermore, for these objects with movement characteristics, an additional weight increment is added on top of the basic weight to prevent collision accidents caused by the instability of human movement. For backgrounds with no collision risk, such as the ground, a weight value of 0 can be configured, excluding them from the collision risk calculation. This differentiated weighting configuration allows subsequent collision risk calculations to align with the safety requirements of medical scenarios, highlighting the risk proportion of high-hazard obstacles and improving the safety and rationality of motion control.

[0038] Step S105: Calculate the collision risk between the obstacle corresponding to each pixel and the medical device in motion, based on the weight of each pixel in the current frame image and the depth information of each pixel.

[0039] The core influencing factors of collision risk mainly include the degree of collision hazard of the obstacle and the distance between the obstacle and the medical equipment. The weight of the pixel can represent the degree of collision hazard of the corresponding obstacle. The higher the weight, the more serious the consequences after the obstacle collides. The depth information of the pixel directly reflects the actual distance between the corresponding obstacle and the medical equipment. The closer the distance, the higher the probability of collision. Therefore, by combining the parameters of these two key dimensions, the collision risk of the obstacle corresponding to a single pixel to the medical equipment can be accurately quantified, providing a quantitative basis for subsequent overall risk assessment and motion control.

[0040] In the specific calculation process, to achieve accurate quantification of collision risk, the depth information of each pixel in the current frame image can first be normalized to obtain the distance parameter corresponding to each pixel. Since the depth information output by the depth estimation model is usually a relative value without physical units, it cannot be directly used for distance-related risk quantification. Therefore, it needs to be converted into a distance parameter of a uniform scale through normalization. This distance parameter is used to characterize the relative distance between the obstacle corresponding to the pixel and the medical equipment. The normalization process can be achieved through the formula... Implementation, in which This is the original depth information of the pixel output by the depth estimation model. The first step is to use a normalization function. This function can employ the same normalization method as the depth estimation model training phase, such as linear normalization or exponential normalization, to ensure a consistent and unbiased mapping between the distance parameters and the original depth information. Secondly, based on the distance parameters corresponding to each pixel, the function value of the collision risk impact function for each pixel is calculated. This collision risk impact function characterizes the degree of influence of the distance between the pixel and the medical device on the collision risk. The distance parameters and the function value of the collision risk impact function are negatively correlated; that is, the smaller the distance parameter (the closer the obstacle is to the medical device), the larger the function value, and the higher the collision risk at the distance level. Finally, the weight corresponding to each pixel is multiplied by the function value of the collision risk impact function corresponding to that pixel to obtain the collision risk between the obstacle and the moving medical device for each pixel. This calculation process can be performed using the formula... This is reflected in For pixels The corresponding collision risk value, The weight of this pixel. Let the collision risk impact function be... This is the distance parameter obtained by normalizing the pixel.

[0041] Specifically, the collision risk impact function can be a linear function, a power function, an exponential function, or other functional forms. However, considering that the collision risk does not change linearly with distance during the movement of medical equipment, but rather exhibits a non-linear characteristic where the risk increases exponentially when the distance is extremely close, in some embodiments, to make the quantification of collision risk more closely reflect the actual collision occurrence patterns, the collision risk impact function can preferably be set as an exponential function. This exponential function allows the impact of distance on collision risk to change gradually when the obstacle is far away from the medical equipment, while the risk value rises rapidly when the obstacle approaches within the safety threshold range. This accurately matches the high-risk warning requirements for close-range obstacles in medical scenarios, making the collision risk calculation results more consistent with actual environmental perception and safety needs.

[0042] Step S106: Detect whether the average collision risk of each pixel in the specified area of ​​the current movement direction of the medical device exceeds the second risk threshold. If yes, proceed to step S107. If no, control the medical device to continue moving along the current movement direction at the current speed.

[0043] The collision risk of medical equipment is mainly concentrated in the current direction of movement. Therefore, the average collision risk can be detected only in a designated area along the current direction of movement. This allows for precise targeting of core risk areas, avoiding meaningless global calculations and improving the response efficiency of motion control. This approach also aligns with the actual obstacle avoidance needs of medical equipment, adjusting only when there is a potential collision risk in the direction of movement. Specifically, the designated area along the current direction of movement in this embodiment can be flexibly set by the user based on the size of the medical equipment, the movement scenario, and obstacle avoidance requirements. For example, it can be set as a fan-shaped area with the direction of movement of the medical equipment as the central axis, or as a regular area such as a rectangle or square. Parameters such as the left and right angles of the fan-shaped area and the width and length of the rectangular area can be adjusted as needed. For instance, in a narrow hospital corridor scenario, the fan angle can be set to 30°, while in an open operating room scenario, the angle can be expanded to 60°, ensuring comprehensive risk detection coverage of the core area along the direction of movement of the medical equipment.

[0044] The average collision risk of each pixel within the designated area is a quantitative representation of the overall collision risk in the current direction of movement of the medical device. The second risk threshold is a pre-defined safety risk threshold, the value of which can be set according to the moving speed, braking performance, and scene safety requirements of the medical device. When the average value is detected to be below the second risk threshold, it indicates that the overall collision risk in the designated area in the current direction of movement of the medical device is within a safe range, with no obstacles to avoid or obstacles that are far away. In this case, there is no need to adjust the movement state of the device, and the medical device can continue to move along the current direction of movement at the current speed. However, when the average value is detected to exceed the second risk threshold, it indicates that there is a certain collision risk in the current direction of movement of the device, and maintaining the original movement state is likely to cause a collision. In this case, the subsequent step S107 needs to be executed to dynamically adjust the moving speed and turning direction of the device to achieve active obstacle avoidance.

[0045] Step S107: Calculate the target speed of the medical device using the average collision risk and the current speed of the medical device. The target speed is negatively correlated with the average collision risk. Also, take the direction from the medical device toward the center pixel of the image region with the smallest average collision risk in the current frame as the target turning direction and control the medical device to move along the target turning direction at the target speed.

[0046] When controlling the motion of medical equipment, the target speed of the medical equipment can be adjusted based on the average collision risk and the current speed of the equipment. The higher the risk, the lower the corresponding target speed, thus allowing sufficient reaction time for steering and obstacle avoidance. Specifically, in this embodiment, this can be achieved using the formula... Calculate the target velocity, where V is the target velocity. Let F be the current speed of the medical device, and F be the average collision risk within the specified area in the current direction of movement of the medical device. As a preset upper limit for collision risk, this formula can achieve negative correlation control between speed and the average collision risk. Users can adjust the formula for calculating the target speed of medical equipment according to actual scenario requirements and equipment performance, as long as the target speed is negatively correlated with the average collision risk.

[0047] While determining the target speed, it is also necessary to plan a safe turning and obstacle avoidance direction for the medical device. In this embodiment, the current frame image can be divided into several image regions of preset shapes. The shape of these image regions can be consistent with the aforementioned specified regions or set separately as needed, such as multiple fan-shaped sub-regions divided at equal angles, multiple rectangular sub-regions divided at equal sizes, etc. Then, the average collision risk of each pixel in each image region is calculated, and the image region with the smallest average collision risk is selected. This region is the optimal obstacle avoidance region for the medical device. Subsequently, the direction from the location of the medical device to the center pixel of the optimal obstacle avoidance region is determined as the target turning direction of the medical device. Finally, the medical device is controlled to move along the target turning direction at the calculated target speed, realizing coordinated obstacle avoidance of speed and direction. For specific methods of adjusting the turning of the medical device, please refer to the appendix. Figure 2 A schematic diagram.

[0048] Furthermore, before calculating the target speed and planning the steering direction, it is considered that there may be extreme situations where there is an extremely high collision risk within the designated area of ​​the medical device's current direction of movement. In such cases, simply slowing down and steering cannot effectively avoid a collision. Therefore, in some embodiments, before calculating the target speed using the average collision risk and the current speed of the medical device, it is also possible to detect whether the collision risk of any pixel within the designated area of ​​the medical device's current direction of movement exceeds a third risk threshold. This third risk threshold is the upper limit of the collision risk, and its value is higher than the second risk threshold. It can be an emergency stop threshold calibrated by actual measurement. If it is detected that the collision risk of any pixel exceeds the third risk threshold, it indicates that there is a high-risk obstacle at close range in the current direction of movement of the medical device, and there is no space for obstacle avoidance adjustment. At this time, it is necessary to immediately control the medical device to stop moving to fundamentally avoid the occurrence of a collision accident. If it is detected that the collision risk of all pixels does not exceed the third risk threshold, then the steps of calculating the target speed and planning the target steering direction in step S107 are continued. Through this hierarchical risk assessment, the motion control of the medical device is made more in line with various risk situations in actual scenarios, further improving the safety and reliability of obstacle avoidance.

[0049] In summary, to provide a clearer understanding of this application, the following example illustrates the overall flow of the motion control method for the medical device provided in this application.

[0050] like Figure 3 As shown, in this embodiment, the overall flow of the motion control method for the medical device includes the following steps: Step 301: Obtain real-time video of the current area of ​​the medical device. Step 302: Calculate the depth information of each pixel in each frame of the video, and the rate of change of the depth information within a preset time window. Step 303: Detect whether the rate of change of depth information of any pixel exceeds the first risk threshold. If yes, control the medical device to stop moving. If no, execute step S304. Step 304: Using the image features of each pixel in the current frame image, identify the type of obstacle corresponding to each pixel. Different types of pixels have different weights, which are used to characterize the degree of collision hazard of the obstacle corresponding to the pixel to the medical device. Step 305: Calculate the collision risk between the obstacle corresponding to each pixel and the medical device in motion, based on the weight of each pixel in the current frame image and the depth information of each pixel. Step 306: Detect whether the collision risk of any pixel in the specified area of ​​the current movement direction of the medical device exceeds the third risk threshold. If yes, control the medical device to stop moving. If no, execute step S307. Step 307: Detect whether the average collision risk of each pixel in the specified area of ​​the current movement direction of the medical device exceeds the second risk threshold. If yes, execute step S308. If no, control the medical device to continue moving along the current movement direction at the current speed. Step S308: Calculate the target speed of the medical device using the average collision risk and the current speed of the medical device. The target speed is negatively correlated with the average collision risk. Also, take the direction from the medical device toward the center pixel of the image region with the smallest average collision risk in the current frame image as the target turning direction, and control the medical device to move along the target turning direction at the target speed.

[0051] Corresponding to the embodiments of the aforementioned motion control methods, this application also provides a motion control device for a medical device. Figure 4 This is a schematic diagram illustrating the structure of a motion control device for a medical device according to an exemplary embodiment of this application. Figure 4 As shown, the device includes: The video acquisition module 401 is used to acquire real-time video of the current area of ​​the medical device; The depth and rate of change calculation module 402 is used to calculate the depth information of the pixels in each frame of the video, as well as the rate of change of the depth information within a preset time window. The rate of change control module 403 is used to control the medical device to stop moving when the rate of change of the depth information of any pixel exceeds a first risk threshold. The obstacle type identification module 404 is used to identify the type of obstacle corresponding to each pixel by utilizing the image features of each pixel in the current frame image when the rate of change of the depth information of each pixel does not exceed the first risk threshold. Different types of pixels have different weights, and the weights are used to characterize the degree of collision hazard of the obstacle corresponding to the pixel to the medical device. The collision risk calculation module 405 is used to calculate the collision risk between the obstacle corresponding to each pixel and the medical device in motion state based on the weight corresponding to each pixel in the current frame image and the depth information of each pixel. The motion control module 406 is used to calculate the target speed of the medical device by using the average collision risk and the current speed of the medical device when the average collision risk of each pixel in a specified area in the current direction of movement of the medical device exceeds a second risk threshold. The target speed is negatively correlated with the average collision risk level. The module also controls the medical device to move at the target speed along the target turning direction by using the direction from the medical device to the center pixel of the image region with the smallest average collision risk in the current frame image as the target turning direction.

[0052] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0053] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0054] Corresponding to the embodiments of the aforementioned motion control method, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor; wherein, when the processor executes the computer program, it implements the steps of the motion control method of the medical device described in any of the above embodiments.

[0055] For example, processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs).

[0056] For example, the memory may include at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0057] Figure 5 This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment of this application. Figure 5 As shown, at the hardware level, the computer device includes a processor 501, an internal bus 502, a network interface 503, memory 504, and non-volatile memory 505, and may also include other hardware required for business operations. One or more embodiments of this application can be implemented in software, for example, the processor 501 reads the corresponding computer program from the non-volatile memory 505 into memory 504 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0058] Corresponding to the embodiments of the aforementioned motion control method, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the motion control method for the medical device described in any of the above embodiments.

[0059] Corresponding to the embodiments of the aforementioned motion control method, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the motion control method for the medical device described in any of the above embodiments.

[0060] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention filed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the foregoing claims.

[0062] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0063] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A motion control method for a medical device, characterized in that, include: Acquire real-time video of the current area of ​​the medical device; Calculate the depth information of each pixel in each frame of the video, and the rate of change of the depth information within a preset time window; If the rate of change of depth information at any pixel exceeds a first risk threshold, the medical device is controlled to stop moving. If the rate of change of depth information at each pixel does not exceed the first risk threshold, perform the following steps; By utilizing the image features of each pixel in the current frame image, the type of obstacle corresponding to each pixel is identified. Different types of pixels have different weights, and the weights are used to characterize the degree of collision hazard posed by the obstacle corresponding to the pixel to the medical device. Based on the weights of each pixel in the current frame image and the depth information of each pixel, the collision risk between the obstacle corresponding to each pixel and the medical device in motion is calculated. If the average collision risk of each pixel in a designated area within the current direction of movement of the medical device exceeds a second risk threshold, the target speed of the medical device is calculated using the average collision risk and the current speed of the medical device. The target speed is negatively correlated with the average collision risk level. Furthermore, the direction from the medical device to the center pixel of the image region with the lowest average collision risk in the current frame image is taken as the target turning direction, and the medical device is controlled to move along the target turning direction at the target speed.

2. The method according to claim 1, characterized in that, Before calculating the target speed of the medical device using the average value of the collision risk and the current speed of the medical device, the method further includes: If the collision risk of any pixel within a designated area in the current direction of movement of the medical device exceeds a third risk threshold, the medical device is controlled to stop moving, wherein the third risk threshold is greater than the second risk threshold.

3. The method according to claim 1, characterized in that, Based on the weights of each pixel in the current frame image and the depth information of each pixel, the collision risk between the obstacle corresponding to each pixel and the medical device in motion is calculated, including: The depth information of each pixel in the current frame image is normalized to obtain the distance parameter corresponding to each pixel. The distance parameter is used to characterize the distance between the pixel and the medical device. Based on the distance parameters corresponding to each pixel, the function value of the collision risk impact function corresponding to each pixel is calculated. The collision risk impact function is used to characterize the degree of influence of the distance between the pixel and the medical device on the collision risk. The distance parameters are negatively correlated with the function value of the collision risk impact function. The weight corresponding to each pixel is multiplied by the function value of the collision risk influence function corresponding to that pixel to obtain the collision risk between the obstacle and the medical device in motion for each pixel.

4. The method according to claim 3, characterized in that, The collision risk impact function is an exponential function.

5. The method according to claim 1, characterized in that, Calculating the depth information of pixels in each frame of the video includes: Using a pre-trained depth estimation model, the depth of pixels in each frame of the video is estimated to obtain the depth information of pixels in each frame of the video.

6. The method according to claim 1, characterized in that, By utilizing the image features of each pixel in the current frame image, the type of obstacle corresponding to each pixel is identified, including: Using a pre-trained deep learning model, the current frame image is segmented based on the image features of each pixel, and the type of obstacle corresponding to each pixel is identified.

7. The method according to claim 6, characterized in that, The training samples for the deep learning model include several randomly flipped images of medical device regions, images of medical device regions with different brightness and / or contrast, images of medical device regions with different cropping processes, and images of medical device regions with different shooting angles and / or lighting conditions.

8. A motion control device for a medical device, characterized in that, include: The video acquisition module is used to acquire real-time video of the current area of ​​the medical device; The depth and rate of change calculation module is used to calculate the depth information of the pixels in each frame of the video, and the rate of change of the depth information within a preset time window. The rate of change control module is used to control the medical device to stop moving when the rate of change of the depth information of any pixel exceeds a first risk threshold. The obstacle type recognition module is used to identify the type of obstacle corresponding to each pixel by utilizing the image features of each pixel in the current frame image, provided that the rate of change of the depth information of each pixel does not exceed the first risk threshold. Different types of pixels have different weights, and the weights are used to characterize the degree of collision hazard of the obstacle corresponding to the pixel to the medical device. The collision risk calculation module is used to calculate the collision risk between the obstacle corresponding to each pixel and the medical device in motion, based on the weight of each pixel in the current frame image and the depth information of each pixel. The motion control module is configured to, when the average collision risk of each pixel within a specified area in the current direction of movement of the medical device exceeds a second risk threshold, calculate a target speed for the medical device using the average collision risk and the current speed of the medical device, wherein the target speed is negatively correlated with the average collision risk; and control the medical device to move at the target speed along the target turning direction, using the direction from the medical device toward the center pixel of the image region with the smallest average collision risk in the current frame image as the target turning direction.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images