Operating room nursing method and system based on image recognition

Abstract

The invention provides an operating room nursing method and system based on image recognition, relates to an image processing technology, and aims to monitor the use conditions of consumables of different types in real time through shooting equipment in an operation process, so that the consumables can be accurately supplied and automatically processed through operation equipment, and the operation efficiency is improved. Therefore, medical staff do not need to be distracted for treating medical waste and supplementing medical supplies any more and can concentrate on surgical operation, and the overall surgical efficiency is improved. In order to distinguish different types of consumables, the consumables of different types can be placed through different areas, so that the consumables are identified in different modes in combination with the areas where the consumables are located, the areas with obstacles can be avoided in the supplementing or discarding operation process, the safety in the operation process is improved, and the operation efficiency is improved. The smooth operation is ensured.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to an operating room nursing method and system based on image recognition. Background Art

[0002] In a modern operating room environment, the efficiency and safety of the surgical process are of utmost importance. During the operation, medical staff not only need to focus on complex and delicate surgical operations but also constantly pay attention to the usage of medical consumables, such as the replenishment of gauze and the handling of contaminated gauze. These additional tasks distract the medical staff's attention and affect the concentration and smoothness of the operation.

[0003] Currently, in the operating room nursing process, there are many problems in the management of medical consumables. On the one hand, in terms of monitoring the usage of consumables, it mainly relies on the naked eye observation and experience judgment of medical staff, and it is impossible to accurately and real-time grasp the status information of different types of consumables, such as the remaining quantity and contamination degree of gauze. This may lead to situations where the shortage of gauze affects the surgical process during the operation, or the contaminated gauze is not processed in time, resulting in an infection risk. On the other hand, in the replenishment and handling operations of consumables, there is a lack of intelligent and automated means. Medical staff need to manually complete the replenishment of gauze and the cleaning of garbage, which not only takes up valuable surgical time but also easily increases the potential risk of surgical infection due to cross-interference with the surgical area during the operation process.

[0004] Therefore, how to achieve real-time and accurate monitoring of the usage of medical consumables and build an automated consumable handling mechanism has become an urgent problem to be solved today. Summary of the Invention

[0005] The present invention provides an operating room nursing method and system based on image recognition, which can achieve real-time and accurate monitoring of the usage of medical consumables and build an automated consumable handling mechanism.

[0006] In the first aspect of the present invention, an operating room nursing method based on image recognition is provided, including: Identifying functional areas according to image data and collecting consumable parameters in each of the functional areas; When the consumable parameters meet the operation trigger conditions of the functional area, identifying the obstacle area in the diagnosis and treatment space, where the operation trigger conditions include a replenishment trigger condition and a discard trigger condition; Real-time planning a dynamic path for the operation device to the functional area and avoiding the obstacle area, and sending a movement instruction to the operation device; Constructing a candidate area based on the end position of the operation device, screening the docking points of the candidate area and sending a synchronous movement instruction to the collaborative device.

[0007] Optionally, in a possible implementation of the first aspect, function partitions are identified based on the image data, and consumable parameters in each of the function partitions are collected, including: Identify the protected area in the image data, and determine each function area according to the area of the pixel points in the protected area located in the preset pixel interval corresponding to each function partition. The function areas include an emergency processing area, an identification processing area, and a reserve area; Collect the contour parameters of the consumables in the emergency processing area, the dirtiness parameters of the consumables in the identification processing area, and the quantity parameters of the consumables in the reserve area. The consumable parameters include contour parameters, dirtiness parameters, and quantity parameters.

[0008] Optionally, in a possible implementation of the first aspect, when the consumable parameters meet the operation trigger conditions of the function partition, identify the obstacle areas in the diagnosis and treatment space. The operation trigger conditions include a replenishment trigger condition and a discard trigger condition, including: When there are contour parameters in the emergency processing area, it is determined that the discard trigger condition of the emergency processing area is met. When the dirtiness parameter in the verification processing area is greater than the first threshold, it is determined that the discard trigger condition of the verification processing area is met. When the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is met; Identify the fixed obstacle areas where the facilities are located and the dynamic obstacle areas where the physicians are located in the diagnosis and treatment space. The obstacle areas include fixed obstacle areas and dynamic obstacle areas. The dynamic obstacle areas are obtained based on the center position of the physician's contour according to a preset radius.

[0009] Optionally, in a possible implementation of the first aspect, the quantity parameter in the reserve area is counted and statistically analyzed through the following steps, including: Obtain the pixel change value of the reserve area in real time. When the pixel change value is greater than or equal to the pixel change threshold, identify the first trajectory corresponding to the hand contour in the reserve area; Determine the original area of the consumables in the reserve area, and obtain the target area generated by the pixel points that are not in the original area and whose pixel values are in the consumable pixel interval; Obtain the second trajectory of the target area, compare the characteristic parameters of the first trajectory and the second trajectory. When the difference between the characteristic parameters is within the deviation range, subtract the reference constant from the quantity parameter. The characteristic parameters include at least speed, acceleration, and curvature.

[0010] Optionally, in a possible implementation of the first aspect, dynamically plan the path of the operation device to the function partition and avoid the obstacle areas in real time, and send a movement instruction to the operation device, including: Generate a straight-line path from the operation device to the functional area. If the straight-line path does not pass through the obstacle area, use the straight-line path as the dynamic path and send a movement instruction to the operation device; If the straight-line path passes through the obstacle area, expand the obstacle area according to the safety distance, generate an avoidance path for the operation device to move tangentially along the obstacle area, and send a movement instruction to the operation device; After the operation device meets the avoidance conditions of the obstacle area, control the operation device to move to the functional area according to the shortest path. The dynamic path includes the avoidance path and the shortest path.

[0011] Optionally, in a possible implementation manner of the first aspect, if the straight-line path passes through the obstacle area, expanding the obstacle area according to the safety distance, generating an avoidance path for the operation device to move tangentially along the obstacle area, and sending a movement instruction to the operation device includes: Add the safety distance to the preset radius of the obstacle area to obtain a safety radius, and perform an expansion process on the obstacle area according to the safety radius; Based on the current position of the operation device, generate multiple tangents tangent to the expanded obstacle area, generate an avoidance path along the direction corresponding to the shortest tangent, and send a movement instruction to the operation device.

[0012] Optionally, in a possible implementation manner of the first aspect, determining the shortest path through the following steps includes: During the process of the operation device moving along the avoidance path, generate a real-time straight-line path from the operation device to the functional area; When there is no intersection between the real-time straight-line path and the obstacle area, determine that the avoidance condition is met, and use the corresponding real-time straight-line path as the shortest path.

[0013] Optionally, in a possible implementation manner of the first aspect, constructing a candidate area according to the end position of the operation device, screening the docking points of the candidate area, and sending a synchronous movement instruction to the collaborative device includes: When the operation device meets the discard trigger condition, obtain the end position of the operation device when the pick-up operation is completed; With the end position as the center, determine multiple fan-shaped candidate areas according to the division angle. In the candidate areas, determine multiple reference points in the radial direction according to half of the length of the collaborative device; Determine the reference point that meets the positioning conditions of the collaborative device and is the closest to the end position as the docking point, and send a synchronous movement instruction to control the collaborative device to move to the docking point.

[0014] Optionally, in a possible implementation manner of the first aspect, determining the reference point that meets the positioning condition of the collaborative device and is closest to the end position as the docking point includes: Taking each of the reference points located outside the obstacle area as the center, determining a positioning area corresponding to the device specification of the collaborative device. When the positioning area is not located in the obstacle area, it is determined that the positioning condition is met, and the reference point closest to the end position is determined as the docking point.

[0015] In a second aspect of the present invention, there is provided an operating room nursing system based on image recognition, including: A zoning module, configured to identify functional zones according to image data and collect consumable parameters in each of the functional zones; A determination module, configured to identify an obstacle area in the diagnosis and treatment space when the consumable parameters meet the operation trigger conditions of the functional zone, where the operation trigger conditions include a replenishment trigger condition and a discard trigger condition; A planning module, configured to real-time plan a dynamic path for the operating device to the functional zone and avoid the obstacle area, and send a movement instruction to the operating device; A collaboration module, configured to construct a candidate area according to the end position of the operating device, screen the docking point of the candidate area, and send a synchronous movement instruction to the collaborative device.

[0016] The beneficial effects of the present invention are as follows: 1. By installing cameras at different positions in the operating room to collect image data in real time and using image recognition technology, the present invention can accurately identify functional zones divided based on surgical cloth markings, including garbage fixed points, clean item fixed points, and temporary points. For different functional zones, consumable parameters can be accurately collected, realizing real-time, comprehensive, and accurate monitoring of the usage of consumables, providing a reliable basis for subsequent processing and replenishment decisions, greatly improving the efficiency and refinement level of operating room consumable management, and ensuring the smooth progress of surgeries.

[0017] 2. When the consumable parameters meet the operation trigger conditions of the corresponding functional zone, the system can automatically identify obstacle areas in the diagnosis and treatment space, including fixed obstacle areas and dynamic obstacle areas. For contaminated gauze that meets the discard trigger condition, the operating device can real-time plan a dynamic path to avoid the obstacle area, go to the target position to pick it up, and accurately dock with the collaborative device to efficiently transfer the contaminated gauze to the garbage disposal area. For the case where the replenishment trigger condition is met, the operating device can plan a path to go to the replenishment item storage point to suck gauze and send it to the clean item fixed point. This intelligent replenishment and safety processing mechanism reduces the workload of medical staff in handling medical waste and replenishing medical supplies, reduces the risk of surgical infection, and improves the safety and smoothness of the surgical process.

[0018] 3. In the docking process between the operating device and the collaborative device, the present invention takes the end position of the operating device when it completes the picking operation as the center, determines multiple fan-shaped candidate areas according to the divided angle, and determines reference points within the candidate areas based on half of the length of the collaborative device. By comprehensively considering factors such as the trolley size, motion performance, and path planning complexity, the divided angle is set to ensure that suitable docking positions can be screened out. Then, the reference point that meets the positioning conditions of the collaborative device and is closest to the end position of the operating device is determined as the docking point, and a synchronous motion instruction is sent to control the collaborative device to move to the docking point. This realizes the precise docking between the operating device and the collaborative device, optimizes the device path planning, improves the docking efficiency, ensures the smooth progress of the garbage disposal process, helps maintain the cleanliness and hygiene of the operating room, and provides strong support for the smooth conduct of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present invention; Figure 2 is a schematic flow chart of a method for operating room nursing based on image recognition provided by an embodiment of the present invention; Figure 3 is a schematic structural diagram of a system for operating room nursing based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] See Figure 1, which is a schematic diagram of an application scenario provided by an embodiment of the present invention. In this solution, during the surgical process, the usage of different types of consumables can be monitored in real time through a shooting device, so that the consumables can be accurately replenished and automatically processed by an operating device, enabling medical staff to no longer be distracted by dealing with medical waste and replenishing medical supplies, and thus be able to focus on the surgical operation and improve the overall surgical efficiency. In order to distinguish different types of consumables, different types of consumables will be placed in different areas in this solution, and then different identification operations will be performed on them in combination with their locations. Moreover, during the replenishment or discarding operation process, areas with obstacles will be avoided to improve the safety during the surgical process and ensure the smooth progress of the surgery. Among them, the shooting device can be a camera, and the operating device can be a robotic arm with a vacuum suction head at the end that can suck up lightweight garbage such as gauze. The consumables can be medical supplies with consumable properties that are easy to suck up, such as gauze. In the following content, the gauze is used as the consumable for illustration.

[0022] See Figure 2 , which is a schematic flowchart of a method for operating room nursing based on image recognition provided by an embodiment of the present invention. Figure 2 The execution subject of the method shown can be a software and / or hardware device. The execution subject of this application can include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, the user equipment can include but is not limited to computers, smartphones, personal digital assistants (Personal Digital Assistant, abbreviated as: PDA), and the above-mentioned electronic devices, etc. The network equipment can include but is not limited to a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which is composed of a group of loosely coupled computers to form a super virtual computer. This embodiment does not make any restrictions on this. It includes steps S1 to S4, specifically as follows: S1, Identify functional areas according to the image data and collect the consumable parameters in each of the functional areas.

[0023] During the surgical process, accurately managing consumables is crucial for ensuring the smooth progress of the surgery and maintaining the hygiene of the operating room. By identifying the functional areas and collecting the consumable parameters, the status of the consumables can be grasped in real time, providing a basis for subsequent reasonable handling of contaminated consumables and timely replenishment of clean consumables, thereby optimizing the surgical process and improving the surgical safety and efficiency.

[0024] In practical applications, the image data of the operating room can be collected in real time through cameras installed at different positions in the operating room.

[0025] Among them, functional zoning refers to different use areas divided based on the markings on the surgical cloth, including fixed garbage points for centralized placement of used garbage gauze, fixed clean item points for storage of clean gauze to be used, and temporary points set by doctors on demand for temporary placement of gauze that is specially contaminated or requires special treatment. Consumables parameters refer to the relevant status data for consumables. In garbage-related areas, it refers to the contamination degree parameters of gauze and the contour parameters of garbage gauze in the corresponding area, and in clean item fixed points, it refers to the quantity parameters of gauze.

[0026] Accurate functional zoning identification and consumable parameter collection can clearly understand the distribution and status of gauze. Doctors can focus on surgical operations without being distracted by gauze management details; the system can make accurate decisions based on parameters, such as determining whether gauze needs to be discarded or replenished, providing a reliable basis for subsequent processes and improving the overall efficiency of surgery and the refinement of management.

[0027] In some embodiments, step S1 can be implemented by the following steps: Identify the protection area in the image data, and determine each functional area according to the area where the pixel points in the protection area are located in the preset pixel interval corresponding to each functional area, the functional area includes an emergency treatment area, an identification treatment area and a reserve area; collect contour parameters of consumables in the emergency treatment area, contamination parameters of consumables in the identification treatment area and quantity parameters of consumables in the reserve area, the consumable parameters include contour parameters, contamination parameters and quantity parameters.

[0028] Among them, the protection area is the area where the surgical cloth is located. The emergency treatment area is a fixed garbage point for consumables that need to be processed immediately, such as heavily contaminated gauze that has been exposed to special pathogens or critical surgical sites. Rapid processing of these items can effectively prevent and control infection risks. The identification and processing area is a temporary point for placing used consumables that need to be further judged to determine the degree of dirtiness to decide whether to discard them. The reserve area is a fixed point for storing clean items for spare clean consumables, such as unused gauze, to ensure sufficient supply of materials during the operation. Contour parameters refer to parameters such as the contour features of consumables in the emergency treatment area, such as contaminated gauze. Turbidity parameters refer to parameters reflecting the degree of dirtiness obtained by image recognition technology for consumables in the identification and processing area. Quantity parameters refer to the quantity information of consumables in the reserve area.

[0029] In practical applications, before surgery, medical staff can divide areas on the surgical drape according to the type and requirements of the surgery using medical-grade Velcro. For example, near the operating table and in a convenient location for quick access on the surgical drape, a rush treatment area can be marked out with Velcro; an identification and treatment area can be demarcated in a slightly farther but convenient location for centralized processing; a reserve area can be set near the corresponding special storage rack on the surgical drape. Different areas can be demarcated using medical-grade Velcro with different preset pixel ranges, so that the positions of each area can be quickly identified based on the collected image data. Using the surgical drape as a protection area, combined with image recognition and manually adjustable Velcro zoning, can flexibly adapt to different operating room layouts and surgical requirements.

[0030] Since the consumable attributes of different areas are different, the consumable parameters collected for different areas can also be different. In the rush treatment area, when a contaminated gauze in contact with a special pathogen is placed in this area, the outline of the gauze can be identified through an edge detection algorithm. In the identification and treatment area, when a used gauze is placed in, image segmentation technology can be used to separate the stain from the background color of the gauze, analyze the parameters of the stain in the HSV color model, and obtain the fouling parameters of the gauze by synthesizing parameters such as hue, saturation, lightness, and the proportion of the stain-covered area to the total area of the gauze. A corresponding table of multiple parameters and fouling degrees can be set in advance by combining multiple parameters, and then the specific fouling parameters can be determined by traversing the data in the table. This table can be pre-set by the staff according to the actual situation. For example, multiple intervals can be set for each parameter, and different interval combinations correspond to different fouling degrees. In the reserve area, the camera can capture the situation of gauze being taken from the position corresponding to the reserve area on the surgical drape, so as to determine the remaining amount of the consumable in combination with the usage situation of the consumable.

[0031] S2, when the consumable parameters meet the operation trigger conditions of the functional area, identify the obstacle area in the diagnosis and treatment space, and the operation trigger conditions include a replenishment trigger condition and a discard trigger condition.

[0032] It can be understood that timely handling of contaminated gauze can reduce the risk of infection and ensure the hygiene of the operating room; timely replenishment of clean gauze can prevent the surgery from being interrupted due to a shortage of gauze. Identifying the obstacle area is to plan a safe path for the robotic arm to avoid interfering with the surgical operation during the execution of tasks, ensure the safety and order of the entire operating room environment, and guarantee the smooth progress of the surgery.

[0033] The operation trigger condition is the judgment basis for deciding whether to discard or replenish the gauze. The discard trigger condition is that when the contamination degree of the gauze in the temporary point reaches the preset standard or there is gauze in the garbage fixed point; the replenishment trigger condition is that the number of gauzes at the clean item fixed point is lower than the threshold. The obstacle area is an area in the surgical area set as a non-accessible space, that is, an area that the robotic arm cannot enter during movement, to avoid colliding with the personnel and equipment during the surgery and ensure the safety of the surgery. The diagnosis and treatment space is the operating room.

[0034] Based on the above embodiments, the specific implementation manner of step S2 may be as follows: S21. When there are contour parameters in the emergency processing area, it is determined that the discard trigger condition of the emergency processing area is met; when the dirt parameter in the verification processing area is greater than the first threshold, it is determined that the discard trigger condition of the verification processing area is met; when the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is met.

[0035] During the efficient operation of the operating room, timely and accurately determining the processing and replenishment operations of consumables is crucial for ensuring the smooth progress of the operation, maintaining the safety and hygiene of the operating room environment, and rationally utilizing resources. Clearly defining the operation trigger conditions for each functional area and making appropriate decisions can avoid problems such as surgical delays and increased infection risks caused by delayed or incorrect human judgment.

[0036] Since the emergency processing area is used to place consumables that need to be processed immediately, when there are contour parameters in the emergency processing area, it can be determined that the corresponding discard trigger condition is met. The first threshold is a preset critical value used to measure whether the dirt parameter of the consumables in the verification processing area reaches the discard standard. Therefore, when the dirt parameter in the verification processing area is greater than the first threshold, it can be determined that the corresponding discard trigger condition is met. The second threshold is a preset critical value used to judge whether the quantity of consumables in the reserve area is insufficient and needs to be replenished. Therefore, when the quantity parameter in the reserve area is less than the second threshold, it can be determined that the corresponding replenishment trigger condition is met.

[0037] In some embodiments, the quantity parameter in the reserve area can be counted and statistically analyzed through the following steps: S211. Real-time obtain the pixel change value of the reserve area. When the pixel change value is greater than or equal to the pixel change threshold, identify the first trajectory corresponding to the hand contour in the reserve area.

[0038] It can be understood that accurately and real-time counting the number of gauzes in the reserve area is crucial for ensuring the smooth progress of the operation. By real-time obtaining the pixel change value of the reserve area, the image changes caused by operations such as doctors taking gauzes can be captured in a timely manner, and then the hand contour trajectory can be identified, laying a foundation for accurately judging whether the gauzes are taken and the quantity taken subsequently, ensuring the dynamic monitoring of the gauze quantity, and avoiding affecting the material supply for the operation due to inaccurate gauze quantity statistics.

[0039] The pixel change value refers to the change amount of the pixel values of the reserve area image at different times in the image sequence. The pixel change threshold is a preset value used to measure whether the pixel change value is significant. When the pixel change value of a certain area is greater than or equal to the threshold, it is considered that a meaningful action has occurred in that area, such as a doctor picking up a gauze or other operations. In the image, the contour of the doctor's hand that can be identified through image processing technology can be used to further analyze the movement trajectory of the hand by extracting the hand contour. The first trajectory is the movement trajectory corresponding to the contour when the doctor's hand moves in the reserve area. This trajectory is obtained by continuously recording and analyzing the positions of the hand contour in the image sequence.

[0040] S212, determine the original area of the consumable in the reserve area, and obtain the target area generated by the pixel points that are not in the original area and whose pixel values are within the pixel range of the consumable.

[0041] After identifying the hand contour trajectory, determining the original area and the target area of the consumable in the reserve area is to more accurately judge whether the gauze has been picked up and which specific gauze has been picked up. By comparing the changes between the original area and the target area, the position movement of the gauze can be accurately tracked, thereby providing a more reliable basis for quantity statistics.

[0042] The original area is the position area where the consumable is initially located in the reserve area. Before the operation starts or after the gauze is replenished, the gauzes are neatly placed at specific positions in the reserve area, and these positions constitute the original area. The target area is the area composed of the pixel points that are not in the original area and whose pixel values are within the pixel range of the consumable obtained through image processing technology. When the gauze is picked up, its position changes, forming a new area, that is, the target area. The pixel range of the consumable is a preset value range corresponding to the pixel values of the consumable in the image.

[0043] S213, obtain the second trajectory of the target area, compare the characteristic parameters of the first trajectory and the second trajectory. When the difference between the characteristic parameters is within the deviation range, subtract the reference constant from the quantity parameter. The characteristic parameters at least include speed, acceleration, and curvature.

[0044] The second trajectory is the movement trajectory formed when the target area moves in the image sequence. This trajectory is obtained by recording and analyzing the positions of the target area in consecutive image frames. Feature parameters are parameters used to describe the characteristics of the trajectory, including at least speed, acceleration, and curvature. Speed reflects how fast the trajectory moves, acceleration represents the rate of change of speed, and curvature describes the degree of bending of the trajectory. By comparing these parameters, it can be determined whether the movement of the target area is related to hand movement. Deviation range: The range of allowable differences in feature parameters set in advance. When the difference in feature parameters between the first trajectory and the second trajectory is within this range, it is considered that the movement of the target area is consistent with hand movement, that is, the gauze is picked up. The reference constant is a fixed value used to adjust the quantity parameter after it is determined that the gauze is picked up. Usually, the reference constant is 1, indicating that for each piece of gauze picked up, the quantity parameter is decreased by 1.

[0045] Obtain the second trajectory of the target area, while the first trajectory of the hand contour is known. Calculate and compare the feature parameters such as speed, acceleration, and curvature of the first trajectory and the second trajectory. Assume that the speed of the first trajectory is 5 pixel units per second, the acceleration is 2 pixel units per second squared, and the curvature is 0.1; the speed of the second trajectory is 4.8 pixel units per second, the acceleration is 1.9 pixel units per second squared, and the curvature is 0.11. The differences in these feature parameters (the speed difference is 0.2, the acceleration difference is 0.1, and the curvature difference is 0.01) are within the preset deviation range (assuming the speed deviation range is ±0.5, the acceleration deviation range is ±0.3, and the curvature deviation range is ±0.2), it is determined that the gauze is picked up. At this time, the system subtracts the reference constant 1 from the quantity parameter. If the original quantity parameter of the gauze in the storage area is 20, it becomes 19 after subtracting 1.

[0046] Through the steps of trajectory comparison and quantity statistics, the system can accurately determine whether the gauze is picked up and update the quantity parameter of the gauze in the storage area in a timely manner. This accurate quantity statistical method ensures the real-time and accurate grasp of the gauze quantity by the operating room nursing system, helps to reasonably arrange the replenishment and use of gauze, avoids problems with the supply of surgical supplies caused by inaccurate gauze quantity, and improves the working efficiency of the operating room.

[0047] S22. Identify the fixed obstacle area where the facilities are located and the dynamic obstacle area where the physician is located in the diagnosis and treatment space. The obstacle area includes the fixed obstacle area and the dynamic obstacle area. The dynamic obstacle area is obtained based on the center position of the physician's contour according to a preset radius.

[0048] During the operation, in order to ensure that the automated device can move safely and efficiently in the operating room and perform tasks such as picking up and placing consumables, it is crucial to accurately identify the obstacle areas in the operating room. Defining the fixed obstacle area and the dynamic obstacle area can provide a key basis for the device path planning, prevent the device from colliding with obstacles, and ensure the normal order of the operation and the safety of personnel and equipment.

[0049] Among them, the fixed obstacle area is the space area occupied by the fixed facilities in the operating room, such as the operating table, medical instrument cabinet, anesthesia equipment, etc. These facilities are relatively fixed in position and will not move randomly during the operation, and the areas where they are located constitute the fixed obstacle range for the movement of the automated device.

[0050] The dynamic obstacle area is a movable area delimited according to a preset radius with the central position of the physician's contour as the reference point. Since the physician will move continuously during the operation, the space within a certain range around him poses a collision risk to the automated device. Therefore, this area can be set as the dynamic obstacle area, and the setting of the radius needs to comprehensively consider factors such as the physician's activity range and the safe distance of the device. The preset radius is the radius value preset for determining the range of the dynamic obstacle area.

[0051] Accurate identification of the obstacle area provides a strong guarantee for the safe movement of the automated device in the complex environment of the operating room. By generating the dynamic obstacle area in real time by monitoring the physician's position and determining the fixed obstacle area, the occurrence of device collision accidents is effectively avoided, ensuring the continuity and safety of the operation. At the same time, the working efficiency of the automated device is also improved, enabling it to assist the operating room nursing work more stably and efficiently.

[0052] S3. Real-time plan the dynamic path of the operation device to the functional area and avoid the obstacle area, and send a movement instruction to the operation device.

[0053] The robotic arm needs to safely and efficiently complete tasks such as picking up contaminated gauze and placing clean gauze in the complex environment of the operating room. Planning the dynamic path and avoiding the obstacle area can ensure that the robotic arm accurately reaches the target position, avoid the collision risk, save time, improve the working efficiency, and ensure the smoothness of the gauze handling link during the operation.

[0054] A dynamic path refers to a moving route planned based on the current position of the robotic arm, the position of the target functional area, and the real-time state of the obstacle area, and this route will be adjusted due to environmental changes. When the discard trigger condition is met, a path for the operating device to pick up garbage from the current position to a temporary point or a fixed garbage point can be planned, and then a movement instruction is sent to control the operating device to move to the target position along the corresponding path to pick up the garbage. When the replenishment trigger condition is met, the operating device can first be made to go to the point where the replenishment items are stored to suck up gauze, and then a path from the position of the operating device to the fixed point of the clean item is planned, and a movement instruction is sent to control the operating device to go to the corresponding target position to place the consumables.

[0055] Real-time dynamic path planning enables the robotic arm to respond flexibly in a complex and ever-changing operating room environment. It can safely avoid the obstacle area, effectively avoid collisions with the surgical area, and ensure surgical safety.

[0056] Based on the above embodiments, the specific implementation manner of step S3 can be: S31. Generate a straight-line path from the operating device to the functional area. If the straight-line path does not pass through the obstacle area, then use the straight-line path as the dynamic path and send a movement instruction to the operating device.

[0057] In the operating room environment, the operating device needs to quickly and accurately reach the functional area to perform tasks, such as picking up contaminated gauze or replenishing clean gauze. Generating a straight-line path is the most direct and efficient path planning method. By first checking whether the straight-line path passes through the obstacle area, it can quickly determine whether this simple and efficient path can be adopted, thus saving path planning time and improving the working efficiency of the operating device.

[0058] S32. If the straight-line path passes through the obstacle area, expand the obstacle area according to the safety distance, generate an avoidance path for the operating device to move tangentially along the obstacle area, and send a movement instruction to the operating device.

[0059] When the straight-line path cannot meet the safety requirements, that is, it passes through the obstacle area, in order to ensure the normal progress of the surgery and the safety of personnel and equipment, it is necessary to re-plan the path of the operating device so that it can bypass the obstacle area and safely reach the target functional area. By generating an avoidance path, it not only ensures that the operating device can complete the task, but also avoids interference with the surgical process and potential collision risks.

[0060] The safety distance is the minimum distance that the operating device needs to maintain from the boundary of the obstacle area when bypassing the obstacle area to ensure that the device will not accidentally approach the doctor or collide with fixed equipment. The avoidance path is the path that the operating device re-plans to move tangentially along the boundary of the expanded obstacle area to avoid the obstacle when the straight-line path passes through the obstacle area.

[0061] In some embodiments, an avoidance path can be generated through the following steps: Add a safety distance to the preset radius of the obstacle area to obtain a safety radius, and expand the obstacle area according to the safety radius; taking the current position of the operating device as a reference, generate multiple tangent lines tangent to the expanded obstacle area, generate an avoidance path along the direction corresponding to the shortest tangent line, and send a movement instruction to the operating device.

[0062] It can be understood that in the complex environment of the operating room, ensuring the safety distance between the operating device and the obstacle area is crucial. By adding a safety distance to the preset radius of the obstacle area to obtain a safety radius and expanding the obstacle area accordingly, sufficient safety space can be reserved when the device plans an avoidance path, effectively reducing the risk that the device accidentally approaches the obstacle during movement, interfering with the surgery or causing a collision, and ensuring the smooth progress of the surgery and the safety of personnel and equipment.

[0063] By reasonably expanding the range of the obstacle area, more reliable safety protection is provided for the path planning of the operating device. A larger safety area can effectively accommodate the possible deviations of the device during movement, reducing the possibility of the device approaching or colliding with the obstacle dangerously, ensuring the safety of personnel and equipment in the operating room, and maintaining the stable progress of the surgery.

[0064] Taking the current position of the operating device as a reference, generating multiple tangent lines tangent to the expanded obstacle area, and selecting the path along the direction corresponding to the shortest tangent line as the avoidance path can enable the operating device to reach the target with a relatively short travel distance under the premise of meeting safety requirements, improve the operating efficiency of the device, and reduce the time waste caused by circuitous paths.

[0065] S33, after the operating device meets the avoidance conditions of the obstacle area, control the operating device to move to the functional area along the shortest path, and the dynamic path includes the avoidance path and the shortest path.

[0066] After the operating device bypasses the obstacle area, continuing to move along the avoidance path may increase unnecessary travel distance and time consumption. When the operating device meets the avoidance conditions of the obstacle area, switching to the shortest path to move to the functional area can improve the speed of the device reaching the target area under the premise of ensuring safety and improve the working efficiency of the operating device.

[0067] The avoidance condition of the obstacle area means that the operating device successfully bypasses the obstacle area, and the straight-line path between it and the target functional area no longer intersects with the obstacle area. The shortest path is the shortest straight-line connection path from the current position to the target functional area after the operating device meets the avoidance conditions of the obstacle area.

[0068] In some embodiments, the shortest path can be determined through the following steps: During the movement of the operation device along the avoidance path, a real-time straight-line path from the operation device to the functional area is generated; when there is no intersection between the real-time straight-line path and the obstacle area, it is determined that the avoidance condition is satisfied, and the corresponding real-time straight-line path is used as the shortest path.

[0069] During the movement of the operation device along the avoidance path, the situation in the operating room may change. For example, the movement of the physician may cause a change in the range or position of the obstacle area, or other temporary obstacles may appear. By continuously generating the real-time straight-line path from the operation device to the functional area, it is possible to evaluate in real time whether the device can directly go to the target functional area, providing a basis for judging whether the avoidance condition is satisfied and switching to the shortest path, thereby dynamically optimizing the movement path of the device and improving work efficiency. The real-time straight-line path is the straight-line connection path calculated based on the current position of the device and the position of the target functional area at each moment during the movement of the operation device along the avoidance path. This path will be updated in real time as the position of the device changes.

[0070] When the operation device moves along the avoidance path, once the real-time straight-line path between it and the target functional area no longer intersects with the obstacle area, it means that the device can safely and more efficiently go directly to the target. At this time, it is determined that the avoidance condition is satisfied and switched to the shortest path, which can reduce unnecessary detours of the device, save time and energy, improve the overall work efficiency of the operation device in the operating room environment, and ensure the timely supply and handling of surgical supplies.

[0071] S4. Construct a candidate area based on the end position of the operation device, screen the docking points in the candidate area, and send a synchronous movement instruction to the collaborative device.

[0072] During the surgical process, timely cleaning of the used contaminated gauze is crucial for maintaining the cleanliness of the operating room and reducing the risk of infection. After the robotic arm picks up the contaminated gauze, it needs to be accurately docked with the collaborative device to ensure that the contaminated gauze can be accurately and efficiently transferred to the waste treatment area, avoiding garbage spillage or processing delays caused by docking errors, thereby ensuring the safety of the operating room environment and the smoothness of the surgical process.

[0073] Among them, the candidate area is a plurality of fan-shaped areas divided with the end position of the robotic arm after grasping the contaminated gauze as the center. Its function is to provide a range for screening suitable positions for docking with the collaborative device to facilitate accurate positioning. The docking point is the point that meets the conditions for transferring the contaminated gauze and is the closest to the end position of the robotic arm screened from the candidate area. The collaborative device is a movable trolley used to transport the contaminated gauze. The synchronous movement instruction is an instruction sent to the movable trolley, which includes the docking point coordinates and related motion parameters such as driving speed and direction. This instruction is used to coordinate the actions of the movable trolley and the robotic arm to ensure that the two can cooperate precisely during the docking process to achieve seamless handover of the contaminated gauze.

[0074] Candidate areas are constructed and precise screening and docking points are combined with synchronous motion instructions to achieve efficient cooperation between the operating device and the movable cooperative device during the garbage throwing process. This ensures that the contaminated gauze is accurately handed over to the trolley, avoids the risk of garbage spilling, improves the garbage cleaning efficiency, promptly removes the contaminated gauze from the surgical area, reduces the infection risk, and guarantees the continuity of the surgical process and the hygienic safety of the operating room environment.

[0075] Based on the above embodiments, the specific implementation manner of step S4 can be: S41. When the operating device meets the discard trigger condition, obtain the end position of the operating device when it completes the picking operation.

[0076] When the operating device meets the discard trigger condition, that is, after detecting the item to be discarded and completing the picking operation, accurately obtaining its end position is the basis for precise docking with the cooperative device. Only by clarifying the end position of the operating device can subsequent operations such as candidate area division and docking point determination be carried out centered around it, so as to ensure that the item can be accurately transferred to the cooperative device and an efficient garbage disposal process can be achieved. The end position is the coordinate position of the part of the operating device used for picking up the item in the operating room space when it completes the picking operation.

[0077] S42. Centered on the end position, determine multiple fan-shaped candidate areas according to the division angle. In the candidate areas, determine multiple reference points in the radial direction according to half of the length of the cooperative device.

[0078] Determining multiple fan-shaped candidate areas centered on the end position of the operating device and determining reference points in the candidate areas according to half of the length of the cooperative device is to screen out suitable positions in the space around the operating device for docking with the cooperative device. In this way, possible docking positions can be comprehensively covered, the probability of finding a suitable docking point can be increased, and the length of the cooperative device is considered to ensure the feasibility and stability of the docking.

[0079] Among them, the division angle is an angle value set for dividing the circumference centered on the end position of the operating device. Through this angle division, the circumference can be divided into multiple sector areas, facilitating the search for suitable docking positions in different directions. The division angle can be set in combination with the specifications of the trolley. The length and width of the trolley have a decisive impact on its docking space. In the radial direction, usually half of the trolley's length is used as the interval to determine the reference points. In the circumferential direction, the width of the trolley plays a key role. If the trolley is wider, to ensure that the rectangular area adapted to the trolley size constructed with the discrete reference points as the center can be completely within the sector area, the angle of the sector area must be increased accordingly. Therefore, a corresponding table of the width and angle values of multiple trolleys can be set in advance, and the current division angle can be determined through the table.

[0080] For example, suppose there is a trolley with a width of 1 meter, and docking planning is carried out on a circumference with a radius of 2 meters. To ensure that the trolley has sufficient space to complete the docking action, the sector angle may need to be set in the range of 30° - 45°, so the corresponding division angle can be a value within 30° - 45°. Such a setting can enable the trolley to smoothly find a suitable docking point within this sector area, avoiding docking failures or collisions with surrounding objects due to insufficient space.

[0081] The candidate areas are multiple sector areas centered on the end position of the operating device and determined according to the division angle. The reference points are multiple points determined in the candidate areas in the radial direction according to half of the length of the collaborative device.

[0082] S43. Determine the reference point that meets the positioning conditions of the collaborative device and is closest to the end position as the docking point, and send a synchronous movement instruction to control the collaborative device to move to the docking point.

[0083] Determine the reference point that meets the positioning conditions of the collaborative device and is closest to the end position of the operating device as the docking point from multiple reference points, and send a synchronous movement instruction to control the collaborative device to move to this docking point, in order to achieve the precise docking of the operating device and the collaborative device, ensuring that items can be safely and efficiently transferred from the operating device to the collaborative device. Selecting the closest reference point can reduce the moving distance and time of the collaborative device, improve the docking efficiency, and at the same time meeting the positioning conditions can ensure the accuracy and stability of the docking.

[0084] Among them, the positioning conditions of the collaborative device are conditions set for judging whether a reference point is suitable as a docking point. These conditions can include whether there is sufficient space at the location of the reference point for the collaborative device to dock and whether other obstacles in the operating room are avoided.

[0085] In some embodiments, the docking point can be determined through the following steps: Taking each of the reference points located outside the obstacle area as the center, a positioning area corresponding to the device specifications of the collaborative device is determined. When the positioning area is not located in the obstacle area, it is determined that the positioning condition is satisfied, and the reference point closest to the end position is determined as the docking point.

[0086] By determining a positioning area that matches the specifications of the collaborative device with a reference point as the center and judging whether it is outside the obstacle area, the docking position of the collaborative device can be ensured to be safe. Then, by selecting the reference point closest to the end position of the operating device as the docking point, the docking path can be shortened to the greatest extent, the docking efficiency can be improved, the waste treatment process can be ensured to be smooth, and the safety and order of the operating room environment can be maintained.

[0087] Among them, the positioning area is a specific area determined according to the device specifications of the collaborative device with a reference point located outside the obstacle area as the center. The size and shape of this area are adapted to the size of the collaborative device, and it is the key area for judging whether the collaborative device can dock safely.

[0088] For example, after preliminary screening, some reference points located outside the obstacle area are obtained. It is known that the length of the mobile cart is 2 meters and the width is 1.5 meters. Taking reference point A as the center, a positioning area is determined according to the device specifications of the cart. This positioning area is a rectangular area with a length of 2 meters (the same as the length of the cart) and a width of 1.5 meters (the same as the width of the cart) centered on reference point A. By calculating the spatial coordinates and comparing with the range of the obstacle area, it is judged whether this positioning area is within the obstacle area. When the positioning area is completely outside the obstacle area, the positioning condition is satisfied, and the corresponding reference point is determined as the docking point.

[0089] Through the above steps, the docking point most suitable for the collaborative device to dock can be accurately found from numerous reference points. Ensure the safety of the collaborative device during the docking process, avoid collisions with personnel or equipment in the obstacle area, and ensure the normal progress of the operation. At the same time, by selecting the reference point closest to the end position of the operating device as the docking point, the docking path is shortened to the greatest extent, the movement time and energy consumption of the collaborative device are reduced, the docking efficiency is improved, the waste treatment process is made more efficient and smooth, which helps to maintain the cleanliness and hygiene of the operating room and provides strong support for the smooth progress of the operation.

[0090] See Figure 3 , which is a schematic structural diagram of an operating room nursing system based on image recognition provided by an embodiment of the present invention. The operating room nursing system based on image recognition includes: A zoning module for identifying functional zones according to image data and collecting consumable parameters in each of the functional zones; A determination module, configured to identify an obstacle area in the diagnosis and treatment space when the consumable parameter meets the operation trigger condition of the functional partition, where the operation trigger condition includes a replenishment trigger condition and a discard trigger condition; A planning module, configured to plan a dynamic path of the operation device to the functional partition and avoid the obstacle area in real time, and send a movement instruction to the operation device; A cooperation module, configured to construct a candidate area according to the end position of the operation device, screen the docking points of the candidate area, and send a synchronous movement instruction to the cooperation device.

[0091] Figure 3 The device of the illustrated embodiment can correspondingly be used to execute Figure 2 the steps in the illustrated method embodiment, and its implementation principle and technical effect are similar, and will not be described in detail here.

[0092] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operating room nursing method based on image recognition, characterized in that: include: Identify functional zones according to image data, and collect consumable parameters in each functional zone; When the consumables parameter meets the operation triggering condition of the functional partition, identifying the obstacle area in the diagnosis and treatment space, the operation triggering condition includes a replenishment triggering condition and a discarding triggering condition; Plan in real time the dynamic path of the operating equipment to the functional area and avoid the obstacle area, and send movement instructions to the operating equipment; A candidate area is constructed according to the end position of the working equipment, the docking points in the candidate area are screened and a synchronous motion instruction is sent to the cooperative equipment.

2. The method according to claim 1, characterized in that: Identifying functional zones according to image data and collecting consumable parameters in each functional zone include: Identify the protection area in the image data, and determine each functional area according to the area where the pixel points in the protection area are located in the preset pixel interval corresponding to each functional area, wherein the functional area includes an emergency processing area, an identification processing area, and a reserve area; The profile parameters of the consumables in the emergency treatment area, the contamination parameters of the consumables in the identification treatment area and the quantity parameters of the consumables in the reserve area are collected, and the consumable parameters include profile parameters, contamination parameters and quantity parameters.

3. The method according to claim 1, characterized in that When the consumables parameter meets the operation triggering condition of the functional partition, the obstacle area in the diagnosis and treatment space is identified, and the operation triggering condition includes a replenishment triggering condition and a discarding triggering condition, including: When there are contour parameters in the emergency treatment area, it is determined that the discard trigger condition of the emergency treatment area is met; when the contamination parameter in the verification treatment area is greater than the first threshold, it is determined that the discard trigger condition of the verification treatment area is met; when the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is met; Identify the fixed obstacle area where the facilities in the diagnosis and treatment space are located and the dynamic obstacle area where the physician is located. The obstacle area includes a fixed obstacle area and a dynamic obstacle area. The dynamic obstacle area is obtained according to a preset radius with the center position of the physician's outline as a reference point.

4. The method according to claim 3, characterized in that The following steps are used to count the quantity parameters in the reserve area, including: Acquire a pixel change value of the reserve area in real time, and when the pixel change value is greater than or equal to a pixel change threshold, identify a first track corresponding to a hand contour in the reserve area; Determine an original area of ​​consumables in the reserve area, and obtain a target area generated by pixels that are not in the original area and whose pixel values ​​are located in a consumables pixel interval; A second trajectory of the target area is obtained, characteristic parameters of the first trajectory and the second trajectory are compared, and when a difference between the characteristic parameters is within a deviation range, a reference constant is subtracted from the quantity parameter, wherein the characteristic parameters include at least velocity, acceleration, and curvature.

5. The method according to claim 1, characterized in that Real-time planning of a dynamic path for the operating equipment to the functional area and avoiding obstacles, and sending movement instructions to the operating equipment, including: Generate a straight path from the operating equipment to the functional area, and if the straight path does not pass through the obstacle area, use the straight path as a dynamic path and send a movement instruction to the operating equipment; If the straight path passes through the obstacle area, the obstacle area is expanded according to the safety distance, an avoidance path for the operating equipment to move tangentially along the obstacle area is generated, and a movement instruction is sent to the operating equipment; After the operating equipment meets the avoidance conditions of the obstacle zone, the operating equipment is controlled to go to the functional partition along the shortest path, and the dynamic path includes the avoidance path and the shortest path.

6. The method according to claim 5, characterized in that If the straight path passes through the obstacle area, the obstacle area is expanded according to the safety distance, an avoidance path for the operating device to move tangentially along the obstacle area is generated, and a movement instruction is sent to the operating device, including: The preset radius of the obstacle zone is added with the safety distance to obtain a safety radius, and the obstacle zone is expanded according to the safety radius; Based on the current position of the operating device, multiple tangent lines tangent to the enlarged obstacle area are generated, an avoidance path along the direction corresponding to the shortest tangent line is generated, and a movement instruction is sent to the operating device.

7. The method according to claim 5, characterized in that The shortest path is determined by the following steps, including: During the movement of the operating equipment along the avoidance path, a real-time straight line path from the operating equipment to the functional partition is generated; When there is no intersection between the real-time straight path and the obstacle area, it is determined that the avoidance condition is met, and the corresponding real-time straight path is used as the shortest path.

8. The method according to claim 1, characterized in that: Constructing a candidate area according to the end position of the working equipment, selecting the docking point of the candidate area and sending a synchronous motion instruction to the collaborative equipment, including: When the operating device meets the discard trigger condition, obtaining the end position of the operating device when the picking operation is completed; Taking the end position as the center, determining a plurality of sector-shaped candidate areas according to the division angles, and determining a plurality of reference points in the candidate areas according to half the length of the collaborative device in the radial direction; A reference point that meets the positioning conditions of the collaborative device and is closest to the end position is determined as the docking point, and a synchronous motion instruction is sent to control the collaborative device to move to the docking point.

9. The method according to claim 8, characterized in that Determining a reference point that meets the positioning condition of the collaborative device and is closest to the terminal position as the docking point includes: Taking each reference point located outside the obstacle area as the center, determine the positioning area corresponding to the device specification of the collaborative device; when the positioning area is not located in the obstacle area, determine that the positioning condition is met, and determine the reference point closest to the end position as the docking point.

10. An operating room nursing system based on image recognition, characterized in that: include: A partitioning module, used to identify functional partitions according to image data and collect consumable parameters in each functional partition; A determination module, configured to identify an obstacle zone in the diagnosis and treatment space when the consumables parameter meets the operation trigger condition of the functional partition, wherein the operation trigger condition includes a replenishment trigger condition and a discard trigger condition; A planning module, used for planning in real time a dynamic path for the operating equipment to the functional partition and avoiding obstacle areas, and sending movement instructions to the operating equipment; The collaboration module is used to construct a candidate area according to the end position of the working equipment, select the docking points in the candidate area and send a synchronous motion instruction to the collaboration equipment.

Images