Operating Room Nursing Method and System Based on Image Recognition

Abstract

The present invention provides an operating room nursing method and system based on image recognition, which relates to image processing technology. The present invention can, during the surgical process, use a shooting device to monitor in real time the usage of different categories of consumables, so that the consumables can be accurately replenished and automatically processed through operating equipment, enabling medical staff not to be distracted by dealing with medical waste and replenishing medical supplies, and allowing them to focus on the surgical operation, thereby improving the overall surgical efficiency. In order to distinguish different categories of consumables, this solution will place different categories of consumables in different areas, and then perform different recognition operations on them in combination with their locations. Moreover, during the process of replenishment or disposal operations, areas with obstacles will be avoided to improve the safety during the surgical process and ensure the smooth progress of the surgery.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular, to an operating room nursing method and system based on image recognition. Background Art

[0002] In a modern operating room environment, the efficiency and safety of the surgical process are of crucial importance. During the operation, medical staff not only have to focus on complex and delicate surgical operations but also need to constantly monitor the use of medical consumables, such as the replenishment of gauze and the handling of contaminated gauze. These additional tasks distract the energy of medical staff and affect the concentration and smoothness of the operation.

[0003] Currently, in the operating room nursing process, there are many problems in the management of medical consumables. On the one hand, in terms of monitoring the use of consumables, it mainly relies on the naked eye observation and experience judgment of medical staff, and it is impossible to accurately and real-time grasp the status information of different types of consumables, such as the remaining quantity and contamination degree of gauze. This may lead to situations such as shortage of gauze affecting the surgical process or failure to timely handle contaminated gauze, resulting in an infection risk during the operation. On the other hand, in terms of the replenishment and handling operations of consumables, there is a lack of intelligent and automated means. Medical staff need to manually complete the replenishment of gauze and the cleaning of garbage, which not only takes up valuable surgical time but also easily increases the potential risk of surgical infection due to cross-interference with the surgical area during the operation process.

[0004] Therefore, how to achieve real-time and accurate monitoring of the use of medical consumables and build an automated consumable handling mechanism has become an urgent problem to be solved today. Summary of the Invention

[0005] The present invention provides an operating room nursing method and system based on image recognition, which can achieve real-time and accurate monitoring of the use of medical consumables and build an automated consumable handling mechanism.

[0006] In the first aspect of the present invention, there is provided an operating room nursing method based on image recognition, including:

[0007] Identifying functional areas according to image data and collecting consumable parameters in each of the functional areas;

[0008] When the consumable parameters meet the operation trigger conditions of the functional area, identifying the obstacle area in the diagnosis and treatment space, where the operation trigger conditions include a replenishment trigger condition and a discard trigger condition;

[0009] Real-time planning of a dynamic path for the operation device to the functional area and avoiding the obstacle area, and sending a movement instruction to the operation device;

[0010] Construct a candidate area based on the end position of the operation device, screen the docking points of the candidate area, and send a synchronous movement instruction to the collaborative device.

[0011] Optionally, in a possible implementation manner of the first aspect, identify functional areas according to the image data, and collect consumable parameters in each of the functional areas, including:

[0012] Identify the protection area in the image data, and determine each functional area according to the area where the pixel points in the protection area are located in the preset pixel interval corresponding to each functional area. The functional areas include an emergency treatment area, an identification treatment area, and a reserve area;

[0013] Collect the contour parameters of the consumables in the emergency treatment area, the dirtiness parameters of the consumables in the identification treatment area, and the quantity parameters of the consumables in the reserve area. The consumable parameters include contour parameters, dirtiness parameters, and quantity parameters.

[0014] Optionally, in a possible implementation manner of the first aspect, when the consumable parameters meet the operation trigger conditions of the functional area, identify the obstacle areas in the diagnosis and treatment space. The operation trigger conditions include a replenishment trigger condition and a discard trigger condition, including:

[0015] When there are contour parameters in the emergency treatment area, it is determined that the discard trigger condition of the emergency treatment area is met. When the dirtiness parameter in the verification treatment area is greater than the first threshold, it is determined that the discard trigger condition of the verification treatment area is met. When the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is met;

[0016] Identify the fixed obstacle areas where the facilities are located and the dynamic obstacle areas where the physicians are located in the diagnosis and treatment space. The obstacle areas include fixed obstacle areas and dynamic obstacle areas. The dynamic obstacle areas are obtained based on the central position of the physician's contour according to a preset radius.

[0017] Optionally, in a possible implementation manner of the first aspect, count and statistics the quantity parameters in the reserve area through the following steps, including:

[0018] Obtain the pixel change value of the reserve area in real time. When the pixel change value is greater than or equal to the pixel change threshold, identify the first trajectory corresponding to the hand contour in the reserve area;

[0019] Determine the original area of the consumables in the reserve area, and obtain the target area generated by the pixel points that are not in the original area and whose pixel values are in the consumable pixel interval;

[0020] Obtain the second trajectory of the target area, compare the characteristic parameters of the first trajectory and the second trajectory, and when the difference of the characteristic parameters is within the deviation range, subtract the reference constant from the quantity parameter. The characteristic parameters at least include speed, acceleration, and curvature.

[0021] Optionally, in a possible implementation manner of the first aspect, a dynamic path for the operation device to the functional area and avoiding the obstacle area is planned in real time, and a movement instruction is sent to the operation device, including:

[0022] Generate a straight-line path for the operation device to the functional area. If the straight-line path does not pass through the obstacle area, use the straight-line path as the dynamic path and send a movement instruction to the operation device;

[0023] If the straight-line path passes through the obstacle area, expand the obstacle area according to the safety distance, generate an avoidance path for the operation device to move tangentially along the obstacle area, and send a movement instruction to the operation device;

[0024] After the operation device meets the avoidance conditions of the obstacle area, control the operation device to move to the functional area along the shortest path. The dynamic path includes the avoidance path and the shortest path.

[0025] Optionally, in a possible implementation manner of the first aspect, if the straight-line path passes through the obstacle area, expand the obstacle area according to the safety distance, generate an avoidance path for the operation device to move tangentially along the obstacle area, and send a movement instruction to the operation device, including:

[0026] Add the safety distance to the preset radius of the obstacle area to obtain the safety radius, and perform an expansion process on the obstacle area according to the safety radius;

[0027] Taking the current position of the operation device as the reference, generate multiple tangents tangent to the expanded obstacle area, generate an avoidance path along the direction corresponding to the shortest tangent, and send a movement instruction to the operation device.

[0028] Optionally, in a possible implementation manner of the first aspect, the shortest path is determined through the following steps, including:

[0029] During the process of the operation device moving along the avoidance path, generate a real-time straight-line path for the operation device to the functional area;

[0030] When there is no intersection between the real-time straight-line path and the obstacle area, it is determined that the avoidance condition is met, and the corresponding real-time straight-line path is used as the shortest path.

[0031] Optionally, in a possible implementation of the first aspect, constructing a candidate area according to the end position of the operation device, screening the docking points of the candidate area, and sending a synchronous movement instruction to the collaborative device includes:

[0032] When the operation device meets the discard trigger condition, obtain the end position of the operation device when the picking operation is completed;

[0033] With the end position as the center, determine multiple fan-shaped candidate areas according to the division angle. In the candidate areas, determine multiple reference points in the radial direction according to half of the length of the collaborative device;

[0034] Determine the reference point that meets the positioning condition of the collaborative device and is closest to the end position as the docking point, and send a synchronous movement instruction to control the collaborative device to move to the docking point.

[0035] Optionally, in a possible implementation of the first aspect, determining the reference point that meets the positioning condition of the collaborative device and is closest to the end position as the docking point includes:

[0036] With each reference point located outside the obstacle area as the center, determine the positioning area corresponding to the device specifications of the collaborative device. When the positioning area is not located in the obstacle area, it is determined that the positioning condition is met, and the reference point closest to the end position is determined as the docking point.

[0037] In the second aspect of the present invention, there is provided an operating room nursing system based on image recognition, including:

[0038] A zoning module for identifying functional zones according to image data and collecting consumable parameters in each functional zone;

[0039] A determination module for identifying the obstacle area in the diagnosis and treatment space when the consumable parameters meet the operation trigger conditions of the functional zone, and the operation trigger conditions include a replenishment trigger condition and a discard trigger condition;

[0040] A planning module for real-time planning of the dynamic path of the operation device to the functional zone and avoiding the obstacle area, and sending a movement instruction to the operation device;

[0041] A collaboration module for constructing a candidate area according to the end position of the operation device, screening the docking points of the candidate area, and sending a synchronous movement instruction to the collaborative device.

[0042] The beneficial effects of the present invention are as follows:

[0043] 1. The present invention can collect image data in real time through cameras installed at different positions in the operating room, and by using image recognition technology, it can accurately identify the functional areas divided based on surgical cloth markings, including fixed garbage points, fixed clean item points, and temporary points. For different functional areas, it can accurately collect consumable parameters, realizing real-time, comprehensive, and accurate monitoring of the usage of consumables, providing a reliable basis for subsequent processing and replenishment decisions, greatly improving the efficiency and refinement level of operating room consumable management, and ensuring the smooth progress of surgeries.

[0044] 2. When the consumable parameters meet the operation trigger conditions of the corresponding functional area, the system can automatically identify the obstacle areas in the diagnosis and treatment space, including fixed obstacle areas and dynamic obstacle areas. For contaminated gauze that meets the discard trigger condition, the operating device can plan a dynamic path to avoid the obstacle area in real time, go to the target position to pick it up, and accurately dock with the collaborative device to efficiently transfer the contaminated gauze to the garbage disposal area. For situations that meet the replenishment trigger condition, the operating device can plan a path to go to the replenishment item storage point to suck gauze and send it to the fixed clean item point. This intelligent replenishment and safety processing mechanism reduces the workload of medical staff in dealing with medical waste and replenishing medical supplies, reduces the risk of surgical infection, and improves the safety and smoothness of the surgical process.

[0045] 3. In the docking link between the operating device and the collaborative device, the present invention takes the end position of the operating device when it completes the picking operation as the center, determines multiple fan-shaped candidate areas according to the division angle, and determines the reference points within the candidate areas based on half of the length of the collaborative device. By comprehensively considering factors such as the size of the trolley, its motion performance, and the complexity of path planning to set the division angle, it ensures that a suitable docking position can be selected. Then, the reference point that meets the positioning conditions of the collaborative device and is the closest to the end position of the operating device is determined as the docking point, and a synchronous motion instruction is sent to control the collaborative device to move to the docking point. It realizes the precise docking of the operating device and the collaborative device, optimizes the device path planning, improves the docking efficiency, ensures the smooth progress of the garbage disposal process, helps maintain the cleanliness and hygiene of the operating room, and provides strong support for the smooth development of surgeries. Description of the Drawings

[0046] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0047] Figure 2 is a schematic flowchart of a method for operating room nursing based on image recognition provided by an embodiment of the present invention;

[0048] Figure 3 is a schematic structural diagram of a system for operating room nursing based on image recognition provided by an embodiment of the present invention. Detailed Embodiments

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0050] See Figure 1 , which is a schematic diagram of an application scenario provided by an embodiment of the present invention. This solution can monitor the usage of different types of consumables in real time through a shooting device during the operation, so that the consumables can be accurately replenished and automatically processed through an operating device, enabling medical staff to no longer be distracted by dealing with medical waste and replenishing medical supplies, and thus be able to focus on the surgical operation and improve the overall surgical efficiency. To distinguish different types of consumables, this solution will place different types of consumables in different areas, and then perform different identification operations on them based on the area where they are located. Moreover, during the replenishment or discarding operation, it will also avoid the areas with obstacles to improve the safety during the operation and ensure the smooth progress of the operation. Among them, the shooting device can be a camera, and the operating device can be a robotic arm equipped with a vacuum suction head at the end, which can suck lightweight garbage such as gauze. The consumables can be medical supplies with consumable properties such as gauze that are easy to suck. In the following content, the gauze is used as the consumable for illustration.

[0051] See Figure 2 , which is a schematic flowchart of a method for operating room nursing based on image recognition provided by an embodiment of the present invention. Figure 2 The execution subject of the method shown can be a software and / or hardware device. The execution subject of the present application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. Among them, the user equipment can include, but is not limited to, a computer, a smart phone, a personal digital assistant (Personal Digital Assistant, abbreviated as: PDA), and the above-mentioned electronic devices, etc. The network equipment can include, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which is composed of a group of loosely coupled computers to form a super virtual computer. This embodiment does not make any restrictions on this. It includes steps S1 to S4, specifically as follows:

[0052] S1, identify the functional areas according to the image data, and collect the consumable parameters in each of the functional areas.

[0053] During surgery, accurate management of consumables is crucial to ensure smooth operation and maintain the hygiene of the operating room. By identifying functional partitions and collecting consumable parameters, the status of consumables can be understood in real time, providing a basis for subsequent reasonable disposal of contaminated consumables and timely replenishment of clean consumables, thereby optimizing the surgical process and improving surgical safety and efficiency.

[0054] In practical applications, the image data of the operating room can be collected in real time by installing cameras at different locations in the operating room.

[0055] Among them, functional zoning refers to different use areas divided based on the markings on the surgical cloth, including fixed garbage points for centralized placement of used garbage gauze, fixed clean item points for storage of clean gauze to be used, and temporary points set by doctors on demand for temporary placement of gauze that is specially contaminated or requires special treatment. Consumables parameters refer to the relevant status data for consumables. In garbage-related areas, it refers to the contamination degree parameters of gauze and the contour parameters of garbage gauze in the corresponding area, and in clean item fixed points, it refers to the quantity parameters of gauze.

[0056] Accurate functional zoning identification and consumable parameter collection can clearly understand the distribution and status of gauze. Doctors can focus on surgical operations without being distracted by gauze management details; the system can make accurate decisions based on parameters, such as determining whether gauze needs to be discarded or replenished, providing a reliable basis for subsequent processes and improving the overall efficiency of surgery and the refinement of management.

[0057] In some embodiments, step S1 can be implemented by the following steps:

[0058] Identify the protection area in the image data, and determine each functional area according to the area where the pixel points in the protection area are located in the preset pixel interval corresponding to each functional area, the functional area includes an emergency treatment area, an identification treatment area and a reserve area; collect contour parameters of consumables in the emergency treatment area, contamination parameters of consumables in the identification treatment area and quantity parameters of consumables in the reserve area, the consumable parameters include contour parameters, contamination parameters and quantity parameters.

[0059] Among them, the protected area is the area where the surgical drape is located. The emergency treatment area is a fixed garbage point for placing consumables that need to be immediately processed, such as heavily soiled gauzes that come into contact with special pathogens or critical surgical sites. Quickly processing these items can effectively prevent and control the infection risk. The identification and treatment area is a temporary point for placing used consumables that need to be further judged for the degree of soiling to determine whether to discard them. The storage area is a fixed point for clean items for storing spare clean consumables, such as unused gauzes, to ensure sufficient material supply during the surgical process. The contour parameters refer to the contour characteristics and other parameters of consumables such as soiled gauzes in the emergency treatment area. The soiling parameter refers to the parameter reflecting the degree of soiling obtained by means of image recognition technology for the consumables in the identification and treatment area. The quantity parameter refers to the quantity information of the consumables in the storage area.

[0060] In practical applications, before the operation, medical staff can divide the area on the surgical drape according to the type and requirements of the operation by using medical-grade Velcro. For example, in the area of the surgical drape close to the operating table and convenient for quick access, use Velcro to circle out the emergency treatment area; in a slightly farther but convenient for centralized processing area, divide out the identification and treatment area; set the storage area near the corresponding special storage rack on the surgical drape. Different areas can be demarcated with medical-grade Velcro with different preset pixel ranges, so that the positions of each area can be quickly identified based on the collected image data. Taking the surgical drape as the protected area, combined with image recognition and manually adjustable Velcro zoning, it can flexibly adapt to different operating room layouts and surgical requirements.

[0061] Since the properties of consumables in different areas are different, therefore, for different areas, the collected consumable parameters can also be different. In the emergency treatment area, when a soiled gauze that comes into contact with special pathogens is placed in this area, the contour of the gauze can be identified by an edge detection algorithm. In the identification and treatment area, when a used gauze is placed in, image segmentation technology can be used to separate the stain from the background color of the gauze, analyze the parameters of the stain in the HSV color model, and obtain the soiling parameter of the gauze by synthesizing parameters such as hue, saturation, lightness, and the proportion of the stain-covered area to the total area of the gauze. A corresponding table of multiple parameters and soiling degree can be set in advance by combining multiple parameters, and then the specific soiling parameter can be determined by traversing the data in the table. This table can be pre-set by the staff according to the actual situation. For example, multiple intervals can be set for each parameter, and different interval combinations correspond to different soiling degrees. In the storage area, the camera can take pictures of the gauze taking situation at the position of the storage area corresponding to the surgical drape, so as to determine the remaining amount of consumables in combination with the usage situation of the consumables.

[0062] S2. When the consumable parameters meet the operation trigger conditions of the functional partition, identify the obstacle area in the diagnosis and treatment space, and the operation trigger conditions include a replenishment trigger condition and a discard trigger condition.

[0063] It is understandable that timely disposal of contaminated gauze can reduce the risk of infection and ensure the hygiene of the operating room; timely replenishment of clean gauze can prevent the interruption of the operation due to gauze shortage. Identifying the obstacle area is to plan a safe path for the robotic arm to avoid interfering with the surgical operation during the task execution, ensure the safety and order of the entire operating room environment, and guarantee the smooth progress of the operation.

[0064] The operation trigger condition is the judgment basis for deciding whether to discard or replenish the gauze. The discard trigger condition is that when the contamination degree of the gauze in the temporary point reaches the preset standard or there is gauze in the garbage fixed point; the replenishment trigger condition is that the number of gauze on the clean item fixed point is lower than the threshold. The obstacle area is the area where the surgical area is set as a non-intervenable space, that is, the area that the robotic arm cannot enter during the movement process, to avoid colliding with the personnel and equipment during the operation and ensure the safe progress of the operation. The diagnosis and treatment space is the operating room.

[0065] Based on the above embodiments, the specific implementation manner of step S2 can be:

[0066] S21, when there is a contour parameter in the emergency treatment area, it is determined that the discard trigger condition of the emergency treatment area is satisfied. When the turbidity parameter in the verification treatment area is greater than the first threshold, it is determined that the discard trigger condition of the verification treatment area is satisfied. When the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is satisfied.

[0067] During the efficient operation of the operating room, timely and accurately determining the disposal and replenishment operations of consumables is crucial for ensuring the smooth progress of the operation, maintaining the safety and hygiene of the operating room environment, and reasonably utilizing resources. Clearly defining the operation trigger conditions of each functional area and making appropriate decisions can avoid problems such as surgical delays and increased infection risks caused by delayed or incorrect human judgment.

[0068] Since the emergency treatment area is used to place consumables that need to be immediately processed, when there is a contour parameter in the emergency treatment area, it can be determined that the corresponding discard trigger condition is satisfied. The first threshold is a preset critical value used to measure whether the turbidity parameter of the consumables in the verification treatment area reaches the discard standard. Therefore, when the turbidity parameter in the verification treatment area is greater than the first threshold, it can be determined that the corresponding discard trigger condition is satisfied. The second threshold is a preset critical value used to judge whether the quantity of consumables in the reserve area is insufficient and needs to be replenished. Therefore, when the quantity parameter in the reserve area is less than the second threshold, it can be determined that the corresponding replenishment trigger condition is satisfied.

[0069] In some embodiments, the quantity parameter in the reserve area can be counted and statistically analyzed through the following steps:

[0070] S211. Obtain the pixel change value of the reserve area in real time. When the pixel change value is greater than or equal to the pixel change threshold, identify the first trajectory corresponding to the hand contour in the reserve area.

[0071] It can be understood that accurately and real-time counting the number of gauzes in the reserve area is crucial for ensuring the smooth progress of the operation. By obtaining the pixel change value of the reserve area in real time, the image changes caused by the doctor's operations such as taking gauzes can be captured in a timely manner, and then the hand contour trajectory can be identified, laying a foundation for accurately judging whether the gauze is taken and the number taken subsequently, ensuring the dynamic monitoring of the gauze quantity, and avoiding the impact on the surgical material supply due to inaccurate gauze quantity statistics.

[0072] The pixel change value refers to the change amount of the pixel values of the reserve area image at different times in the image sequence. The pixel change threshold is a preset value used to measure whether the pixel change value is significant. When the pixel change value of a certain area is greater than or equal to this threshold, it is considered that a meaningful action has occurred in this area, such as the doctor taking gauzes and other operations. In the image, the contour of the doctor's hand that can be identified through image processing technology can be used to further analyze the movement trajectory of the hand by extracting the hand contour. The first trajectory is the movement trajectory corresponding to the contour of the doctor's hand when it moves in the reserve area. This trajectory is obtained by continuously recording and analyzing the positions of the hand contour in the image sequence.

[0073] S212. Determine the original area of the consumables in the reserve area, and obtain the target area generated by the pixel points that are not in the original area and whose pixel values are within the consumable pixel interval.

[0074] After identifying the hand contour trajectory, determining the original area and the target area of the consumables in the reserve area is to more accurately judge whether the gauze is taken and the specific gauze taken. By comparing the changes between the original area and the target area, the position movement of the gauze can be accurately tracked, thus providing a more reliable basis for quantity statistics.

[0075] The original area is the position area where the consumables in the reserve area are initially located. Before the operation starts or after the gauze is replenished, the gauzes are neatly placed at specific positions in the reserve area, and these positions constitute the original area. The target area is the area composed of the pixel points that are not in the original area and whose pixel values are within the consumable pixel interval obtained through image processing technology. When the gauze is taken, its position changes, forming a new area, that is, the target area. The consumable pixel interval is a preset value range corresponding to the pixel values of the consumables in the image.

[0076] S213. Obtain the second trajectory of the target area, compare the characteristic parameters of the first trajectory and the second trajectory, and when the difference between the characteristic parameters is within the deviation range, subtract the reference constant from the quantity parameter. The characteristic parameters at least include speed, acceleration, and curvature.

[0077] The second trajectory is the movement trajectory formed when the target area moves in the image sequence. This trajectory is obtained by recording and analyzing the positions of the target area in consecutive image frames. Characteristic parameters are parameters used to describe the characteristics of the trajectory, at least including speed, acceleration, and curvature. Speed reflects how fast the trajectory moves, acceleration represents the rate of change of speed, and curvature describes the degree of bending of the trajectory. By comparing these parameters, it can be determined whether the movement of the target area is related to the hand movement. Deviation range: The range of allowable differences in characteristic parameters set in advance. When the difference in characteristic parameters between the first trajectory and the second trajectory is within this range, it is considered that the movement of the target area is consistent with the hand movement, that is, the gauze has been picked up. The reference constant is a fixed value used to adjust the quantity parameter after it is determined that the gauze has been picked up. Usually, the reference constant is 1, indicating that for each piece of gauze picked up, the quantity parameter is decreased by 1.

[0078] Obtain the second trajectory of the target area, and at the same time, the first trajectory of the hand contour is known. Calculate and compare the characteristic parameters such as speed, acceleration, and curvature of the first trajectory and the second trajectory. Suppose the speed of the first trajectory is 5 pixel units per second, the acceleration is 2 pixel units per second squared, and the curvature is 0.1; the speed of the second trajectory is 4.8 pixel units per second, the acceleration is 1.9 pixel units per second squared, and the curvature is 0.11. The differences in these characteristic parameters (the speed difference is 0.2, the acceleration difference is 0.1, and the curvature difference is 0.01) are within the pre-set deviation range (suppose the speed deviation range is ±0.5, the acceleration deviation range is ±0.3, and the curvature deviation range is ±0.2), and it is determined that the gauze has been picked up. At this time, the system subtracts the reference constant 1 from the quantity parameter. If the original quantity parameter of the gauze in the storage area is 20, it becomes 19 after subtracting 1.

[0079] Through the steps of trajectory comparison and quantity statistics, the system can accurately determine whether the gauze has been picked up and update the quantity parameter of the gauze in the storage area in a timely manner. This accurate quantity statistical method ensures that the operating room nursing system can accurately grasp the real-time quantity of gauze, helps to reasonably arrange the replenishment and use of gauze, avoids problems with the supply of surgical supplies caused by inaccurate gauze quantities, and improves the working efficiency of the operating room.

[0080] S22. Identify the fixed obstacle areas where facilities are located and the dynamic obstacle areas where the physician is located in the diagnosis and treatment space. The obstacle areas include fixed obstacle areas and dynamic obstacle areas. The dynamic obstacle area is obtained based on the center position of the physician's contour according to a preset radius.

[0081] During the operation, to ensure that the automated equipment can move safely and efficiently in the operating room, perform tasks such as picking up and placing consumables, it is crucial to accurately identify the obstacle areas in the operating room. Defining the fixed obstacle areas and dynamic obstacle areas can provide a key basis for the equipment path planning, prevent the equipment from colliding with obstacles, and ensure the normal order of the operation and the safety of personnel and equipment.

[0082] Among them, the fixed obstacle area is the space area occupied by the fixed facilities in the operating room, such as the operating table, medical instrument cabinet, anesthesia equipment, etc. These facilities are relatively fixed in position and will not move randomly during the operation. The areas where they are located constitute the fixed obstacle range for the movement of the automated equipment.

[0083] The dynamic obstacle area is a movable area delimited based on the center position of the physician's contour according to a preset radius. Since the physician will move continuously during the operation, the space within a certain range around him poses a collision risk to the automated equipment. Therefore, this area can be set as the dynamic obstacle area. The setting of the radius needs to comprehensively consider factors such as the physician's activity range and the safety distance of the equipment. The preset radius is the radius value preset for determining the range of the dynamic obstacle area.

[0084] Accurate identification of the obstacle areas provides a strong guarantee for the safe movement of the automated equipment in the complex environment of the operating room. By generating the dynamic obstacle area through real-time monitoring of the physician's position and determining the fixed obstacle area, the occurrence of equipment collision accidents is effectively avoided, ensuring the continuity and safety of the operation. At the same time, it also improves the working efficiency of the automated equipment, enabling it to assist the operating room nursing work more stably and efficiently.

[0085] S3. Real-time plan the dynamic path of the operation equipment to the functional area and avoid the obstacle areas, and send a movement instruction to the operation equipment.

[0086] The robotic arm needs to safely and efficiently complete tasks such as picking up contaminated gauze and placing clean gauze in the complex environment of the operating room. Planning the dynamic path and avoiding the obstacle areas can ensure that the robotic arm accurately reaches the target position, avoid the collision risk, save time, improve the working efficiency, and ensure the smoothness of the gauze handling link during the operation.

[0087] A dynamic path refers to a moving route planned based on the current position of the robotic arm, the position of the target functional area, and the real-time state of the obstacle area, and this route will be adjusted due to environmental changes. When the discard trigger condition is met, a path for the operation device to pick up garbage from the current position to a temporary point or a fixed garbage point can be planned, and then a movement instruction is sent to control the operation device to move to the target position along the corresponding path for garbage picking. When the replenishment trigger condition is met, the operation device can first be made to go to the point where the replenishment items are stored to suck gauze, and then a path from the position of the operation device to the fixed clean item point is planned, and a movement instruction is sent to control the operation device to go to the corresponding target position for the placement of consumables.

[0088] Real-time dynamic path planning enables the robotic arm to respond flexibly in a complex and changeable operating room environment, safely avoid the obstacle area, effectively avoid collisions with the surgical area, and ensure surgical safety.

[0089] Based on the above embodiments, the specific implementation manner of step S3 can be:

[0090] S31, generate a straight-line path from the operation device to the functional area. If the straight-line path does not pass through the obstacle area, use the straight-line path as the dynamic path and send a movement instruction to the operation device.

[0091] In the operating room environment, the operation device needs to quickly and accurately reach the functional area to perform tasks, such as picking up contaminated gauze or replenishing clean gauze. Generating a straight-line path is the most direct and efficient path planning method. By first checking whether the straight-line path passes through the obstacle area, it can quickly determine whether this simple and efficient path can be adopted, thus saving path planning time and improving the working efficiency of the operation device.

[0092] S32, if the straight-line path passes through the obstacle area, expand the obstacle area according to the safety distance, generate an avoidance path for the operation device to move tangentially along the obstacle area, and send a movement instruction to the operation device.

[0093] When the straight-line path cannot meet the safety requirements, that is, it passes through the obstacle area, in order to ensure the normal progress of the surgery and the safety of personnel and equipment, it is necessary to re-plan the path of the operation device so that it can bypass the obstacle area and safely reach the target functional area. By generating an avoidance path, it not only ensures that the operation device can complete the task, but also avoids interference with the surgical process and potential collision risks.

[0094] The safety distance is the minimum distance that the operation device needs to maintain from the boundary of the obstacle area when bypassing the obstacle area to ensure that the device will not accidentally approach the physician or collide with fixed equipment. The avoidance path is the path that the operation device re-plans to move tangentially along the boundary of the expanded obstacle area to avoid the obstacle when the straight-line path passes through the obstacle area.

[0095] In some embodiments, an avoidance path can be generated through the following steps:

[0096] Add a safety distance to the preset radius of the obstacle area to obtain a safety radius, and expand the obstacle area according to the safety radius; taking the current position of the operating device as a reference, generate multiple tangents tangent to the expanded obstacle area, generate an avoidance path along the direction corresponding to the shortest tangent, and send a movement instruction to the operating device.

[0097] It can be understood that in the complex environment of the operating room, ensuring the safety distance between the operating device and the obstacle area is crucial. By adding a safety distance to the preset radius of the obstacle area to obtain a safety radius and expanding the obstacle area accordingly, sufficient safety space can be reserved when the device plans the avoidance path, effectively reducing the risk that the device accidentally approaches the obstacle during movement, interfering with the operation or causing a collision, and ensuring the smooth progress of the operation and the safety of personnel and equipment.

[0098] By reasonably expanding the range of the obstacle area, more reliable safety protection is provided for the path planning of the operating device. A larger safety area can effectively accommodate the possible deviations of the device during movement, reducing the possibility of the device approaching the obstacle dangerously or colliding, ensuring the safety of personnel and equipment in the operating room, and maintaining the stable progress of the operation.

[0099] Taking the current position of the operating device as a reference, generating multiple tangents tangent to the expanded obstacle area, and selecting the path along the direction corresponding to the shortest tangent as the avoidance path can enable the operating device to reach the target with a relatively short travel distance under the premise of meeting safety requirements, improve the operating efficiency of the device, and reduce the time waste caused by circuitous paths.

[0100] S33, after the operating device meets the avoidance conditions of the obstacle area, control the operating device to move to the functional area according to the shortest path, and the dynamic path includes the avoidance path and the shortest path.

[0101] After the operating device bypasses the obstacle area, continuing to move along the avoidance path may increase unnecessary travel and time consumption. When the operating device meets the avoidance conditions of the obstacle area, switching to the shortest path to move to the functional area can improve the speed of the device reaching the target area under the premise of ensuring safety and improve the working efficiency of the operating device.

[0102] The avoidance condition of the obstacle area means that the operating device successfully bypasses the obstacle area, and the straight-line path between it and the target functional area no longer intersects the obstacle area. The shortest path is the shortest straight-line connection path from the current position to the target functional area after the operating device meets the avoidance conditions of the obstacle area.

[0103] In some embodiments, the shortest path can be determined through the following steps:

[0104] During the movement of the operating device along the avoidance path, generate a real-time straight-line path from the operating device to the functional area; when there is no intersection between the real-time straight-line path and the obstacle area, it is determined that the avoidance condition is satisfied, and the corresponding real-time straight-line path is used as the shortest path.

[0105] During the movement of the operating device along the avoidance path, the situation in the operating room may change. For example, the movement of the physician may cause a change in the range or position of the obstacle area, or other temporary obstacles may appear. By continuously generating the real-time straight-line path from the operating device to the functional area, it is possible to evaluate in real time whether the device can directly go to the target functional area, providing a basis for determining whether the avoidance condition is satisfied and switching to the shortest path, thereby dynamically optimizing the movement path of the device and improving work efficiency. The real-time straight-line path is the straight-line connection path calculated based on the current position of the device and the position of the target functional area at each moment during the movement of the operating device along the avoidance path. This path is updated in real time as the position of the device changes.

[0106] When the operating device moves along the avoidance path, once the real-time straight-line path between it and the target functional area no longer intersects with the obstacle area, it means that the device can directly go to the target safely and more efficiently. At this time, it is determined that the avoidance condition is satisfied and switched to the shortest path, which can reduce unnecessary detours of the device, save time and energy, improve the overall work efficiency of the operating device in the operating room environment, and ensure the timely supply and handling of surgical supplies.

[0107] S4. Construct a candidate area according to the end position of the operating device, screen the docking points of the candidate area, and send a synchronous movement instruction to the collaborative device.

[0108] During the surgical process, it is crucial to promptly clean the used contaminated gauze to maintain the cleanliness of the operating room and reduce the risk of infection. After the robotic arm picks up the contaminated gauze, it needs to be accurately docked with the collaborative device to ensure that the contaminated gauze can be accurately and efficiently transferred to the waste treatment area, avoiding garbage spillage or processing delays caused by docking errors, thereby ensuring the safety of the operating room environment and the smoothness of the surgical process.

[0109] Among them, the candidate areas are multiple fan-shaped areas divided with the end position after the robotic arm grabs the contaminated gauze as the center. Its function is to provide a range for screening suitable positions for docking with the collaborative device, facilitating precise positioning. The docking point is the point that meets the conditions for transferring the contaminated gauze and is the closest to the end position of the robotic arm selected from the candidate areas. The collaborative device is a movable trolley used to transport the contaminated gauze. The synchronous motion instruction is an instruction sent to the movable trolley, which contains the docking point coordinates and related motion parameters, such as driving speed, direction, etc. This instruction is used to coordinate the actions of the movable trolley and the robotic arm to ensure that the two can cooperate precisely during the docking process and achieve seamless handover of the contaminated gauze.

[0110] By constructing candidate areas and precisely screening docking points, combined with synchronous motion instructions, the efficient cooperation between the operating device and the movable collaborative device during the garbage throwing process is achieved. It ensures that the contaminated gauze is accurately handed over to the trolley, avoids the risk of garbage spilling, improves the garbage cleaning efficiency, removes the contaminated gauze from the surgical area in a timely manner, reduces the infection risk, and guarantees the continuity of the surgical process and the hygienic safety of the operating room environment.

[0111] Based on the above embodiments, the specific implementation manner of step S4 can be:

[0112] S41, when the operating device meets the discard trigger condition, obtain the end position when the operating device completes the picking operation.

[0113] When the operating device meets the discard trigger condition, that is, after detecting the item to be discarded and completing the picking operation, accurately obtaining its end position is the basis for precise docking with the collaborative device. Only by clarifying the end position of the operating device can subsequent operations such as dividing the candidate areas and determining the docking points be carried out centered on this, so as to ensure that the item can be accurately transferred to the collaborative device and achieve an efficient garbage disposal process. The end position is the coordinate position of the part used by the operating device to pick up the item in the operating room space when the picking operation is completed.

[0114] S42, with the end position as the center, determine multiple fan-shaped candidate areas according to the division angle, and in the candidate areas, determine multiple reference points in the radial direction according to half of the length of the collaborative device.

[0115] Determining multiple fan-shaped candidate areas with the end position of the operating device as the center and determining reference points in the candidate areas according to half of the length of the collaborative device is to screen out suitable positions in the space around the operating device for docking with the collaborative device. In this way, it can comprehensively cover possible docking positions, increase the probability of finding a suitable docking point, and consider the length of the collaborative device to ensure the feasibility and stability of the docking.

[0116] Among them, the division angle is an angle value set for dividing the circumference centered on the end position of the operating device. Through this angle division, the circumference can be divided into multiple fan-shaped areas, which is convenient for finding suitable docking positions in different directions. The division angle can be set in combination with the specifications of the trolley. The length and width of the trolley have a decisive impact on its docking space. In the radial direction, usually half of the trolley length is used as the interval to determine the reference points. In the circumferential direction, the width of the trolley plays a key role. If the trolley is wider, in order to ensure that the rectangular area adapted to the trolley size constructed with the discrete reference points as the center can be completely within the fan-shaped area, the angle of the fan-shaped area must be increased accordingly. Therefore, a corresponding table of the width and angle values of multiple trolleys can be set in advance, and the current division angle can be determined through the table.

[0117] For example, suppose there is a trolley with a width of 1 meter, and docking planning is carried out on a circumference with a radius of 2 meters. To ensure that the trolley has sufficient space to complete the docking action, the fan-shaped angle may need to be set in the range of 30° - 45°. Therefore, the corresponding division angle can be a value within 30° - 45°. Such a setting can enable the trolley to smoothly find a suitable docking point within the fan-shaped area, avoiding docking failures or collisions with surrounding objects due to insufficient space.

[0118] The candidate areas are multiple fan-shaped areas centered on the end position of the operating device and determined according to the division angle. The reference points are multiple points determined in the candidate areas in the radial direction according to half of the length of the collaborative device.

[0119] S43, determine the reference point that meets the positioning conditions of the collaborative device and is closest to the end position as the docking point, and send a synchronous movement instruction to control the collaborative device to move to the docking point.

[0120] Determine the reference point that meets the positioning conditions of the collaborative device and is closest to the end position of the operating device among multiple reference points as the docking point, and send a synchronous movement instruction to control the collaborative device to move to this docking point, in order to achieve the precise docking of the operating device and the collaborative device, ensuring that items can be safely and efficiently transferred from the operating device to the collaborative device. Selecting the closest reference point can reduce the movement distance and time of the collaborative device, improve the docking efficiency, and at the same time meeting the positioning conditions can ensure the accuracy and stability of the docking.

[0121] Among them, the positioning conditions of the collaborative device are conditions set for judging whether the reference point is suitable as the docking point. These conditions can include whether there is enough space at the position where the reference point is located for the collaborative device to dock, and whether other obstacles in the operating room are avoided.

[0122] In some embodiments, the docking point can be determined through the following steps:

[0123] Taking each of the reference points located outside the obstacle area as the center, determine the positioning area corresponding to the device specifications of the collaborative device. When the positioning area is not located in the obstacle area, it is determined that the positioning condition is satisfied, and the reference point closest to the end position is determined as the docking point.

[0124] By determining the positioning area that matches the specifications of the collaborative device with the reference point as the center and judging whether it is outside the obstacle area, the safety of the docking position of the collaborative device can be ensured. Then, the reference point closest to the end position of the operating device is selected as the docking point, which can maximize the shortening of the docking path, improve the docking efficiency, ensure the smoothness of the garbage disposal process, and maintain the safety and order of the operating room environment.

[0125] Among them, the positioning area is a specific area determined according to the device specifications of the collaborative device with a reference point located outside the obstacle area as the center. The size and shape of this area are adapted to the size of the collaborative device, and it is the key area for judging whether the collaborative device can dock safely.

[0126] For example, after preliminary screening, some reference points located outside the obstacle area are obtained. It is known that the length of the mobile trolley is 2 meters and the width is 1.5 meters. Taking reference point A as the center, the positioning area is determined according to the device specifications of the trolley. The positioning area is a rectangular area centered on reference point A, with a length of 2 meters (the same as the trolley length) and a width of 1.5 meters (the same as the trolley width). By calculating the spatial coordinates and comparing with the range of the obstacle area, it is judged whether the positioning area is within the obstacle area. When the positioning area is completely outside the obstacle area, the positioning condition is satisfied, and the corresponding reference point is determined as the docking point.

[0127] Through the above steps, the docking point most suitable for the collaborative device to dock can be accurately found from numerous reference points. Ensure the safety of the collaborative device during the docking process, avoid collisions with people or equipment in the obstacle area, and ensure the normal progress of the operation. At the same time, the reference point closest to the end position of the operating device is selected as the docking point, which maximally shortens the docking path, reduces the movement time and energy consumption of the collaborative device, improves the docking efficiency, makes the garbage disposal process more efficient and smooth, helps to maintain the cleanliness and hygiene of the operating room, and provides strong support for the smooth progress of the operation.

[0128] See Figure 3 , which is a schematic structural diagram of an operating room nursing system based on image recognition provided by an embodiment of the present invention. The operating room nursing system based on image recognition includes:

[0129] A zoning module for identifying functional zones according to image data and collecting consumable parameters in each of the functional zones;

[0130] A determination module, configured to identify an obstacle area in a diagnosis and treatment space when the consumable parameter meets the operation trigger condition of the function partition, where the operation trigger condition includes a replenishment trigger condition and a discard trigger condition;

[0131] A planning module, configured to plan a dynamic path of a working device to the function partition and avoid the obstacle area in real time, and send a movement instruction to the working device;

[0132] A cooperation module, configured to construct a candidate area according to the end position of the working device, screen a docking point of the candidate area, and send a synchronous movement instruction to a cooperation device.

[0133] Figure 3 The device in the illustrated embodiment can correspondingly be used to execute Figure 2 the steps in the illustrated method embodiment, and the implementation principle and technical effect are similar, and will not be elaborated here.

[0134] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operating room nursing method based on image recognition, characterized in that, including: Identifying functional partitions according to image data, and collecting consumable parameters in each of the functional partitions; Identifying a protection area in the image data, and determining each functional partition according to an area where pixel points in the protection area are located in a preset pixel interval corresponding to each functional partition, where the functional partitions include an emergency treatment area, an identification treatment area, and a reserve area; Collecting contour parameters of consumables in the emergency treatment area, dirtiness parameters of consumables in the identification treatment area, and quantity parameters of consumables in the reserve area, where the consumable parameters include contour parameters, dirtiness parameters, and quantity parameters; When the consumable parameters meet the operation trigger conditions of the functional partitions, identifying an obstacle area in the diagnosis and treatment space, where the operation trigger conditions include a replenishment trigger condition and a discard trigger condition; When there are contour parameters in the emergency treatment area, it is determined that the discard trigger condition of the emergency treatment area is met. When the dirtiness parameter in the verification treatment area is greater than a first threshold, it is determined that the discard trigger condition of the verification treatment area is met. When the quantity parameter in the reserve area is less than a second threshold, it is determined that the replenishment trigger condition of the reserve area is met; Identifying a fixed obstacle area where facilities are located and a dynamic obstacle area where a physician is located in the diagnosis and treatment space, where the obstacle area includes a fixed obstacle area and a dynamic obstacle area, and the dynamic obstacle area is obtained based on the central position of the physician's contour according to a preset radius; Real-time planning a dynamic path for the operation device to the functional partition and avoiding the obstacle area, and sending a movement instruction to the operation device; Constructing a candidate area according to the end position of the operation device, screening a docking point of the candidate area, and sending a synchronous movement instruction to the collaborative device; Counting and statistics the quantity parameter in the reserve area through the following steps, including: Real-time obtaining a pixel change value of the reserve area. When the pixel change value is greater than or equal to a pixel change threshold, identifying a first trajectory corresponding to the hand contour in the reserve area; Determining an original area of the consumables in the reserve area, and obtaining a target area generated by pixel points that are not in the original area and whose pixel values are in the consumable pixel interval; Obtaining a second trajectory of the target area, comparing characteristic parameters of the first trajectory and the second trajectory. When the difference between the characteristic parameters is within a deviation range, subtracting a reference constant from the quantity parameter, where the characteristic parameters at least include speed, acceleration, and curvature.

2. The method according to claim 1, wherein: Real-time planning a dynamic path for the operation device to the functional partition and avoiding the obstacle area, and sending a movement instruction to the operation device, including: Generating a straight-line path for the operation device to the functional partition. If the straight-line path does not pass through the obstacle area, using the straight-line path as the dynamic path and sending a movement instruction to the operation device; If the straight-line path passes through the obstacle area, expanding the obstacle area according to a safety distance, generating an avoidance path for the operation device to move tangentially along the obstacle area, and sending a movement instruction to the operation device; After the operation device meets the avoidance conditions of the obstacle area, controlling the operation device to move to the functional partition according to the shortest path, where the dynamic path includes an avoidance path and a shortest path.

3. The method according to claim 2, wherein if the straight path passes through the obstacle area, expanding the obstacle area according to a safety distance, generating an avoidance path for the operation device to move tangentially along the obstacle area, and sending a movement instruction to the operation device, including: adding a safety distance to a preset radius of the obstacle area to obtain a safety radius, and performing an expansion process on the obstacle area according to the safety radius; taking the current position of the operation device as a reference, generating multiple tangent lines tangent to the expanded obstacle area, generating an avoidance path in the corresponding direction of the shortest tangent line, and sending a movement instruction to the operation device.

4. The method according to claim 2, wherein the shortest path is determined through the following steps, including: during the movement of the operation device along the avoidance path, generating a real-time straight path from the operation device to the functional area; when there is no intersection between the real-time straight path and the obstacle area, determining that the avoidance condition is satisfied, and taking the corresponding real-time straight path as the shortest path.

5. The method according to claim 1, wherein constructing a candidate area according to the end position of the operation device, screening a docking point of the candidate area, and sending a synchronous movement instruction to the collaborative device, including: when the operation device meets the discard trigger condition, obtaining the end position of the operation device when the picking operation is completed; taking the end position as the center, determining multiple fan-shaped candidate areas according to a division angle, and in the candidate areas, determining multiple reference points in the radial direction according to half of the length of the collaborative device; determining the reference point that meets the positioning condition of the collaborative device and is closest to the end position as the docking point, and sending a synchronous movement instruction to control the collaborative device to move to the docking point.

6. The method according to claim 5, wherein determining the reference point that meets the positioning condition of the collaborative device and is closest to the end position as the docking point, including: taking each of the reference points located outside the obstacle area as the center, determining a positioning area corresponding to the device specification of the collaborative device, and when the positioning area is not located in the obstacle area, determining that the positioning condition is satisfied, and determining the reference point closest to the end position as the docking point.

7. An operating room nursing system based on image recognition, characterized in that, including: a partitioning module, configured to identify functional areas according to image data, and collect consumable parameters in each of the functional areas; identifying a protection area in the image data, and determining each functional area according to an area where pixel points in the protection area are located in a preset pixel interval corresponding to each functional area, where the functional areas include an emergency treatment area, an identification treatment area, and a reserve area; collecting contour parameters of consumables in the emergency treatment area, dirtiness parameters of consumables in the identification treatment area, and quantity parameters of consumables in the reserve area, where the consumable parameters include contour parameters, dirtiness parameters, and quantity parameters; a determination module, configured to identify an obstacle area in the diagnosis and treatment space when the consumable parameters meet the operation trigger conditions of the functional areas, where the operation trigger conditions include a replenishment trigger condition and a discard trigger condition; When there are contour parameters in the emergency processing area, it is determined that the discard trigger condition of the emergency processing area is met. When the dirt parameter in the inspection processing area is greater than the first threshold, it is determined that the discard trigger condition of the inspection processing area is met. When the quantity parameter in the reserve area is less than the second threshold, it is determined that the replenishment trigger condition of the reserve area is met; Identify the fixed obstacle area where the facilities are located and the dynamic obstacle area where the physician is located in the diagnosis and treatment space. The obstacle area includes a fixed obstacle area and a dynamic obstacle area. The dynamic obstacle area is obtained based on the center position of the physician's contour according to a preset radius; The planning module is used to plan the dynamic path of the operation device to the functional area and avoid the obstacle area in real time, and send a movement instruction to the operation device; The cooperation module is used to construct a candidate area according to the end position of the operation device, screen the docking points of the candidate area and send a synchronous movement instruction to the cooperation device; The quantity parameter in the reserve area is counted and statistically analyzed through the following steps, including: Obtain the pixel change value of the reserve area in real time. When the pixel change value is greater than or equal to the pixel change threshold, identify the first trajectory corresponding to the hand contour in the reserve area; Determine the original area of the consumables in the reserve area, and obtain the target area generated by the pixel points that are not in the original area and whose pixel values are in the consumable pixel interval; Obtain the second trajectory of the target area, compare the characteristic parameters of the first trajectory and the second trajectory. When the difference between the characteristic parameters is within the deviation range, subtract the reference constant from the quantity parameter. The characteristic parameters at least include speed, acceleration and curvature.

Images