Intelligent packaging table based on counting and classification of surgical instruments

Abstract

The invention discloses an intelligent packaging table based on counting and classification of surgical instruments, relates to the technical field of augmented reality, and aims at solving the technical problems that after existing surgical instruments are disinfected and cleaned, manual counting, classification and packaging efficiency is low, and accuracy is difficult to guarantee. Comprising a head-mounted display and a plurality of image acquisition modules, the plurality of image acquisition modules are dispersedly arranged on the head-mounted display and are used for acquiring images of surgical instruments and surroundings on a packaging table and displaying the image information acquired by the image acquisition modules, and meanwhile, a display screen is arranged in the head-mounted display to display the image information of the surgical instruments and the surroundings. The names and the number of the corresponding surgical instruments are projected at the position of each counting, classifying and packaging box, and the number is updated in an inverted order along with placement of the surgical instruments; and the identification module is connected to the image acquisition module and is used for identifying various surgical instruments acquired by the image acquisition module. The method has the advantages that the working efficiency is improved, and the counting and classification accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of augmented reality, and more specifically, to an intelligent packing table for inventory classification of surgical instruments. Background Art

[0002] In the medical field, the inventory classification and packing work after the disinfection and cleaning of surgical instruments is crucial, and its accuracy and efficiency are directly related to the smooth progress of subsequent surgeries. Currently, most medical institutions mainly rely on manual operations in this link.

[0003] The process of manual inventory classification and packing is usually that the staff first place the disinfected and cleaned surgical instruments on the operating table, and then, relying on personal experience, check each instrument one by one against the surgical instrument list. In this process, the staff needs to carefully observe the appearance, shape, specifications and other characteristics of the instruments to determine whether their types and quantities are consistent with the list. After the inventory is completed, classification is carried out according to different surgical requirements or department requirements, and finally the classified instruments are put into the corresponding packing boxes.

[0004] However, this traditional manual operation method has significant defects: Since there are a wide variety of surgical instruments, there are multiple models of common scalpels, and there are even more diverse styles of forceps, pliers, etc. In large hospitals, a huge number of surgical instruments need to be processed every day. When manually checking each instrument one by one, it takes a certain amount of time to check each instrument. Especially when dealing with complex combinations of surgical instruments, the staff needs to spend a lot of energy to distinguish different instruments, resulting in an extremely slow inventory classification and packing process. At the same time, manual operations are affected by various factors, such as the fatigue level of the staff, differences in work experience, and environmental interference. Working for a long time is likely to cause the staff to lose concentration, and there may be situations of missing counts, overcounting or miscounting. Staff with different experience levels have different degrees of familiarity with the instruments, which may also lead to classification errors.

[0005] In view of this, we propose an intelligent packing table for inventory classification of surgical instruments. Summary of the Invention

[0006] The purpose of the present invention is to provide an intelligent packing table for inventory classification of surgical instruments to solve the technical problems of low efficiency and difficult to guarantee accuracy in manual inventory classification and packing after the disinfection and cleaning of existing surgical instruments.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: An intelligent packing table for inventory classification of surgical instruments, comprising: An augmented reality display module, including a head-mounted display and a number of image acquisition modules. The number of image acquisition modules are dispersedly arranged on the head-mounted display, used to collect images of surgical instruments and the surrounding environment on the packing table, and display the image information collected by the image acquisition modules. At the same time, through the built-in display screen of the head-mounted display, the corresponding names and quantities of surgical instruments are projected at the position of each inventory classification packing box, and the quantity is updated in reverse order as surgical instruments are placed in; An identification module, connected to the image acquisition module, and used to identify various surgical instruments collected by the image acquisition module; A storage module, used to store information of various surgical instruments; A comparison module, connected to the identification module and the storage module, and used to compare various surgical instruments collected by the image acquisition module with the corresponding surgical instruments stored in the storage module; A determination module, connected to the comparison module, and used to accurately determine whether a surgical instrument is damaged based on the comparison result of the comparison module; A voice prompt module, connected to the determination module, and used to emit a damage prompt sound when the determination module determines that a surgical instrument is damaged; A storage module, including inventory classification packing boxes, used to accurately sense the putting in and taking out of surgical instruments, and update the instrument quantity in real time.

[0008] Preferably, the head-mounted display is built with a high-precision position sensor. When the staff faces the storage box, the position sensor quickly identifies the position and posture of the storage box; Based on augmented reality technology, on the display screen of the head-mounted display, the corresponding names and quantities of surgical instruments are accurately projected at the position corresponding to each storage box. The name and initial quantity information of the surgical instruments are retrieved from the storage module, and the storage module records the types and quantities of surgical instruments that should be placed in each storage box.

[0009] Preferably, the image acquisition module uses a polynomial distortion correction algorithm to eliminate the image distortion effect caused by the lens. Its correction formula is: ; ; Among them, is the coordinate in the distorted image, is the coordinate after correction, , , and are distortion coefficients, and are the offsets caused by radial distortion.

[0010] Preferably, the recognition module uses a convolutional neural network for surgical instrument recognition. By constructing a deep network model, the problem of gradient disappearance and gradient explosion during the training process of the neural network is solved. Its algorithm formula is: ; Among them, is the input image data, and these data are from the images after distortion correction by the image acquisition module. is the output recognition result, including the category and model information of the surgical instrument. represents the convolution and activation function operations. is the weight parameter in the network.

[0011] Preferably, the storage module uses a flash chip with high-speed reading and writing, which has the ability to quickly store and read data. In the storage management strategy based on heat grading, to reasonably allocate storage resources, for the heat value the calculation is performed using the following formula: ; Among them, is the access count of the instrument information, which is obtained by counting through the system log. is the total time, which is calculated based on the starting time of the system operation. the access count in the most recent period of time, which is also counted through the log. is the total access count. and are the weight coefficients.

[0012] Preferably, the comparison module uses a method combining feature point matching and shape context matching to compare various surgical instruments collected by the image acquisition module with the corresponding surgical instruments stored in the storage module. The specific method is as follows: During shape context matching, for two feature points and the shape context distance between them is calculated using the following formula: ; Among them, and are the counts of the feature points and in the th histogram interval, which are obtained through statistical analysis of the pixels in the neighborhood around the feature points. is the number of histogram intervals.

[0013] Preferably, the determination module uses multi-view image fusion technology to judge whether the surgical instrument is damaged. Using the weighted average fusion algorithm, for the pixel value of the fused image The calculation is as follows; ; wherein, is the image pixel value under the th perspective. These images are obtained by a number of image acquisition devices, is the weight corresponding to the perspective, which is determined by evaluating the clarity and integrity factors of the key parts of the surgical instrument presented in the images of different perspectives, and , is the number of perspectives.

[0014] Preferably, the inventory classification and packing box adopts high-precision infrared sensing technology, which can sensitively detect the putting in and taking out of surgical instruments and has an anti-mis-trigger mechanism. For the validity judgment of the sensing signal, a threshold judgment algorithm based on a sliding window is adopted. Let the sum of the signal values in the sliding window be , the window size be , and the threshold be , then the judgment formula for the signal validity is: ; Through this algorithm, it is ensured that the sensing signal accurately reflects the putting in and taking out of surgical instruments, avoiding mis-sensing caused by external interference.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. Through the coordinated operation of the augmented reality display module, recognition module, storage module, comparison module, determination module, voice prompt module and storage module, the present invention realizes the intelligentization of the inventory classification and packing of surgical instruments, greatly improving the work efficiency. The staff wears a head-mounted display, and the image acquisition module can quickly acquire images of surgical instruments and the surrounding environment. The recognition module uses advanced algorithms to quickly recognize the instruments. The storage module provides rich instrument information support. The comparison and determination modules accurately judge the instrument status. The voice prompt module issues alarms in a timely manner. The storage module automatically senses and updates the number of instruments. The entire process is efficient and smooth, greatly shortening the time required for inventory classification and packing. And through the built-in display screen of the head-mounted display, the name and quantity of the corresponding surgical instrument are projected at the position of each inventory classification and packing box, and the corresponding packing box where the surgical instrument is located is located through the combination of virtual and real, which is beneficial for the staff to quickly and accurately place the surgical instrument in the corresponding packing box. And the virtual numbers are displayed on the display screen at the position of the packing box, which is convenient for the staff to understand the number of the corresponding surgical instruments in the packing box, thus avoiding the situation of missing count, overcount or wrong count of surgical instruments, and solving the problems of low efficiency and difficult accuracy guarantee in the manual inventory classification and packing after the disinfection and cleaning of existing surgical instruments.

[0016] 2. With the help of high-precision image acquisition technology and advanced recognition algorithms, the present invention significantly improves the accuracy of counting and classifying surgical instruments. The image acquisition module adopts a polynomial distortion correction algorithm to effectively eliminate the image distortion caused by the lens and provide clear and accurate image data for the recognition module. The recognition module uses a convolutional neural network to accurately identify various surgical instruments. Even instruments with similar appearances can be accurately distinguished, avoiding common mistakes in manual operations and ensuring the correct quantity and types of the packed instruments.

[0017] 3. Through multi-view image fusion technology and precise comparison algorithms, the present invention can promptly detect damage to surgical instruments. The determination module combines the comparison results and the fused images, analyzes them according to the preset multi-dimensional damage determination rules. Once damage to an instrument is detected, a signal with detailed damage information is immediately generated and transmitted to the voice prompt module and related display systems to promptly alert the staff, effectively preventing damaged instruments from entering the surgical procedure and ensuring the safety of the surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To facilitate the understanding of the technical solution of the present invention by those skilled in the art, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings of the specification.

[0020] Embodiment 1. As Figure 1 shown, the present invention provides an intelligent packing table for counting and classifying surgical instruments, including: An augmented reality display module, including a head-mounted display and a plurality of image acquisition modules. The plurality of image acquisition modules are dispersedly arranged on the head-mounted display, used to collect images of surgical instruments and the surrounding environment on the packing table, display the image information collected by the image acquisition modules, and at the same time project the corresponding surgical instrument names and quantities at the position of each counting, classifying and packing box through the built-in display screen of the head-mounted display. The quantity is updated in reverse order as surgical instruments are placed in. A recognition module, connected to the image acquisition module and used to recognize various surgical instruments collected by the image acquisition module; A storage module, used to store information about various surgical instruments; A comparison module, connected to the recognition module and the storage module, used to compare various surgical instruments collected by the image acquisition module with the corresponding surgical instruments stored in the storage module; A determination module, connected to the comparison module, used to accurately determine whether a surgical instrument is damaged based on the comparison result of the comparison module; A voice prompt module, connected to the determination module, used to emit a damage prompt sound when the determination module determines that a surgical instrument is damaged; The storage module includes an inventory classification and packing box, which is used to accurately sense the putting in and taking out of surgical instruments and update the instrument quantity in real time.

[0021] In an embodiment of the present invention, the head-mounted display is built-in with a high-precision position sensor. When the staff faces the storage box, the position sensor quickly identifies the position and attitude of the storage box. Based on the augmented reality technology, on the display screen of the head-mounted display, the name and quantity of the corresponding surgical instrument are accurately projected at the position corresponding to each storage box. The name and initial quantity information of the surgical instrument are retrieved from the storage module, and the storage module records the types and quantities of surgical instruments that should be placed in each storage box.

[0022] In an embodiment of the present invention, the image acquisition module uses a polynomial distortion correction algorithm to eliminate the image distortion effect caused by the lens. Its correction formula is: ; ; Among them, is the coordinate in the distorted image, is the coordinate after correction, , , and are distortion coefficients, which are obtained by fitting a large amount of experimental data. and are the offsets caused by radial distortion. The image processed by this correction algorithm can provide a more accurate image data basis for the subsequent recognition module and effectively improve the recognition accuracy.

[0023] In an embodiment of the present invention, the recognition module uses a convolutional neural network to recognize surgical instruments. By constructing a deep network model, the problem of gradient disappearance and gradient explosion in the training process of the neural network is solved. Its algorithm formula is: ; Among them, is the input image data, and these data are from the image after distortion correction by the image acquisition module. is the output recognition result, including the category and model information of the surgical instrument. represents the convolution and activation function operations, and through these operations, feature extraction and classification are performed on the input image. are the weight parameters in the network, which are continuously optimized and adjusted through supervised learning and unsupervised learning on a large number of surgical instrument sample images. The deep network model constructed by this formula can use the image data corrected by the image acquisition module to achieve accurate recognition of surgical instruments, and can accurately distinguish even for surgical instruments of different brands, models and with slight differences.

[0024] In an embodiment of the present invention, the storage module uses a flash memory chip with high-speed read and write capabilities, which has the ability to quickly store and read data. In the storage management strategy based on heat level classification, to reasonably allocate storage resources, for the heat value the calculation is performed using the following formula: ; wherein, is the number of accesses to the instrument information, which is obtained by counting the system logs, is the total time, which is calculated based on the start time of the system operation, is the number of accesses in a recent period of time, which is also counted through the logs, is the total number of accesses, and are the weight coefficients, and their values are determined through experimental tests and data analysis according to the actual application scenario and data access rules. Generally, the value range is between 0 and 1 and According to this heat value, the surgical instrument information is reasonably stored in the cache area or the extended storage area. For the frequently used instrument information with a high heat value, it is stored in the cache area so that the recognition module and the comparison module can quickly retrieve it, improving the system operation efficiency.

[0025] In an embodiment of the present invention, the comparison module uses a method that combines feature point matching and shape context matching to compare various surgical instruments collected by the image acquisition module with the corresponding surgical instruments stored in the storage module. The specific method is as follows: During shape context matching, for two feature points and the shape context distance between them is calculated using the following formula: ; wherein, and are the counts of the feature points and in the th histogram bin, respectively, which are obtained by statistical analysis of the pixels in the neighborhood around the feature points. is the number of histogram bins, which is determined according to the complexity of the surgical instrument features and the accuracy requirements. Generally, the value ranges from 10 to 100. Through this distance formula, combined with the instrument features output by the recognition module and the standard instrument features provided by the storage module, accurate feature comparison can be achieved. For example, for similar surgical forceps, the subtle differences can be accurately distinguished through this algorithm, providing accurate comparison results for the determination module to assist in determining the accuracy and integrity of the surgical instruments.

[0026] In an embodiment of the present invention, the determination module determines whether a surgical instrument is damaged through multi-view image fusion technology, and adopts a weighted average fusion algorithm for the pixel values of the fused image. The calculation is as follows: ; Wherein, is the pixel value of the image from the th view, and these images are obtained by several image acquisition devices. is the weight corresponding to the view, which is determined by evaluating the clarity and integrity factors of the key parts of the surgical instrument presented in different view images, and , is the number of views. According to the result of the comparison module, the fused image is analyzed according to the preset multi-dimensional damage determination rules, and these rules include the quantitative judgment criteria for the degree of deformation of the instrument shape, the depth and length of the surface crack, and the wear condition of the key parts. If it is determined that the instrument is damaged, a damage signal with detailed damage information is immediately generated and transmitted to the voice prompt module and the augmented reality display and interaction system to timely remind the staff.

[0027] In an embodiment of the present invention, the inventory classification packing box adopts high-precision infrared sensing technology, which can sensitively detect the putting in and taking out of surgical instruments and has an anti-mis-trigger mechanism. For the judgment of the validity of the sensing signal, a threshold judgment algorithm based on a sliding window is adopted. Let the sum of the signal values in the sliding window be , and the window size is , which is determined according to the fluctuation characteristics of the sensing signal and the actual use environment, and generally takes a value between 5 and 20. The threshold is , which is obtained through experimental tests and data analysis. The thresholds of different types of sensing devices are different, and the judgment formula for the signal validity is: ; Through this algorithm, it is ensured that the sensing signal accurately reflects the putting in and taking out of surgical instruments and avoids mis-sensing caused by external interference.

[0028] The embodiments disclosed in the present invention are preferred embodiments, but are not limited thereto. Those of ordinary skill in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes, but as long as they do not depart from the spirit of the present invention, they are within the protection scope of the present invention.

Claims

1. An intelligent packaging station based on surgical instrument inventory and classification, characterized in that: include: The augmented reality display module includes a head-mounted display and a plurality of image acquisition modules. The plurality of image acquisition modules are dispersedly arranged on the head-mounted display and are used to acquire images of surgical instruments and surrounding environments on the packing table, and to display the image information acquired by the image acquisition modules. At the same time, the name and quantity of the corresponding surgical instruments are projected at each inventory classification packing box position through the built-in display screen of the head-mounted display, and the quantity is updated in reverse order as the surgical instruments are put in. An identification module, connected to the image acquisition module, and used to identify various surgical instruments acquired by the image acquisition module; A storage module, used to store information of various surgical instruments; A comparison module, connected to the recognition module and the storage module, for comparing various surgical instruments acquired by the image acquisition module with corresponding surgical instruments stored in the storage module; A determination module, connected to the comparison module, for accurately determining whether the surgical instrument is damaged according to the comparison result of the comparison module; A voice prompt module, connected to the determination module, for issuing a damage prompt sound when the determination module determines that the surgical instrument is damaged; The storage module includes an inventory and classification packaging box, which is used to accurately sense the placement and removal of surgical instruments and update the number of instruments in real time.

2. According to claim 1, an intelligent packaging station based on surgical instrument inventory and classification is characterized in that: The head mounted display has a built-in high-precision position sensor. When the worker faces the storage box, the position sensor quickly identifies the position and posture of the storage box. Based on augmented reality technology, the name and quantity of the corresponding surgical instruments are accurately projected at the corresponding position of each storage box on the display screen of the head-mounted display. The name and initial quantity information of the surgical instruments are retrieved from the storage module, which records the type and quantity of surgical instruments that should be placed in each storage box.

3. According to claim 1, the intelligent packaging station based on surgical instrument inventory and classification is characterized in that: The image acquisition module uses a polynomial distortion correction algorithm to eliminate the image distortion effect caused by the lens. The correction formula is: ； ； in, are the coordinates in the distorted image, is the corrected coordinate, , , and is the distortion coefficient, and is the offset caused by radial distortion.

4. The intelligent packaging station based on surgical instrument inventory and classification according to claim 1 is characterized in that: The recognition module uses a convolutional neural network to recognize surgical instruments. By building a deep network model, it solves the gradient vanishing and gradient exploding problems of the neural network during training. The algorithm formula is: ； in, The input image data comes from the image after distortion correction in the image acquisition module. The output recognition results include the category and model information of the surgical instrument. represents the convolution and activation function operations, is the weight parameter in the network.

5. The intelligent packaging station based on surgical instrument inventory and classification according to claim 1 is characterized in that: The storage module uses a high-speed read-write flash memory chip, which has the ability to quickly store and read data. In the storage management strategy based on heat classification, in order to reasonably allocate storage resources, the heat value The calculation of is as follows: ； in, The number of times the device information is accessed is obtained through system log records. is the total time, calculated based on the start time of the system operation. The number of visits in the recent period is also counted through logs. is the total number of visits, and is the weight coefficient.

6. The intelligent packaging station based on surgical instrument inventory and classification according to claim 1 is characterized in that: The comparison module uses a method based on a combination of feature point matching and shape context matching to compare various surgical instruments acquired by the image acquisition module with the corresponding surgical instruments stored in the storage module. The specific method is as follows: When shape context matching, for two feature points and The shape context distance between The calculation formula is as follows: ； in, and They are feature points and In the The counts within a histogram interval are obtained by statistical analysis of the neighborhood pixels around the feature point. is the number of histogram bins.

7. The intelligent packaging station based on surgical instrument inventory and classification according to claim 1 is characterized in that: The determination module determines whether the surgical instrument is damaged by using multi-view image fusion technology, and adopts a weighted average fusion algorithm to calculate the pixel value of the fused image. The calculation is as follows; ； in, For the The image pixel values ​​at each viewing angle are acquired by several image acquisition devices. The weight of the corresponding viewing angle is determined by evaluating the clarity and integrity factors of the key parts of the surgical instrument in images from different viewing angles, and , is the number of viewing angles.

8. The intelligent packaging station based on surgical instrument inventory and classification according to claim 1 is characterized in that: The counting and classification packaging box adopts high-precision infrared sensing technology, which can sensitively detect the insertion and removal of surgical instruments and has an anti-false triggering mechanism. For the validity judgment of the sensing signal, a threshold judgment algorithm based on a sliding window is adopted. The sum of the signal values ​​in the sliding window is set to , the window size is , the threshold is , then the signal validity The judgment formula is: ； This algorithm ensures that the sensing signal accurately reflects the insertion and removal of surgical instruments, avoiding false sensing due to external interference.

Images