Operation management method and system for smart operating room

Abstract

The invention provides an operation management method and system for a smart operating room, and relates to the technical field of smart operating room operation management, image information of surgical instruments is acquired, and instrument identifiers and instrument image surface features of the surgical instruments are extracted from the image information; obtaining an instrument dissimilation degree of each surgical instrument according to the corresponding instrument image surface features, and determining a visual state code of each surgical instrument according to the corresponding instrument identifier and the instrument dissimilation degree; performing spatial mapping by using augmented reality terminal equipment, generating an augmented reality-based instrument dynamic visual image, and determining a state correction result of each surgical instrument according to the interactive operation information and the instrument dynamic visual image; according to the state correction result of each surgical instrument, postoperative instrument state statistical information is generated, and surgical instrument counting is completed. According to the method, a dynamic visualization mechanism based on augmented reality can be constructed in a complicated post-operation environment, so that the counting efficiency and precision of the surgical instruments are improved.

Description

Technical Field

[0001] The present application relates to the technical field of smart operating room operation and management. More specifically, the present application relates to a smart operating room operation and management method and system. Background Art

[0002] With the continuous improvement of medical technology, the types and complexity of surgeries continue to increase, and the management requirements of operating rooms are becoming increasingly stringent. Surgical instruments are indispensable key materials in the surgical process. Their inventory, management and use traceability are directly related to the safety of the operation and the quality of medical care. At present, most hospitals still use manual methods to count surgical instruments, judge their status and conduct postoperative statistics. This method relies on the experience of medical staff and is easily affected by human factors. There are problems such as inventory omissions, recording errors, and inaccurate instrument status judgment. This is especially prominent in the environment of multiple instruments and high frequency of surgery.

[0003] Although some existing surgical instrument management solutions are based on QR codes or radio frequency identification, they rely on labels, are easily affected by the environment, and lack detailed judgment of the instrument status. In recent years, the rapid development of technologies such as computer vision, augmented reality (AR), and spatial mapping (SLAM) has provided new technical paths for intelligent management of operating rooms. By intelligently analyzing surgical instrument image information and combining it with an augmented reality interactive interface, not only can the accuracy of recognition be improved, but dynamic visualization and interactive correction of instrument status can also be achieved, thus providing reliable support for postoperative inventory, quality tracking, and risk warning. Therefore, how to build a dynamic visualization mechanism based on augmented reality in complex postoperative environments to improve the efficiency and accuracy of surgical instrument inventory has become a difficult problem facing the industry. Summary of the Invention

[0004] This application provides an operation and management method and system for a smart operating room, which can build a dynamic visualization mechanism based on augmented reality in a complex postoperative environment to improve the efficiency and accuracy of surgical instrument counting.

[0005] In a first aspect, the present application provides an operation and management method for a smart operating room, the operation and management method comprising the following steps: Acquire image information of surgical instruments after surgery in the operating room, and extract instrument identification and instrument image surface features of each surgical instrument from the image information; The instrument alienation measurement is performed on each surgical instrument based on the corresponding instrument image surface features, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and the visual status code of each surgical instrument is determined by the corresponding instrument identification and instrument alienation degree; Use an augmented reality terminal device to spatially map the postoperative scene in the operating room, and then generate an augmented reality-based dynamic visualization image of the instrument based on the spatial mapping results and the visual status code of each surgical instrument. Determine the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument; According to the status correction results of each surgical instrument, the postoperative instrument status statistics of the operating room are generated to complete the inventory of surgical instruments.

[0006] In this embodiment, extracting the instrument identification and instrument image surface features of each surgical instrument from the image information specifically includes: Performing distortion correction on the image information to obtain corrected image data; Extracting the contour identification area of ​​the surgical instrument from the corrected image data, thereby obtaining the identification area of ​​each surgical instrument; Performing two-dimensional geometric feature extraction on the marked area of ​​each surgical instrument to obtain two-dimensional shape information of each surgical instrument; Determining the instrument identification of each surgical instrument by corresponding two-dimensional shape information; Surface features of surgical instruments are extracted from the corrected image data to obtain instrument image surface features of each surgical instrument.

[0007] In this embodiment, the instrument alienation measurement of each surgical instrument is performed based on the corresponding instrument image surface features, and the instrument alienation degree of each surgical instrument in the operating room is obtained, which specifically includes: Obtaining reference image features corresponding to each surgical instrument in the historical database; For each surgical instrument in the operating room, determining a feature reference homogeneity of the surgical instrument based on the instrument image surface features and the corresponding reference image features; The instrument alienation degree of the surgical instrument is determined by the feature reference homogeneity, and then the instrument alienation degree of each surgical instrument in the operating room is obtained.

[0008] In this embodiment, determining the visual status code of each surgical instrument through the corresponding instrument identification and instrument alienation degree specifically includes: Classify the status of each surgical instrument according to the corresponding instrument identification and instrument alienation degree, and then obtain the status level of each surgical instrument; A visual status code for each surgical instrument is determined based on the corresponding status level.

[0009] In this embodiment, generating an augmented reality-based dynamic visualization image of an instrument based on the spatial mapping results and the visual status code of each surgical instrument specifically includes: Based on the spatial mapping results, the visual status codes of each surgical instrument are three-dimensionally mapped to obtain an instrument space mapping relationship table; Constructing an augmented reality visualization layer through the instrument space mapping relationship table; The augmented reality visualization layer is dynamically superimposed on the real image of the operating room to obtain an augmented reality-based dynamic visualization image of the instrument.

[0010] In this embodiment, determining the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument specifically includes: Collect interactive operation information during the equipment inventory process; Generate a state correction decision based on the interactive operation information and the visual state code of each surgical instrument in the dynamic visualization image of the instrument; The state correction result of each surgical instrument is determined according to the state correction decision and the instrument dynamic visualization image.

[0011] In this embodiment, generating the postoperative instrument status statistics of the operating room according to the status correction results of each surgical instrument specifically includes: Obtaining the surgical instruments involved in the surgical process to generate a surgical instrument list, matching the state correction results of each surgical instrument with the surgical instrument list, and establishing an instrument state statistical model; All surgical instruments are clustered and statistically analyzed according to the instrument status statistical model to obtain postoperative instrument status information in the operating room.

[0012] In a second aspect, the present application provides a smart operating room operation management system for executing a smart operating room operation management method, the operation management system comprising: A feature extraction module is used to obtain image information of surgical instruments after surgery in the operating room, and extract instrument identification and instrument image surface features of each surgical instrument from the image information; A status determination module is used to measure the instrument alienation of each surgical instrument based on the surface features of the corresponding instrument image, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and determining the visual status code of each surgical instrument based on the corresponding instrument identification and instrument alienation degree; A state correction module is used to use an augmented reality terminal device to perform spatial mapping of the postoperative scene in the operating room, and then generate an augmented reality-based dynamic visualization image of the instrument based on the spatial mapping result and the visual status code of each surgical instrument. The state correction result of each surgical instrument is determined based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument; The instrument statistics module is used to generate postoperative instrument status statistics in the operating room based on the status correction results of each surgical instrument and complete the inventory of surgical instruments.

[0013] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned smart operating room operation and management method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned smart operating room operation and management method when executed.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: Obtain image information of surgical instruments after surgery in the operating room, extract the instrument identification and instrument image surface features of each surgical instrument from the image information; perform instrument alienation measurement on each surgical instrument based on the corresponding instrument image surface features, and then obtain the instrument alienation degree of each surgical instrument in the operating room, and determine the visual status code of each surgical instrument through the corresponding instrument identification and instrument alienation degree; use an augmented reality terminal device to perform spatial mapping of the postoperative scene in the operating room, and then generate an augmented reality-based instrument dynamic visualization image based on the spatial mapping result and the visual status code of each surgical instrument, determine the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the instrument dynamic visualization image; generate statistical information on the postoperative instrument status of the operating room based on the status correction result of each surgical instrument, and complete the surgical instrument inventory.

[0016] It can be seen that in this application, firstly, by extracting the instrument identification and instrument image surface features of each surgical instrument, the automatic recognition of surgical instruments after surgery is realized, which provides a basis for the inventory of surgical instruments; then, by comparing the instrument image surface features of the surgical instrument with its historical reference image features, the refined measurement of the degree of instrument alienation is realized, and then the instrument alienation degree of each surgical instrument is accurately calculated, and its visual status code is determined in combination with the instrument identification, which can intelligently identify the wear, contamination or damage of the instrument, effectively avoid human subjective judgment errors, and greatly improve the intelligence and refinement level of postoperative instrument management; secondly, by using augmented reality terminal devices to The postoperative scene in the operating room is spatially mapped, and dynamic visualization images are generated in combination with the visual status codes of surgical instruments. This can achieve real-time, intuitive, and spatially positioned presentation of the instrument status. At the same time, combined with the interactive operation information during the inventory process, the instrument status can be corrected in a timely manner, reducing manual omissions and misjudgments, and effectively improving the efficiency and accuracy of the postoperative surgical instrument management link in the smart operating room; finally, by generating postoperative instrument status information based on the status correction results of each surgical instrument and combining it with the dynamic visualization mechanism of augmented reality, it can not only clearly and intuitively display the spatial distribution and status level of surgical instruments in the postoperative scene, but also significantly improve the efficiency and accuracy of surgical instrument inventory.

[0017] In summary, the technical solution adopted in this application can construct a dynamic visualization mechanism based on augmented reality in a complex postoperative environment to improve the efficiency and accuracy of surgical instrument counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 is an exemplary flow chart of the operation and management method of the smart operating room provided in this application; Figure 2 is an exemplary flow chart for determining the instrument alienation degree of each surgical instrument in an operating room according to the present application; Figure 3 is an exemplary flow chart for determining a dynamic visualization image of an instrument based on augmented reality according to the present application; Figure 4 This is a module structure diagram of the smart operating room operation and management system provided by this application; Figure 5This is a structural diagram of a computer device for implementing a smart operating room operation and management method provided in this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] The present application provides an operation and management method, system, device, and storage medium for a smart operating room. The core of the method is to obtain image information of surgical instruments in the operating room after surgery, extract the instrument identification and instrument image surface features of each surgical instrument from the image information, measure the instrument alienation of each surgical instrument based on the corresponding instrument image surface features, and then obtain the instrument alienation degree of each surgical instrument in the operating room. The visual status code of each surgical instrument is determined based on the corresponding instrument identification and instrument alienation degree. The postoperative scene in the operating room is spatially mapped using an augmented reality terminal device, and then an augmented reality-based instrument dynamic visualization image is generated based on the spatial mapping result and the visual status code of each surgical instrument. The state correction result of each surgical instrument is determined based on the interactive operation information during the instrument inventory process and the instrument dynamic visualization image. The postoperative instrument status statistical information of the operating room is generated based on the state correction result of each surgical instrument to complete the surgical instrument inventory. The above scheme can be used to build an augmented reality-based dynamic visualization mechanism in a complex postoperative environment to improve the efficiency and accuracy of surgical instrument inventory.

[0022] Example 1: In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG, this figure is an exemplary flow chart of an operation management method of a smart operating room according to this embodiment of the present application, and the operation management method includes the following steps: In step S1 , image information of surgical instruments after surgery in an operating room is acquired, and instrument identifications and instrument image surface features of each surgical instrument are extracted from the image information.

[0023] In specific implementation, an RGB camera and an AR (Augmented Reality) spatial positioning module can be installed in the operating room in advance to obtain the initial image information of the postoperative surgical instruments in the operating room, and the image information can be preprocessed. The image information can be grayscaled, binarized, median or Gaussian filtered to reduce noise, and obtain the preprocessed image information of the postoperative surgical instruments in the operating room. It should be noted that the preprocessed image information can reduce the complexity of the image, improve the accuracy and robustness of the recognition algorithm, and improve the image quality and contrast; the image information of the postoperative surgical instruments in the operating room includes an RGB image sequence and AR spatial positioning information.

[0024] In this embodiment, the following steps may be used to extract the instrument identification and instrument image surface features of each surgical instrument from the image information: Performing distortion correction on the image information to obtain corrected image data; Extracting the contour identification area of ​​the surgical instrument from the corrected image data, thereby obtaining the identification area of ​​each surgical instrument; Performing two-dimensional geometric feature extraction on the marked area of ​​each surgical instrument to obtain two-dimensional shape information of each surgical instrument; Determining the instrument identification of each surgical instrument by corresponding two-dimensional shape information; Surface features of surgical instruments are extracted from the corrected image data to obtain instrument image surface features of each surgical instrument.

[0025] In the specific implementation, first, the camera calibration parameters can be obtained from the RGB camera, and the image information can be distorted using OpenCV to obtain the corrected image data; then, the contour identification area of ​​the surgical instrument can be extracted from the corrected image data, that is, the corrected image data can be grayscaled to obtain a grayscale image of the corrected image data, and the edge detection algorithm can be used to extract the edge information of the grayscale image to obtain a binary edge map, wherein the binary edge map is an image containing only black and white pixel values, which is the basic form of extracting the image structure contour, and the contour search function is called to extract all closed edge paths in the binary edge map, and all extracted contours are searched according to the aspect ratio of the surgical instrument. The lines are filtered to obtain the identification area of ​​each surgical instrument; secondly, the identification area of ​​each surgical instrument can be subjected to two-dimensional geometric feature extraction, that is, the minimum circumscribed rectangle of the identification area of ​​the corresponding surgical instrument is calculated to obtain its center point coordinates, and the OpenCV function is used to perform geometric measurement on the identification area of ​​the surgical instrument to obtain the area and perimeter of the identification area of ​​the surgical instrument, and its aspect ratio and roundness are calculated. The center point coordinates, area and perimeter, aspect ratio and roundness of the identification area of ​​the surgical instrument are combined into a feature vector to obtain the two-dimensional shape information of the identification area of ​​the corresponding surgical instrument. The two-dimensional shape information of each surgical instrument can be obtained in the above manner.

[0026] In addition, in the specific implementation, first, the instrument identification of each surgical instrument can be determined through the corresponding two-dimensional shape information, that is, all standard instrument information in the historical database is obtained, and the two-dimensional shape information of the corresponding surgical instrument is feature matched with the various standard instrument information in the historical database, that is, the existing similarity algorithm can be used to calculate the information similarity between the two-dimensional shape information and the various standard instrument information, so that the standard instrument information with the highest information similarity and higher than the preset recognition threshold is used as the instrument identification of the corresponding surgical instrument, where it should be noted that the preset recognition threshold can be set through historical experience, and the instrument identification includes the category, function and model of the standard surgical instrument; then, the surface features of the surgical instrument can be extracted from the corrected image data, that is, the structural features, surface gloss and texture surface features of each surgical instrument are extracted through image feature extraction technology, so that the feature vector composed of the structural features, surface gloss and texture surface features is used as the instrument image surface feature of each surgical instrument.

[0027] It should be noted that by obtaining image information of postoperative surgical instruments in the operating room and extracting the instrument identification and instrument image surface features of each surgical instrument, automatic identification of postoperative surgical instruments is achieved, providing a basis for subsequent inventory of surgical instruments.

[0028] In step S2, the instrument alienation measurement is performed on each surgical instrument based on the corresponding instrument image surface features, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and the visual status code of each surgical instrument is determined by the corresponding instrument identification and instrument alienation degree.

[0029] Preferably, in this embodiment, reference Figure 2 As shown in FIG. 1 , this figure is an exemplary flow chart for determining the instrument alienation degree of each surgical instrument in an operating room according to an embodiment of the present application. In this embodiment, the instrument alienation degree of each surgical instrument is measured based on the surface features of the corresponding instrument image, and the instrument alienation degree of each surgical instrument in the operating room is obtained. Specifically, the following steps can be used to achieve this: In step S21, the reference image features corresponding to each surgical instrument in the historical database are obtained; In step S22, for each surgical instrument in the operating room, the feature reference homogeneity of the surgical instrument is determined based on the instrument image surface features and the corresponding reference image features of the surgical instrument; In step S23, the instrument heterogeneity of the surgical instrument is determined by using the feature reference homogeneity, thereby obtaining the instrument heterogeneity of each surgical instrument in the operating room.

[0030] It should be noted that in this application, the reference image features are standard features in the surgical instrument image, which include standard structural features, standard glossiness and standard texture surface features. Among them, local changes in texture surface features (such as fine scratches, changes in surface roughness) can reflect the physical wear and tear caused during the use of the instrument, the surface glossiness can reflect the aging and cleaning conditions of the surgical instrument, and the changes in structural features can reflect the damage, wear and deformation of the surgical instrument during use; feature reference homogeneity is an indicator used to represent the degree of similarity between the instrument image surface features and the reference image features of the surgical instrument; the instrument alienation degree represents the degree of abnormal deviation between the postoperative state of the surgical instrument and the standard state.

[0031] In specific implementation, first, the benchmark image features corresponding to each surgical instrument can be extracted from the historical database; then, for each surgical instrument in the operating room, the product of the vector modulus of the instrument image surface feature of the surgical instrument and the vector modulus of the corresponding benchmark image feature can be calculated, and the ratio of the dot product of the instrument image surface feature and the benchmark image feature to the calculation result is used as the feature benchmark homogeneity of the surgical instrument. The feature benchmark homogeneity of each surgical instrument can be obtained in the above manner, and then a homogeneity inversion mapping is performed according to the feature benchmark homogeneity of the corresponding surgical instrument, that is, 1 is subtracted from the feature benchmark homogeneity corresponding to the surgical instrument, and the result is used as the instrument alienation degree of the surgical instrument. The instrument alienation degree of each surgical instrument in the operating room can be obtained through the above steps.

[0032] In this embodiment, the visual status code of each surgical instrument can be determined by the corresponding instrument identification and instrument alienation degree by the following steps: Classify the status of each surgical instrument according to the corresponding instrument identification and instrument alienation degree, and then obtain the status level of each surgical instrument; A visual status code for each surgical instrument is determined based on the corresponding status level.

[0033] In a specific implementation, first, the status level of each surgical instrument can be divided according to the corresponding instrument identification and instrument alienation degree, that is, a multi-level alienation degree threshold can be preset for each surgical instrument based on a historical database, wherein the multi-level alienation degree threshold is a plurality of interval critical values ​​used to divide the instrument status level. According to the instrument identification, the grading rules corresponding to the surgical instrument are matched, and one surgical instrument corresponds to a set of multi-level alienation degree thresholds. By determining the multi-level alienation degree threshold corresponding to the instrument alienation degree of the surgical instrument, the interval of the instrument alienation degree of the surgical instrument can be obtained, and the status level of the surgical instrument can be divided according to the interval of the instrument alienation degree of the surgical instrument, thereby obtaining the status level of the surgical instrument; then, based on the status level of the surgical instrument, the status level of the surgical instrument can be recorded using a visual status code. The visual status code of each surgical instrument can be determined in the above manner, wherein the visual status code is identification information used to describe the current health status and availability of the surgical instrument.

[0034] It should be noted that by comparing the surface features of the surgical instrument image with its historical reference image features, a refined measurement of the degree of instrument alienation can be achieved, and then the degree of instrument alienation of each surgical instrument can be accurately calculated. In combination with the instrument identification, its visual status code can be determined. This can not only intelligently identify the wear, contamination or damage of the instrument and effectively avoid human subjective judgment errors, but also achieve differentiated management and risk warning, provide a scientific basis for the reuse, safe scrapping and maintenance decisions of the instrument, and greatly improve the intelligence and refinement of postoperative instrument management.

[0035] In step S3, an augmented reality terminal device is used to spatially map the postoperative scene in the operating room, and then an augmented reality-based dynamic visualization image of the instrument is generated based on the spatial mapping results and the visual status code of each surgical instrument. The status correction result of each surgical instrument is determined based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument.

[0036] In this embodiment, an augmented reality terminal device is used to perform spatial mapping of the postoperative scene in the operating room. In specific implementation, the augmented reality terminal device can be started to obtain postoperative image information of the operating room, and the SLAM (Simultaneous Localization and Mapping) algorithm can be used to dynamically map the postoperative scene in the operating room. During the mapping process, an image recognition model (such as YOLO) is combined to detect and identify surgical instruments remaining after the operation, and the identified surgical instruments are anchored in the spatial mapping to obtain the spatial coordinates of each surgical instrument.

[0037] Preferably, in this embodiment, reference Figure 3 As shown in FIG. 1 , this figure is an exemplary flow chart for determining an augmented reality-based dynamic visualization image of an instrument in an embodiment of the present application. In this embodiment, the augmented reality-based dynamic visualization image of an instrument is generated based on the spatial mapping results and the visual status codes of each surgical instrument. Specifically, the following steps can be used to implement this: In step S31, the visual status code of each surgical instrument is three-dimensionally mapped according to the spatial mapping result, thereby obtaining an instrument space mapping relationship table; In step S32, an augmented reality visualization layer is constructed using the instrument space mapping relationship table; In step S33, the augmented reality visualization layer is dynamically superimposed on the real image of the operating room to obtain an augmented reality-based dynamic visualization image of the instrument.

[0038] In the specific implementation, first, the spatial coordinates of each surgical instrument identified are sorted according to the spatial mapping results, and the visual status code of each surgical instrument is bound to its coordinate point in the spatial mapping result to form an instrument space mapping relationship table, wherein it should be noted that the instrument space mapping relationship table records the mapping relationship between each surgical instrument and its spatial coordinates and status information in the form of a data table; then, an augmented reality visualization layer can be constructed through the instrument space mapping relationship table, that is, according to the instrument space mapping relationship table, a corresponding augmented reality visualization layer is generated for each surgical instrument, wherein the visual layer is generated in real time by the rendering engine (Unity3D) in the AR terminal; secondly, the augmented reality visualization layer is dynamically superimposed with the real image of the operating room to obtain an augmented reality-based instrument dynamic visualization image, that is, the rendering pipeline of the AR device can be used to accurately map the augmented reality visualization layer according to the spatial coordinates of each surgical instrument and superimpose it on the spatial position of the real surgical instrument to obtain an augmented reality-based instrument dynamic visualization image.

[0039] In this embodiment, the following steps may be used to determine the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument: Collect interactive operation information during the equipment inventory process; Generate a state correction decision based on the interactive operation information and the visual state code of each surgical instrument in the dynamic visualization image of the instrument; The state correction result of each surgical instrument is determined according to the state correction decision and the instrument dynamic visualization image.

[0040] In specific implementation, first, the dynamic visualization image of the instrument can be displayed on the augmented reality terminal device (such as AR head display, AR tablet), and by starting the interactive acquisition interface in the augmented reality terminal device, the captured touch screen click instructions are used as interactive operation information in the instrument inventory process, and the augmented reality terminal device monitors and records the interactive operation information in real time; then, the state correction decision is generated through the interactive operation information and the visual status code of each surgical instrument in the dynamic visualization image of the instrument, that is, the interactive operation information in the instrument inventory process and the visual status code of each surgical instrument in the dynamic visualization image of the instrument can be matched through the augmented reality terminal device. The interactive operation information corresponding to the visual status code of each surgical instrument serves as the state correction decision of the surgical instrument. It should be noted that the state correction decision is a decision used to correct or confirm the visual status code of the corresponding surgical instrument. Finally, the state correction result of each surgical instrument is determined based on the state correction decision and the dynamic visualization image of the instrument. That is, the state correction decision can be used by the augmented reality terminal device to correct or confirm the visual status code of each surgical instrument in the dynamic visualization image of the instrument, thereby regenerating the visual status code of each surgical instrument, and the regenerated visual status code of each surgical instrument is used as the state correction result of each surgical instrument.

[0041] It should be noted that by using augmented reality terminal devices to spatially map the postoperative scene in the operating room, and combining the visual status codes of surgical instruments to generate dynamic visual images, real-time, intuitive, and spatially positioned presentation of the instrument status can be achieved, significantly improving the visualization level and interactive efficiency of postoperative surgical instrument inventory; at the same time, combined with the interactive operation information during the inventory process, the instrument status can be corrected in a timely manner, reducing manual omissions and misjudgments, enhancing the accuracy and traceability of information, and effectively improving the efficiency and accuracy of smart operating rooms in the postoperative surgical instrument management link.

[0042] In step S4, postoperative instrument status statistics of the operating room are generated based on the status correction results of each surgical instrument, and the surgical instrument inventory is completed.

[0043] In this embodiment, generating the postoperative instrument status statistics of the operating room according to the status correction results of each surgical instrument can be specifically implemented by the following steps: Obtaining the surgical instruments involved in the surgical process to generate a surgical instrument list, matching the state correction results of each surgical instrument with the surgical instrument list, and establishing an instrument state statistical model; All surgical instruments are clustered and statistically analyzed according to the instrument status statistical model to obtain postoperative instrument status information in the operating room.

[0044] In specific implementation, first, the surgical instruments involved in the operation can be obtained from the hospital instrument library to generate a surgical instrument list, and the state correction results of each surgical instrument can be mapped with the surgical instrument list to obtain an instrument state correction mapping table. Based on the state correction mapping table, an instrument state statistical model is constructed. The instrument state statistical model jointly analyzes the number of uses, actual detection number, and visual status code of each surgical instrument, and classifies the surgical instruments according to different states; then, all surgical instruments are clustered and counted according to the instrument state statistical model to obtain the postoperative instrument status information of the operating room, that is, the instrument state statistical model can be used to classify all surgical instruments. Cluster statistics are performed on surgical instruments to automatically identify whether there are surgical instruments with inconsistent inventory quantities, abnormal status or potential cross-contamination risks. The cluster statistics process of the instrument status statistical model can use the usage quantity, actual detection quantity and visual status code of each surgical instrument as the input of cluster analysis, and then use the DBSCAN (density-based clustering algorithm) clustering algorithm for cluster statistics, so that the cluster statistics results can be output in the form of a structured report to obtain postoperative instrument status information, wherein the postoperative instrument status information includes: the total number of surgical instruments, the number of intact instruments, the list of missing instruments, the details of damaged or contaminated instruments and their location markings, recommended treatment measures, etc.

[0045] It should be noted that by generating postoperative instrument status information based on the status correction results of each surgical instrument and combining it with the dynamic visualization mechanism of augmented reality, it can not only clearly and intuitively display the spatial distribution and status level of surgical instruments in the postoperative scene, but also assist medical staff in quickly locating, checking and troubleshooting instrument status in complex and changeable postoperative environments, significantly improving the efficiency and accuracy of surgical instrument inventory, reducing the risk of missed and wrong checks, and ensuring the standardization and safety of the postoperative management process.

[0046] It can be seen that in this application, firstly, by extracting the instrument identification and instrument image surface features of each surgical instrument, the automatic recognition of surgical instruments after surgery is realized, which provides a basis for the inventory of surgical instruments; then, by comparing the instrument image surface features of the surgical instrument with its historical reference image features, the refined measurement of the degree of instrument alienation is realized, and then the instrument alienation degree of each surgical instrument is accurately calculated, and its visual status code is determined in combination with the instrument identification, which can intelligently identify the wear, contamination or damage of the instrument, effectively avoid human subjective judgment errors, and greatly improve the intelligence and refinement level of postoperative instrument management; secondly, by using augmented reality terminal devices to The postoperative scene in the operating room is spatially mapped, and dynamic visualization images are generated in combination with the visual status codes of surgical instruments. This can achieve real-time, intuitive, and spatially positioned presentation of the instrument status. At the same time, combined with the interactive operation information during the inventory process, the instrument status can be corrected in a timely manner, reducing manual omissions and misjudgments, and effectively improving the efficiency and accuracy of the postoperative surgical instrument management link in the smart operating room; finally, by generating postoperative instrument status information based on the status correction results of each surgical instrument and combining it with the dynamic visualization mechanism of augmented reality, it can not only clearly and intuitively display the spatial distribution and status level of surgical instruments in the postoperative scene, but also significantly improve the efficiency and accuracy of surgical instrument inventory.

[0047] In summary, the technical solution adopted in this application can construct a dynamic visualization mechanism based on augmented reality in a complex postoperative environment to improve the efficiency and accuracy of surgical instrument counting.

[0048] Example 2: This application provides a reference for a smart operating room operation management system Figure 4 As shown in FIG, this figure is a module structure diagram of the operation management system shown in this embodiment of the present application, and the operation management system includes: A feature extraction module 100 is used to obtain image information of surgical instruments after surgery in the operating room, and extract instrument identification and instrument image surface features of each surgical instrument from the image information; A state determination module 200 is configured to measure the instrument alienation of each surgical instrument based on the surface features of the corresponding instrument image, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and determining the visual state code of each surgical instrument based on the corresponding instrument identifier and instrument alienation degree; A state correction module 300 is configured to use an augmented reality terminal device to spatially map the postoperative scene in the operating room, generate an augmented reality-based dynamic visualization image of the instrument based on the spatial mapping result and the visual status code of each surgical instrument, and determine the state correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument; The instrument statistics module 400 is used to generate postoperative instrument status statistics of the operating room according to the status correction results of each surgical instrument, and complete the inventory of surgical instruments.

[0049] The above describes in detail the examples of the operation and management methods and systems of the smart operating room provided by the embodiments of the present application. It can be understood that in order to realize the above functions, the corresponding devices include hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0050] In embodiment three, the present application also provides a computer device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned operation and management method of a smart operating room.

[0051] In this embodiment, reference Figure 5 , the dotted line in the figure indicates that the unit or module is optional. The figure is a structural diagram of a computer device according to an operation and management method of a smart operating room provided by an embodiment of the present application. The operation and management method of a smart operating room in the above embodiment can be Figure 5 The computer device shown in the figure is implemented, and the computer device includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device can be a terminal device, a server or a chip.

[0052] The processor 501 may be a general-purpose processor or a dedicated processor. For example, the processor 501 may be a central processing unit (CPU). The CPU may be used to control the computer device, execute software programs, and process data from the software programs. The computer device may also include a communication unit 505 to implement signal input (reception) and output (transmission).

[0053] For example, the computer device may be a chip, the communication unit 505 may be an input and / or output circuit of the chip, or the communication unit 505 may be a communication interface of the chip, and the chip may be a component of a terminal device, a network device, or other device.

[0054] For another example, the computer device may be a terminal device or a server, and the communication unit 505 may be a transceiver of the terminal device or the server, or the communication unit 505 may be a transceiver circuit of the terminal device or the server.

[0055] The computer device may include one or more memories 502, on which a program 504 is stored. Program 504 can be executed by processor 501 to generate instructions 503, causing processor 501 to execute the method described in the above method embodiment according to instructions 503. Optionally, memory 502 may also store data (such as a target audit model). Optionally, processor 501 may also read data stored in memory 502. This data may be stored at the same storage address as program 504, or at a different storage address.

[0056] The processor 501 and the memory 502 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of a terminal device.

[0057] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or software-based instructions in the processor 501. The processor 501 can be a central processing unit, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0058] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0059] In a fourth embodiment, the present application also provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are executed on a computer, the computer implements the above-mentioned method for operating and managing a smart operating room.

[0060] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0061] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present application fall within the scope of the claims and their equivalents, the present application is intended to include such modifications and variations.

Claims

1. A method for operating and managing a smart operating room, characterized in that: The operation management method comprises the following steps: Acquire image information of surgical instruments after surgery in the operating room, and extract instrument identification and instrument image surface features of each surgical instrument from the image information; The instrument alienation measurement is performed on each surgical instrument based on the corresponding instrument image surface features, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and the visual status code of each surgical instrument is determined by the corresponding instrument identification and instrument alienation degree; Use an augmented reality terminal device to spatially map the postoperative scene in the operating room, and then generate an augmented reality-based dynamic visualization image of the instrument based on the spatial mapping results and the visual status code of each surgical instrument. Determine the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument; According to the status correction results of each surgical instrument, the postoperative instrument status statistics of the operating room are generated to complete the inventory of surgical instruments.

2. The operation and management method of a smart operating room according to claim 1, characterized in that: Extracting the instrument identification and instrument image surface features of each surgical instrument from the image information specifically includes: Performing distortion correction on the image information to obtain corrected image data; Extracting the identification area of ​​the surgical instrument from the corrected image data, thereby obtaining the identification area of ​​each surgical instrument; Performing two-dimensional geometric feature extraction on the marked area of ​​each surgical instrument to obtain two-dimensional shape information of each surgical instrument; Determining an instrument identification of each surgical instrument through corresponding two-dimensional shape information; Surface features of surgical instruments are extracted from the corrected image data to obtain instrument image surface features of each surgical instrument.

3. The operation and management method of a smart operating room according to claim 1, characterized in that: The instrument alienation measurement of each surgical instrument is performed based on the corresponding instrument image surface features, and the instrument alienation degree of each surgical instrument in the operating room is obtained, which specifically includes: Obtaining reference image features corresponding to each surgical instrument in the historical database; For each surgical instrument in the operating room, determining a feature reference homogeneity of the surgical instrument based on the instrument image surface features and the corresponding reference image features; The instrument alienation degree of the surgical instrument is determined by the feature reference homogeneity, and then the instrument alienation degree of each surgical instrument in the operating room is obtained.

4. The operation and management method of a smart operating room according to claim 1, characterized in that: The visual status codes of each surgical instrument are determined by the corresponding instrument identification and instrument alienation degree, including: Classify the status of each surgical instrument according to the corresponding instrument identification and instrument alienation degree, and then obtain the status level of each surgical instrument; A visual status code for each surgical instrument is determined based on the corresponding status level.

5. The operation and management method of a smart operating room according to claim 1, characterized in that: The augmented reality-based dynamic visualization of instruments is generated based on the spatial mapping results and the visual status codes of each surgical instrument, specifically including: Based on the spatial mapping results, the visual status codes of each surgical instrument are three-dimensionally mapped to obtain an instrument space mapping relationship table; Constructing an augmented reality visualization layer through the instrument space mapping relationship table; The augmented reality visualization layer is dynamically superimposed on the real image of the operating room to obtain an augmented reality-based dynamic visualization image of the instrument.

6. The operation and management method of a smart operating room according to claim 1, characterized in that: Determining the status correction result of each surgical instrument based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument specifically includes: Collect interactive operation information during the equipment inventory process; Generate a state correction decision based on the interactive operation information and the visual state code of each surgical instrument in the dynamic visualization image of the instrument; The state correction result of each surgical instrument is determined according to the state correction decision and the instrument dynamic visualization image.

7. The operation and management method of a smart operating room according to claim 1, characterized in that: The postoperative instrument status statistics of the operating room are generated based on the status correction results of each surgical instrument, including: Obtaining the surgical instruments involved in the surgical process to generate a surgical instrument list, matching the state correction results of each surgical instrument with the surgical instrument list, and establishing an instrument state statistical model; All surgical instruments are clustered and statistically analyzed according to the instrument status statistical model to obtain postoperative instrument status information in the operating room.

8. An operation management system for a smart operating room, configured to execute an operation management method for a smart operating room according to any one of claims 1 to 7, characterized in that: The operation management system includes: A feature extraction module is used to obtain image information of surgical instruments after surgery in the operating room, and extract instrument identification and instrument image surface features of each surgical instrument from the image information; A status determination module is used to measure the instrument alienation of each surgical instrument based on the surface features of the corresponding instrument image, thereby obtaining the instrument alienation degree of each surgical instrument in the operating room, and determining the visual status code of each surgical instrument based on the corresponding instrument identification and instrument alienation degree; A state correction module is used to use an augmented reality terminal device to perform spatial mapping of the postoperative scene in the operating room, and then generate an augmented reality-based dynamic visualization image of the instrument based on the spatial mapping result and the visual status code of each surgical instrument. The state correction result of each surgical instrument is determined based on the interactive operation information during the instrument inventory process and the dynamic visualization image of the instrument; The instrument statistics module is used to generate postoperative instrument status statistics in the operating room based on the status correction results of each surgical instrument and complete the inventory of surgical instruments.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the operation and management method of a smart operating room described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements an operation and management method of a smart operating room as described in any one of claims 1 to 7.

Images