Medical equipment maintenance system and method based on augmented reality, terminal and medium

Abstract

The invention discloses a medical equipment maintenance system and method based on augmented reality, a terminal and a medium, and relates to the technical field of intelligent medical treatment, and the system comprises a mobile terminal, AR equipment and a cloud server. And the mobile terminal is used for carrying out security verification on the login information, managing a maintenance order, providing an operation guide and locally previewing and generating a maintenance report. And the AR equipment is AR glasses and is used for projecting each operation step into the visual field of the user in real time at a first visual angle and uploading all key data to the cloud server. And the cloud server is used for generating a maintenance report, dynamically evaluating the health state of the medical equipment, and generating early warning information to be pushed to the mobile terminal and the AR equipment when a performance degradation trend occurs or a potential fault risk exists. According to the invention, a set of closed-loop medical equipment maintenance system is constructed, and the standardization, authenticity, traceability and prospective management level of medical equipment maintenance are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart medical treatment, and in particular to a medical equipment maintenance system and method based on augmented reality, a terminal and a medium. BACKGROUND

[0002] With the rapid development of modern medical technology, various life support medical devices (such as multi-parameter monitors, defibrillators, ventilators, electrocardiogram machines, and anesthetic machines) play a crucial role in clinical diagnosis and treatment. To ensure the safe, stable, and reliable operation of these high-value and high-risk devices, regular preventive maintenance and maintenance (hereinafter referred to as "maintenance") have become a core link of hospital equipment management. Traditional paper or mobile APP recording methods have problems such as easy falsification, low efficiency, and difficulty in tracing.

[0003] Currently, hospitals generally use the following methods for maintenance and maintenance of life support medical devices: 1. Paper work order mode: maintenance engineers carry paper maintenance checklists and check each item on site and manually fill in the results, and finally take photos or scan and archive. This method is inefficient, prone to missed checks and incorrect filling, and paper records are easily lost or damaged, making it difficult to trace.

[0004] 2. Traditional mobile APP mode: maintenance engineers use a dedicated APP on their mobile phones or tablets to record maintenance. Although it is electronic, it still has obvious shortcomings: engineers need to frequently look down at paper manuals, making it difficult to operate in complex device environments, and both hands are occupied, affecting work efficiency. More importantly, the photos and inspection results recorded by the APP are separate, and there is a risk of post-recording and data falsification, which cannot guarantee the authenticity and on-site nature of the maintenance process. Moreover, maintenance data is scattered in paper documents or different engineers' devices, making it difficult to manage and analyze uniformly, and it is not conducive to forming a healthy profile for the entire life cycle of the device. Moreover, different engineers may have different understandings of the maintenance process, leading to non-standard maintenance operations and affecting maintenance quality.

[0005] 3. Simple remote prompts: current market products mostly focus on video remote technical support, lack deep integration with maintenance processes, do not form a closed loop of "guidance - execution - recording - uploading", and lack strong association authentication of the identity and operation time of the operator, with weak anti-fake capabilities. Moreover, existing technologies generally remain in a passive mode of "periodic maintenance" or "post-fault repair", and cannot achieve active, proactive, and intelligent device management.

[0006] Therefore, the existing technology still has defects. SUMMARY

[0007] The technical problems to be solved by the present application are to provide a medical equipment maintenance system and method based on augmented reality, a terminal and a medium to solve the above-mentioned defects of the prior art. In a first aspect, the present application provides a medical equipment maintenance system based on augmented reality, wherein the system comprises a mobile terminal, an AR device and a cloud server, and the mobile terminal, the AR device and the cloud server are all connected to each other by a communication link. The mobile terminal is used for security verification of login information, management of maintenance orders, provision of operation guides, and local preview and generation of maintenance reports. The AR device is an AR glasses, which is used to complete maintenance items in the maintenance order based on a standardized maintenance operation process, and each operation step in the maintenance operation process is projected in real time in the user's field of view at a first perspective, and all key data is uploaded to the cloud server after the completion of the maintenance operation process. The cloud server is used to generate a standardized maintenance report based on all key data, dynamically assess the health status of the medical equipment based on a preset predictive maintenance mechanism, and automatically generate warning information when identifying a performance degradation trend or potential failure risk of the medical equipment, and push the warning information to the mobile terminal and the AR device.

[0008] In an implementation manner, the AR glasses are provided with a high-resolution optical waveguide display module for realizing immersive and paperless visual maintenance operation process operation instructions. The AR glasses are provided with a capacitive touch sensor on the temple, which is used to execute each operation step in the maintenance operation process.

[0009] In an implementation manner, the cloud server comprises a database server layer, an application service layer and a file storage layer. The database server layer adopts a hybrid storage architecture combining a MySQL relational database and a Redis cache database, the MySQL relational database is used to persistently store core structured data, and the Redis cache database is used to cache high-frequency access static data. The application service layer is used to receive various requests sent by the mobile terminal and the AR device, and the application service layer integrates a report generation engine and a predictive maintenance mechanism. The file storage layer is used to store unstructured data generated by the system.

[0010] In a second aspect, the embodiments of the present application provide a medical equipment maintenance method based on augmented reality, wherein the method is applied to the medical equipment maintenance system based on augmented reality in any of the above solutions, and the method comprises the following steps: logging into a mobile terminal, selecting a maintenance order to be processed from a maintenance order management interface, and sending the maintenance order to be processed to an AR device; completing maintenance items in the maintenance order to be processed based on a standardized maintenance operation process, projecting each operation step in the maintenance operation process in a first perspective in the field of view of the user in real time, and uploading all key data to a cloud server after the completion of the maintenance operation process; generating a standardized maintenance report based on all key data, dynamically evaluating the health status of the medical equipment based on a preset predictive maintenance mechanism, and automatically generating early warning information when identifying that the medical equipment has a performance degradation trend or a potential failure risk, and pushing the early warning information to the mobile terminal and the AR device.

[0011] In an implementation manner, completing maintenance items in the maintenance order to be processed based on a standardized maintenance operation process comprises the following steps: obtaining operation instructions of the maintenance operation process; judging the eligibility of each maintenance item in the maintenance order to be processed based on the operation instructions, and recording key data and judgment results of each maintenance item; triggering a photographing function based on a capacitive touch sensor on the AR device, and photographing the detection part of the medical equipment for archiving; after photographing, entering the next maintenance item or the next maintenance order.

[0012] In an implementation manner, photographing the detection part of the medical equipment comprises the following steps: at the photographing moment, automatically embedding multiple anti-counterfeiting watermarks into the photographed photo, and the watermark content comprises the name of the maintenance personnel, the order number, the equipment number, and the timestamp.

[0013] In an implementation manner, dynamically evaluating the health status of the medical equipment based on a preset predictive maintenance mechanism comprises the following steps: obtaining multi-dimensional data, wherein the multi-dimensional data comprises maintenance records, equipment operation logs, usage environment data, failure statistics of similar equipment, and life information, and the maintenance records comprise key data in historical maintenance records and current maintenance records; using a machine learning algorithm to analyze the performance development trend and the potential failure risk of the medical equipment based on the multi-dimensional data, and obtaining the health status evaluation result of the medical equipment.

[0014] In an implementation manner, the performance development trend analysis and the potential failure risk analysis of the medical device on the multi-dimensional data are performed by using a machine learning algorithm, and a health status evaluation result of the medical device is obtained, including: key features reflecting performance changes of the medical device are extracted from the multi-dimensional data, and the key features are input into a trend analysis model constructed by using a time series analysis algorithm, and a performance development trend result of the medical device is obtained; the multi-dimensional data is converted into feature variables, and the feature variables are input into a classification model constructed based on a random forest algorithm, risk level information corresponding to the feature variables is output based on the classification model, and a potential failure analysis result of the medical device is obtained; the performance development trend result and the potential failure analysis result are taken as the health status evaluation result of the medical device.

[0015] In a third aspect, the embodiments of the present application further provide a terminal, wherein the terminal comprises a memory, a processor, and an augmented reality-based medical device maintenance program stored in the memory and executable on the processor, and when the processor executes the augmented reality-based medical device maintenance program, the steps of the augmented reality-based medical device maintenance method in any of the above solutions are implemented.

[0016] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium stores an augmented reality-based medical device maintenance program, and the augmented reality-based medical device maintenance program implements the steps of the augmented reality-based medical device maintenance method in any of the above solutions on the computer readable storage medium.

[0017] Beneficial effects: compared with the prior art, the application provides an augmented reality-based medical equipment maintenance system, which comprises a mobile terminal, an AR device and a cloud server, and the mobile terminal, the AR device and the cloud server are all connected with each other to establish a communication link. The mobile terminal is used for security verification of login information, management of maintenance orders, provision of operation guides and local preview and generation of maintenance reports. The AR device is an AR glasses, which is used for completing maintenance items in the maintenance order based on a standardized maintenance operation process, projecting each operation step in the maintenance operation process in a first perspective in the field of view of a user in real time, and uploading all key data to the cloud server after the completion of the maintenance operation process. The cloud server is used for generating a standardized maintenance report based on all key data, dynamically evaluating the health status of the medical equipment based on a preset predictive maintenance mechanism, and automatically generating early warning information when identifying that the medical equipment has a performance degradation trend or a potential failure risk, and pushing the early warning information to the mobile terminal and the AR device.

[0018] The application constructs a closed-loop medical equipment maintenance system through the deep integration of the visual guidance of the AR device, the automatic data acquisition and the cloud predictive analysis, effectively solves the problems of easy falsification of records, difficult traceability of processes and untimely response in the traditional maintenance mode, and significantly improves the standardization, authenticity, traceability and forward-looking management level of the medical equipment maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The application provides an architecture diagram of the augmented reality-based medical equipment maintenance system.

[0020] Figure 2 The application provides an architecture diagram of the cloud server in the augmented reality-based medical equipment maintenance system.

[0021] Figure 3 The application provides a flowchart of a preferred embodiment of the augmented reality-based medical equipment maintenance method.

[0022] Figure 4 The application provides an application flowchart of the AR device in the augmented reality-based medical equipment maintenance method.

[0023] Figure 5 The application provides a principle block diagram of the terminal. DETAILED DESCRIPTION

[0024] For the purposes of the present invention, the technical solutions and effects, further detailed below with reference to the drawings and examples. It should be understood that the specific examples described herein are intended to explain the present invention and are not intended to limit the present invention.

[0025] The flowcharts shown in the drawings are only illustrative and do not necessarily include all contents and operations or steps, nor are they necessarily executed in the order described. For example, some operations or steps can be broken down, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0026] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification and appended claims of the present invention, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly dictates otherwise.

[0027] It should be understood that, in order to facilitate the clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the terms "first", "second" and the like are used to distinguish the same or similar items with substantially the same function and effect. For example, the first control information and the second control information are only used to distinguish different control information, and do not limit the order.

[0028] Those skilled in the art can understand that the terms "first", "second" and the like do not limit the number and execution order, and the terms "first", "second" and the like do not necessarily mean different.

[0029] It should also be understood that the term "and / or" used in the specification and appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0030] In view of the characteristics of maintenance and repair of devices in the prior art, the prior art has at least the following disadvantages: 1. Low efficiency: maintenance engineers need to frequently refer to paper manuals, manually input device information and inspection results, which is tedious and time-consuming.

[0031] 2. Authenticity is difficult to guarantee: existing maintenance records mainly rely on paper work order checking or mobile phone / camera shooting, which has significant anti-fake loopholes. The photographed photos are easy to be replaced or tampered with, and cannot realize strong association with the identity of the maintenance personnel and the specific operation timestamp, providing opportunities for "substitute signing" or "false maintenance", resulting in serious lack of traceability in the maintenance process.

[0032] 3. Information silos: Maintenance data is scattered in paper documents or different engineers' devices, making it difficult to manage and analyze uniformly, and not conducive to forming a health record for the entire life cycle of the equipment.

[0033] 4. Low standardization: Different engineers may have different understandings of the maintenance process, leading to non-standard maintenance operations and affecting maintenance quality.

[0034] 5. Lack of real-time guidance: When encountering complex problems on site, timely and visual operation guidance cannot be obtained.

[0035] 6. Lack of predictive maintenance capability: Existing technologies generally remain in the passive mode of "periodic maintenance" or "repair after failure", and cannot perform health trend analysis based on equipment historical data, operating status and use environment, making it difficult to identify potential failure risks in advance. This "after-the-fact remediation" mode of operation not only increases the risk of sudden shutdown, but also leads to waste of maintenance resources, shortens the service life of equipment, and cannot achieve proactive and forward-looking intelligent equipment management.

[0036] The emergence of augmented reality (AR) devices provides a new way to solve these problems. By applying AR devices to maintenance scenarios, visual operation guidance in the first person perspective can be achieved, and maintenance personnel can complete inspections step by step without holding devices, significantly improving work efficiency and standardization. AR devices integrate cameras and identity authentication systems, automatically embedding anti-fake watermarks containing information such as personnel, equipment, and time when taking photos, making image records and operation behaviors strongly associated, fundamentally eliminating "fake signatures" or "false maintenance", and achieving full-process traceability and authenticity during maintenance, ensuring the safety of medical equipment.

[0037] Based on this, the embodiment provides an augmented reality-based medical equipment maintenance system, which includes a mobile terminal, an AR device, and a cloud server, combining Figure 1As shown in the figure, the mobile terminal, the AR device and the cloud server establish communication links with each other. The front-end AR device of the embodiment integrates functions such as camera, voice recognition and touch control; the back-end relies on the cloud server to construct a device database, a maintenance task scheduling center and a data analysis engine; and the bidirectional communication between the terminal and the cloud server is realized through Wi-Fi or mobile network. Specifically, Wi-Fi or Bluetooth technology can be used to establish a communication connection between the mobile terminal and the AR device, Wi-Fi or mobile network can be used to establish an encrypted transmission communication link between the mobile terminal and the cloud server, and Wi-Fi or mobile network can also be used to establish an encrypted transmission communication link between the AR device and the cloud server. Of course, in other implementation manners, short-distance communication technologies such as Bluetooth or UWB (Ultra Wideband, a kind of carrierless communication technology) can also be used to realize data transmission between the glasses and the mobile phone.

[0038] The mobile terminal of the embodiment serves as the front-end entrance for user identity authentication and configuration management, and runs on the Android and IOS operating systems. The core functions are to perform security verification on login information, manage maintenance orders, provide operation guides, and preview and generate local maintenance reports, to ensure that the user can conveniently obtain operation support, and the identity information and operation time stamp of the maintenance personnel are encrypted and uploaded to the AR device.

[0039] Specifically, the mobile terminal of the embodiment can be a portable smart terminal such as a mobile phone, and the APP of the mobile terminal provides an intuitive maintenance order management interface, and the card layout is used to clearly display the maintenance orders to be processed, in progress and completed. The user can filter by states such as "to be processed", "in progress" and "completed" through a pull-down menu, and support keyword search by order number, device name or maintenance type to quickly locate the maintenance order corresponding to the target medical device. In specific application, the card of each maintenance order displays key information, including: device name, model, maintenance item, plan content, order date and current execution state, and the execution state can be identified by different colors, such as red for to be processed, green for in progress and gray for completed, to help the user efficiently manage multiple task scenarios. The user can click the card of the maintenance order to enter the detail page to view the complete maintenance plan content, related person in charge and historical execution record.

[0040] The APP of the mobile terminal also helps to provide an electronic version of the operation manual for maintenance, so as to assist the maintenance personnel to accurately perform the maintenance process of the complex equipment. Further, the APP of the mobile terminal is built-in with a maintenance manual function module, which can display the standardized operation manual and professional maintenance manual of various medical equipment by remotely calling the resource library of the cloud server and by real-time pulling. The maintenance personnel can quickly locate the corresponding maintenance manual through classification navigation or scanning the two-dimensional code on the medical equipment. All the maintenance manuals are presented in the format of pictures and texts, and some of the contents support embedded video demonstration, so as to provide an immersive learning experience. The function can not only be used for on-site operation reference, but also be used as a digital teaching material for new employee training, so as to effectively improve the overall professional level of the maintenance team.

[0041] Further, the maintenance items in the maintenance order can be pre-prepared by experienced engineers in the background system, and the detailed maintenance process and maintenance items are structured and uploaded. In an implementation manner, the APP of the mobile terminal supports backtracking and management of the completed maintenance order. In a specific application, based on the uploaded structured data and image data, a standardized electronic version of the maintenance report with uniform format and complete content can be automatically generated. The user can preview, export, share or directly print the maintenance report in the APP, so as to facilitate the submission to the relevant person in charge for signature confirmation or archiving. In addition, the maintenance report generation process supports custom templates and digital signatures, so as to ensure its legal validity and audit compliance. Through the mechanism, the whole process of the maintenance process is left with traces, real and traceable, and the responsibility is to the person, so that the risks such as "substitute signature" and "supplementary record" are completely eliminated, and the standardization, transparency and efficiency level of the hospital equipment management are significantly improved.

[0042] The AR device of the embodiment can be a portable AR wearable device such as AR glasses. When applied to different AR devices, only the corresponding software development kit and gesture recognition logic need to be adjusted. The embodiment is described with AR glasses as an example. When the AR glasses are started, the mobile terminal can automatically project the maintenance order management interface into the AR glasses to present the interface in a first view angle, and the specific maintenance project can be completed by operating the AR glasses. Each maintenance project needs to follow a standardized maintenance operation process to be implemented. Specifically, the AR glasses are internally provided with a high-resolution optical waveguide display module for realizing an immersive and paperless visual maintenance operation process. A capacitive touch sensor is arranged on the temple of the AR glasses, and the capacitive touch sensor is used to execute each operation step in the maintenance operation process. For example, in the maintenance order management interface, the maintenance personnel selects a maintenance order to be processed by sliding the capacitive touch sensor on the right temple, and clicks to enter the maintenance project of the maintenance order. The system displays the specific maintenance content item by item according to the maintenance plan preset in the background database. In addition, in other implementation manners, voice control, electromyography control, head posture recognition or eye movement tracking can also be added as an interactive mode.

[0043] In actual application, the standardized maintenance operation process in the embodiment specifically includes the following steps: Step 1, operation instruction of the maintenance operation process. Specifically, the glasses screen of the AR glasses displays the specific requirements of the current maintenance project (such as "check whether the monitor power cord is damaged"). This step aims to provide clear operation instructions for the user to ensure that each step is executed according to the specification. After the maintenance personnel completes the inspection of the maintenance project, the "next step" button on the display interface can be used to continue the operation. The right temple is clicked to enter the judgment link, or the "previous step" is selected to return to the order list to check other projects.

[0044] Step 2, qualified judgment is made on each maintenance project in the maintenance order to be processed based on the operation instruction, and key data and judgment results of each maintenance project are recorded. In actual application, the maintenance personnel switches between "normal" and "abnormal" options by sliding the capacitive touch sensor on the right temple. After clicking to confirm the selection, the system records the judgment results and some key data in real time to ensure the immediacy and accuracy of the data.

[0045] Step 3, based on the capacitive touch sensor on the AR device, trigger the photo function, take a photo of the part to be detected of the medical device for archiving. Specifically, the user will be prompted on the glasses screen to take a photo of the scene for archiving, such as "Please take a photo for archiving". At this time, the maintenance personnel will aim their eyes at the part to be checked of the medical device and select the "take a photo" button on the interface. Then click the capacitive touch sensor on the right temple to trigger the photo function, and the built-in camera in the AR glasses automatically captures the image. In addition, at the moment of taking a photo, the system automatically embeds multiple anti-fake watermarks in the photo. The anti-fake watermark content includes the name of the maintenance personnel, the order number, the device number and the accurate timestamp, ensuring the strong correlation and non-tamperability of the image data and the actual operation behavior. This embodiment allows 1 to 3 photos to be taken for the same maintenance project, and the interface below prompts "photo success" and "please click the temple to take a photo for archiving X / 3" and other information, facilitating the user to confirm the photo progress.

[0046] Step 4, after taking a photo, enter the next maintenance project or enter the next maintenance order.

[0047] Thus, the maintenance personnel selects the maintenance order to be processed in the AR glasses, and the system pushes the maintenance tasks one by one according to the preset standardized maintenance operation process; each task includes four steps of "operation prompt of maintenance operation process → eligibility judgment → photo archiving → entering next step" closed loop operation. And support code scanning to identify device labels, automatically fill in device information, reduce manual input errors, automatically add digital watermarks containing name, order number, device number and timestamp when taking photos, to ensure the authenticity and credibility of the images.

[0048] When all maintenance projects are completed, the maintenance personnel clicks the right temple, and the system automatically enters the uploading process to upload all key data to the cloud server, including the following: (1) Data aggregation and encrypted transmission: AR glasses securely transmit all key data (including device information, results of eligibility judgment of each maintenance project, all photos with watermarks, and operation timestamp, etc.) in this maintenance order to the cloud server through Wi-Fi or mobile network. High-strength encryption algorithm is used during data transmission to ensure the integrity and security of data during transmission.

[0049] (2) Cloud processing and report generation: after successful uploading, the cloud server analyzes and structures the received key data. Based on the uploaded key data, the server automatically generates a complete electronic version of the maintenance report (in PDF format), which includes detailed maintenance content, results of eligibility judgment of each maintenance project, photos with watermarks and related responsible person information. The generated maintenance report supports multi-channel push, which can be directly sent to the relevant responsible person (such as department head, device administrator, etc.), or archived for subsequent audit use.

[0050] (3) Full traceability and management optimization: The entire maintenance process achieves full traceability, from task allocation, operation execution to result uploading, with detailed records at each link to ensure data authenticity and traceability. Through this mechanism, the hospital equipment management department can real-time grasp the equipment maintenance status, timely find potential problems, optimize the maintenance process, and improve the overall management efficiency and quality.

[0051] The cloud server of the present embodiment is the central nervous system and data cornerstone of the system, deployed on a cloud infrastructure with high availability and elastic scalability, using a microservice architecture for modular design to ensure high concurrency processing capability, business decoupling and long-term stable operation of the system. The server side is composed of three core service layers, which work together to complete data management, business processing and resource scheduling, and support predictive maintenance functions, realizing the transition from passive maintenance to active prevention.

[0052] Specifically, in combination with the description shown in Figure 2 The database server layer adopts a hybrid storage architecture combining MySQL relational database and Redis cache database. The MySQL relational database is used for persistent storage of core structured data, including but not limited to: user account information (such as account, password, name, employee number, affiliated hospital, etc.), medical equipment archives (such as equipment number, equipment serial number, model, manufacturer, service life, risk level, affiliated department, etc.), standardized maintenance plan templates (such as inspection item name, inspection content, execution standard), and the full life cycle status of maintenance orders (such as order number, equipment type, plan content, execution personnel, inspection result, exception summary, completion time, etc.). The Redis cache database is used to cache high-frequency access static data such as maintenance manuals and equipment information, significantly reducing database query pressure and improving system response speed.

[0053] The application service layer, as the business logic processing center of the system, is built based on an enterprise-level framework. It is responsible for receiving various requests from mobile terminals and AR glasses, performing user identity authentication, permission verification, Bluetooth connection state synchronization, maintenance task distribution, inspection result reception and verification, watermark data association, order status update, and other core business logic. The application service layer integrates a report generation engine that can automatically aggregate order information, inspection results, and watermark photos after the completion of maintenance tasks, dynamically generate PDF electronic version maintenance reports with standardized format and complete content, and support version management and digital signature of the reports to ensure their legal effectiveness and audit compliance.

[0054] In another implementation, the application service layer also integrates a predictive maintenance mechanism, which dynamically evaluates the health status of the medical device by analyzing multi-dimensional data, including historical maintenance records (such as test results, abnormal items, and maintenance frequency), device operation logs (such as boot-up time, alarm frequency, and key parameter fluctuations), usage environment data (such as temperature and humidity, usage department, and operator feedback), and failure statistics and life information of similar devices. Then, through machine learning algorithms (such as time series analysis and random forest classification model), the health status of the medical device is dynamically evaluated, and the health status evaluation result of the medical device is obtained. When it is detected that a device has a performance degradation trend or a rising failure risk (such as a periodic increase in the flicker frequency of a certain type of monitor screen), the system automatically generates a risk warning and pushes it to the maintenance engineer through the interface of the mobile terminal APP or AR glasses, prompting him to arrange special inspections or component replacement in advance, thereby realizing the transition from passive maintenance to active prevention, significantly reducing the rate of sudden failures, and improving device availability.

[0055] The file storage service layer is specifically used to store the massive unstructured data generated by the system. Considering that a large number of high-definition images (such as on-site photos with anti-counterfeit watermarks) and electronic versions of operation manuals, training videos, and other documents will be generated during the maintenance process, the system deeply integrates ObjectStorage Service (OSS) as the core object storage middleware. OSS provides high-throughput, low-latency, and data-persistent storage services, supports efficient uploading, safe archiving, and fast retrieval of massive files. All image data captured by AR glasses and uploaded is stored in encrypted form in OSS after being processed by the application service layer, and metadata associated with the corresponding maintenance order is established to ensure data integrity, traceability, and long-term preservation capability.

[0056] In addition, the system of the embodiment can also integrate remote expert collaboration functions to support real-time video calls of AR glasses and remote guidance of on-site maintenance by background experts. It can be connected to hospital or device asset management systems to realize automatic distribution and status synchronization of maintenance tasks, and can also add energy consumption monitoring and environmental sensor integration modules to expand the maintenance dimension.

[0057] Through the above multi-level and highly reliable technical architecture, the cloud server not only realizes centralized and intelligent management of maintenance process data, but also provides solid technical support and data protection for the compliance and standardized operation and maintenance of medical devices. In particular, the introduction of the predictive maintenance function further improves the forward-looking and intelligent level of the system, helping medical institutions to achieve more efficient device management and lower operation and management costs.

[0058] Based on the above embodiments, the application provides a medical equipment maintenance method based on augmented reality, which is applied to the medical equipment maintenance system based on augmented reality in the above embodiments. This embodiment is also described by taking AR equipment as AR glasses as an example. As shown in Figure 3 The medical equipment maintenance method based on augmented reality in this embodiment at least includes the following steps: Step S100, log in the mobile terminal, select a maintenance order to be processed from the maintenance order management interface, and send the maintenance order to be processed to the AR equipment.

[0059] In specific application, the APP of the mobile terminal provides an intuitive maintenance order management interface to clearly display the maintenance orders to be processed, in process and completed in a card layout. The user can filter by the "to be processed", "in process" and "completed" state buttons through a drop-down menu, and support keyword search by order number, equipment name or maintenance type to quickly locate the target maintenance order. The card of each maintenance order displays key information, including the equipment name, model, maintenance item, plan content, order date and current execution state, and can also be identified by different colors, such as red for maintenance orders to be processed, green for maintenance orders in process, and gray for completed maintenance orders, to help the user efficiently manage the multi-task scenario. Clicking the card of the maintenance order can enter the detail page to view the complete maintenance plan content, related person in charge and historical execution record. When the maintenance order to be processed is selected, the maintenance order to be processed can be sent to the AR equipment.

[0060] Step S200, complete the maintenance items in the maintenance order to be processed based on the standardized maintenance operation process, project each operation step in the maintenance operation process in the user's field of view in the first perspective in real time, and upload all key data to the cloud server after the completion of the maintenance operation process.

[0061] When performing the maintenance task, the AR glasses project each operation step in the user's field of view in the first perspective in real time through its high-resolution optical waveguide display module, realizing immersive and paperless visual guidance. Specifically, the application provides a standardized maintenance operation process, as shown in Figure 4 When the maintenance order to be processed is selected, the operation guide of the standardized maintenance operation process can be obtained, that is, the step corresponding to the standardized maintenance operation process prompt in Figure 4 Then, the eligibility of each maintenance item in the maintenance order to be processed is judged based on the operation guide, and the key data and determination results of each maintenance item are recorded, that is, the steps corresponding to the eligibility judgment of each maintenance item in the maintenance order to be processed in Figure 4the step of judging whether it is normal in the step S200. Then, based on the capacitive touch sensor on the AR glasses triggering the photographing function, the part to be detected of the medical equipment is photographed for archiving, that is, the corresponding Figure 4 the step of photographing for archiving in the step S200. After the photographing is completed, the step of Figure 4 the step of judging whether the maintenance item in the maintenance order is completed in the step S200, if not, the next maintenance item is entered, and the step of repeating the execution of Figure 4 the step of prompting the standardized maintenance operation process in the step S200. If the current maintenance order has been completed, the next maintenance order can be selected to enter. And the step of further uploading to the cloud server in the step S200 can be further executed. Figure 4 the step of uploading to the cloud server in the step S200.

[0062] In an implementation manner, the embodiment automatically embeds multiple anti-fake watermarks in the photographed photo at the photographing moment, and the watermark content includes the name of the maintenance personnel, the order number, the equipment number and the timestamp, so that the strong correlation and non-tamperability of the image data and the actual operation behavior can be ensured. When uploaded to the cloud server, the AR glasses securely transmit all the key data of this maintenance (including the equipment information, the judgment result of the eligibility of each maintenance item, all the photos with watermarks and the operation time) to the cloud server through Wi-Fi or mobile network. High-strength encryption algorithm is adopted in the data transmission process to ensure the integrity and security of the data in the transmission process.

[0063] The step S300, based on all the key data, generates a standardized maintenance report, based on a preset predictive maintenance mechanism, dynamically evaluates the health status of the medical equipment, and when identifying that the medical equipment has a performance degradation trend or has a potential failure risk, automatically generates a warning information, and pushes the warning information to the mobile terminal and the AR equipment.

[0064] Specifically, the cloud server analyzes and structures the received key data, and based on the uploaded key data, the cloud server automatically generates a complete electronic version of the maintenance report, which includes detailed maintenance content, inspection results, photos with watermarks and related responsible person information. The generated maintenance report supports multi-channel pushing, which can be directly sent to the related responsible person (such as the department head, the equipment administrator, etc.), and can be archived for subsequent audit use. The whole maintenance process realizes full-process traceability, and each link has detailed records, which ensures the authenticity and traceability of the data. Through the mechanism, the hospital equipment management department can master the equipment maintenance state in real time, find potential problems in time, optimize the maintenance process, and improve the overall management efficiency and quality.

[0065] In addition, the cloud server of the embodiment can also dynamically evaluate the health status of the medical device based on a preset predictive maintenance mechanism. In actual application, the embodiment can obtain multi-dimensional data, including maintenance records, device operation logs, usage environment data, similar device failure statistics and life information, wherein the maintenance records include key data in historical maintenance records and current maintenance records. Then, the multi-dimensional data is analyzed by a machine learning algorithm to obtain the health status evaluation result of the medical device.

[0066] In one implementation, when performing performance trend analysis, the embodiment can extract key features reflecting changes in the performance of the medical device from the multi-dimensional data. For example, maintenance frequency, maintenance interval, and key component replacement period are extracted from maintenance records, which directly reflect the changing trend of device failure risks. Parameter fluctuation amplitude, parameter drift trend, and abnormal parameter occurrence frequency are extracted from device operation logs, which reflect the stability of the real-time running state of the device. Environmental factor change trend and extreme environment duration are extracted from usage environment data to analyze the long-term impact of the environment on the performance of the device. Average failure time of similar devices and life distribution pace of devices of the same batch are extracted from similar device data as a reference benchmark for the current device health status. Then, the extracted key features are input into a trend analysis model constructed using time series analysis algorithm. The trend analysis model of the embodiment is essentially a model of device performance change over time, and the core is to predict trends and identify abnormalities. Therefore, the trend analysis model can automatically analyze the performance development trend of the medical device according to the input key features, thereby obtaining the performance development trend result.

[0067] In addition, the embodiment can also perform hierarchical evaluation on the performance development trend result. For example, the performance development trend result is divided into: Health level: indicating that the performance trend of the medical device is stable, which means that the medical device is predicted to have no abnormal risk within 6 months, the key parameter fluctuation is within the normal range, and the maintenance interval is longer than the average level of similar devices.

[0068] Warning level: indicating that the performance trend of the medical device starts to slowly decline (such as the parameter drift amplitude approaching the threshold), which means that the medical device is predicted to have a slight abnormality in 3-6 months and needs to be monitored more closely.

[0069] Failure risk level: indicating that the performance trend of the medical device rapidly declines (such as a sudden increase in the frequency of abnormal parameters), which means that the medical device is predicted to have a high probability of failure within 1-3 months and needs to be immediately arranged for maintenance.

[0070] In another implementation, in the potential failure risk analysis, the embodiment can convert the multidimensional data into characteristic variables. For example, based on the maintenance records, quantitative indicators related to failures are extracted, including but not limited to the number of maintenance in the past 3 months, the number of days from the last maintenance to today, the cumulative number of replacements of key components (such as circuit boards and sensors), and the average value of the duration of a single maintenance. Based on the operation log, the average value of the core parameters (such as temperature, pressure, and speed) in the past 7 days, the maximum value of the number of daily abnormal alarms, and the cumulative duration of parameters exceeding the normal range are extracted. Based on the data characteristics of the use environment, the average value of the average humidity and temperature in the past 30 days, the number of days exceeding the temperature and humidity range that the device can tolerate, and the like are extracted. Based on the data of similar devices, the average number of failures of the same type of device under the same running time, the average life of the same batch of devices, and the like are extracted. Based on this, the characteristic variables can be extracted. Then, the characteristic variables are input into a classification model constructed based on a random forest algorithm. The classification model of the embodiment is constructed based on a random forest through ensemble learning of multiple decision trees. When training the classification model, the historical data containing the device data with known health status labels can be divided into a training set and a test set in a ratio of 7:3. For example, the multi-dimensional characteristics and corresponding failure records of 1000 devices of the same type in the past 5 years are collected to construct the training set and the test set. Then, the classification model is trained based on a random forest through ensemble learning of multiple decision trees. The trained classification model can be used to capture the nonlinear relationship between the characteristic variables and the health status. Therefore, based on the classification model, the risk level information corresponding to the characteristic variables can be output, and the potential failure analysis result of the medical device can be obtained.

[0071] Finally, the performance development trend result and the potential failure analysis result are taken as the health status evaluation result of the medical device. Based on the health status evaluation result, if the performance degradation trend or the potential failure risk of the medical device is identified, a warning information is automatically generated, and the warning information is pushed to the mobile terminal and the AR device, so as to suggest the relevant person in charge to intervene in advance. The machine learning algorithm used in the embodiment is only used for example. In specific application, a more detailed device health evaluation model can be constructed by combining the original operation data provided by the device manufacturer, so as to realize more accurate monitoring and evaluation of the medical device.

[0072] The augmented reality-based medical device maintenance method and system of the embodiment can be extended from high-value medical devices such as monitors, ventilators, and anesthetizing machines to medium and low-value conventional medical devices such as electrocardiogram machines, infusion pumps, and electric nursing beds, and is suitable for primary medical scenes such as community health service centers, and can also be extended to cross-industry maintenance scenes such as numerical control machine tools in the industrial manufacturing field and traffic equipment in the public service field, and supports the collaborative management of hospitals, third-party service institutions, regional medical alliances, and cross-industry maintenance subjects, and has good replicability and promotion potential.

[0073] Compared with the prior art, the present application has at least the following technical effects: 1. Ensure the authenticity of maintenance: through the first perspective of the AR device to collect the scene photos, and automatically embed multiple information watermarks containing personnel, equipment, order, time, so that the photo is strongly bound with the maintenance operation, fundamentally eliminates the false behaviors such as after-the-fact retake, impersonation, etc., and guarantees the non-tamperability and traceability of the maintenance record.

[0074] 2. Improve operation efficiency and convenience: adopt the "what you see is what you get" AR visualization guidance, the maintenance personnel do not need to look down to check the mobile phone or handheld tablet, and the hands are free, which can focus on equipment inspection. All interactions can be completed through simple glasses gestures (sliding, single clicking) or ring, and the operation is intuitive and smooth, which significantly improves the on-site operation efficiency.

[0075] 3. Realize process standardization: the system forces to execute according to the preset maintenance operation process, and each step has clear prompts and confirmations, which effectively avoids missed inspection and wrong inspection, and ensures the standardization and consistency of the maintenance process. All data are stored in an electronic and structured manner, which is easy to query, statistics and audit. Electronic report is automatically generated, which reduces the management and storage cost of paper documents.

[0076] 4. Introduce predictive maintenance mechanism, break through the limitation of traditional periodic maintenance or post-fault maintenance. The cloud server continuously collects historical maintenance records, equipment operation logs, usage environment data and similar equipment fault statistics, and uses a machine learning model to dynamically evaluate the health status of the equipment. When the system identifies the performance degradation trend or potential failure risk, it will automatically generate a warning message and actively push it to the maintenance personnel through the mobile terminal APP or AR glasses interface, prompting them to arrange special inspection or component replacement in advance. This mechanism realizes the change from passive response to active prevention, significantly reduces the sudden failure rate, prolongs the service life of the equipment, and optimizes the allocation of operation and maintenance resources.

[0077] 5. The present application supports the whole-process digital management of maintenance tasks, including task distribution, on-site execution, data archiving, report generation and quality analysis. Combined with the predictive maintenance capability, it provides scientific decision support for hospital equipment management departments, helps to build a standardized, verifiable and forward-looking intelligent operation and maintenance system, and comprehensively improves the standardization, transparency and intelligent level of medical equipment management.

[0078] The augmented reality-based medical equipment maintenance method of the embodiment is the same as the principles of each end and each module in the above-mentioned system embodiment, which will not be repeated here.

[0079] Based on the above embodiment, the present application also provides a terminal, and the principle block diagram of the terminal can be as Figure 5The terminal can include one or more processors 100 (only one is shown), Figure 5 The one or more processors 100 implement various steps of an embodiment of a method for augmented reality based maintenance of medical devices when executing the computer program 102. Alternatively, the one or more processors 100 implement the functionality of various modules / units in an embodiment of a system for augmented reality based maintenance of medical devices when executing the computer program 102, which is not limited herein.

[0080] In one embodiment, the processor 100 can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor, or the processor can be any conventional processor.

[0081] In one embodiment, the memory 101 can be an internal memory unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory 101 can also be an external memory device of the electronic device, such as a plug-in hard disk, a smart media (SMC) card, a secure digital (SD) card, a flash card, or the like. Further, the memory 101 can include both an internal memory unit and an external memory device of the electronic device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0082] Those skilled in the art can understand that Figure 5 The block diagram shown in the figure is only a block diagram of part of the structure related to the present application, and does not constitute a limitation on the terminal to which the present application is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0083] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0084] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A medical equipment maintenance system based on augmented reality, characterized in that, The system includes: a mobile terminal, an AR device, and a cloud server, wherein the mobile terminal, the AR device, and the cloud server establish communication links with each other; The mobile terminal is used for security verification of login information, management of maintenance orders, provision of operation guides, and local preview and generation of maintenance reports. The AR device is AR glasses, which are used to complete the maintenance items in the maintenance order based on a standardized maintenance operation process. In the maintenance operation process, each operation step is projected into the user's field of vision in real time from a first-person perspective, and after the maintenance operation process is completed, all key data is uploaded to the cloud server. The cloud server is used to generate standardized maintenance reports based on all key data, dynamically assess the health status of medical equipment based on a preset predictive maintenance mechanism, and automatically generate early warning information when the medical equipment is found to have a performance degradation trend or potential failure risk, and push the early warning information to the mobile terminal and the AR device.

2. The augmented reality-based medical equipment maintenance system according to claim 1, characterized in that, The AR glasses have a built-in high-resolution optical waveguide display module, which is used to realize immersive, paperless, and visualized operation guidance for maintenance procedures. The AR glasses are equipped with capacitive touch sensors on the temples, which are used to execute each step of the maintenance process.

3. The augmented reality-based medical equipment maintenance system according to claim 1, characterized in that, The cloud server includes: a database server layer, an application service layer, and a file storage layer; The database server layer adopts a hybrid storage architecture that combines a MySQL relational database and a Redis cache database. The MySQL relational database is used to persistently store core structured data, and the Redis cache database is used to cache frequently accessed static data. The application service layer is used to receive various requests sent by the mobile terminal and the AR device. The application service layer integrates a report generation engine and a predictive maintenance mechanism. The file storage layer is used to store unstructured data generated by the system.

4. A method for maintaining medical equipment based on augmented reality, characterized in that, The method is applied to the augmented reality-based medical device maintenance system according to any one of claims 1-3, and the method includes: Log in to the mobile terminal, select the maintenance order to be processed from the maintenance order management interface, and submit the maintenance order to be processed to the AR device; Based on standardized maintenance operation procedures, the maintenance items in the pending maintenance orders are completed. In the maintenance operation process, each operation step is projected to the user's field of vision in real time from a first-person perspective. After the maintenance operation process is completed, all key data is uploaded to the cloud server. Based on all key data, a standardized maintenance report is generated. Based on a preset predictive maintenance mechanism, the health status of medical equipment is dynamically assessed. When the medical equipment is found to have a performance degradation trend or potential failure risk, an early warning message is automatically generated and pushed to the mobile terminal and the AR device.

5. The augmented reality-based medical device maintenance method according to claim 4, characterized in that, The maintenance items in the pending maintenance orders are completed based on standardized maintenance operation procedures, including: Obtain the operation instructions for the aforementioned maintenance procedure; Based on the aforementioned operation guidelines, the qualification of each maintenance item in the maintenance order to be processed is judged, and the key data and judgment results of each maintenance item are recorded; The camera is triggered by the capacitive touch sensor on the AR device to take a picture of the part of the medical device to be tested for archiving. After the photo is taken, proceed to the next maintenance item or the next maintenance order.

6. The augmented reality-based medical device maintenance method according to claim 4, characterized in that, Taking photographs of the area to be tested from the medical device, including: At the moment of taking a photo, multiple anti-counterfeiting watermarks are automatically embedded into the photo. The watermark content includes the maintenance personnel's name, order number, equipment number, and timestamp.

7. The augmented reality-based medical device maintenance method according to claim 4, characterized in that, Based on a pre-defined predictive maintenance mechanism, the health status of medical equipment is dynamically assessed, including: Acquire multi-dimensional data, including: maintenance records, equipment operation logs, usage environment data, and failure statistics and lifespan information of similar equipment. The maintenance records include key data from historical maintenance records and current maintenance records. Machine learning algorithms are used to analyze the performance development trend and potential failure risk of the medical device based on the multi-dimensional data, thereby obtaining the health status assessment results of the medical device.

8. The augmented reality-based medical device maintenance method according to claim 7, characterized in that, Machine learning algorithms are used to analyze the performance development trend and potential failure risk of the medical device based on the multi-dimensional data, resulting in a health status assessment of the medical device, including: Key features reflecting changes in the performance of the medical device are extracted from the multi-dimensional data, and these key features are input into a trend analysis model constructed using a time series analysis algorithm to obtain the performance development trend results of the medical device. The multi-dimensional data is transformed into feature variables, and the feature variables are input into a classification model built based on the random forest algorithm. The risk level information corresponding to the feature variables is output based on the classification model to obtain the potential failure analysis results of the medical device. The performance development trend results and the potential failure analysis results are used as the health status assessment results of the medical device.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and an augmented reality-based medical device maintenance program stored in the memory and executable on the processor. When the processor executes the augmented reality-based medical device maintenance program, it implements the steps of the augmented reality-based medical device maintenance method as described in any one of claims 4-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an augmented reality-based medical device maintenance program, which implements the steps of the augmented reality-based medical device maintenance method as described in any one of claims 4-8 on the computer-readable storage medium.

Images