Emergency remote visual guidance and training system

Abstract

The application relates to the technical field of AR remote medical assistance, and discloses a first-aid remote visual guidance and training system, which comprises an eye movement tracking unit; the eye movement tracking unit adopts infrared light sources and high-precision image sensors to track and record the eye movement of an operator in real time, and calculates the current visual line range and visual line position of the operator when the operator is staring at a superimposed image; when new virtual information is added, it is judged whether the new virtual information is located in the visual line range; if the new virtual information is located outside the visual line range, a prompt mark is displayed in the visual line range of the operator, which is used for prompting the operator to move the visual line until the new virtual information enters the visual line range of the operator; the problem that the prior art cannot meet the efficiency and accuracy requirements of AR remote guidance is solved.

Description

Technical Field

[0001] This invention relates to the field of remote medical AR-assisted technology, specifically to an emergency remote visual guidance and training system. Background Technology

[0002] In the modern medical field, telemedicine technology is rapidly developing, providing patients with more convenient and efficient medical services. Traditional telemedicine relies primarily on low-bandwidth networks for data transmission, which limits the real-time performance and clarity of medical images and videos, making it difficult for doctors to accurately diagnose conditions and provide timely guidance. With the advent of 5G technology, high-speed, low-latency, and high-bandwidth data transmission capabilities offer new possibilities for telemedicine. 5G technology not only supports the real-time transmission of high-definition medical images and videos but also enables the application of advanced technologies such as augmented reality (AR), thereby significantly improving the efficiency and accuracy of telemedicine.

[0003] For example, the Chinese patent application CN 117174249 A, "A Remote Surgery AR Assistance Method Based on 5G Communication," describes a method in which the surgeon wears AR glasses, launches an augmented reality application, and establishes a communication connection with the assisting client under a 5G network. This invention enables the assisting expert to see the surgical field in real time and communicate with the surgeon on-site via voice. Simultaneously, the expert can make annotations on the surgical field and transmit the two-dimensional normalized annotation data in the screen space and the pose data of the AR glasses at the time of annotation back to the surgeon. Based on the transmitted data, the surgeon obtains three-dimensional AR annotation data through ray detection in the augmented reality coordinate system. The AR annotation content is superimposed on the real organ along the surface of the human organ on the AR glasses.

[0004] However, since the broadcast view seen by the assistant is the operator's first-person perspective, this solution is only suitable for localized surgical scenarios where the operator's position is relatively fixed and the surgical field is small. In emergency and training scenarios, non-professionals often act as the first-person perspective, facing a much larger amount of environmental information and needing to constantly move to gather more real-world information. In these situations, the real environment is usually much larger than the operator's field of vision. Because of their lack of professional knowledge and experience, non-professional operators in the first-person perspective often cannot move quickly enough to find effective footage to display, and remote assistants also cannot see much crucial information from the broadcast. This significantly reduces the effectiveness of the guidance and fails to meet the urgent need for efficient and accurate remote guidance in emergency situations such as first aid. Summary of the Invention

[0005] The present invention aims to provide an emergency remote visual guidance and training system to address the shortcomings of existing technologies in meeting the requirements for efficiency and accuracy of AR remote guidance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an emergency remote visual guidance and training system, comprising an AR device module for providing operators with an overlay image display of real scenes and virtual information, and a remote terminal module for collecting virtual information and providing assistants with an overlay view of the operation site;

[0007] The AR device module includes an image acquisition and processing unit and an operation display unit. The image acquisition and processing unit uses multiple cameras located in different directions to acquire real-world environmental images in a 180° field of view in front of the operator, and generates seamless 180° field-of-view images through image stitching technology. The real-world environmental images are then fused with virtual information to form an overlay view. The operation display unit uses a curved screen to display the overlay view.

[0008] The remote terminal module includes a terminal display unit for displaying an overlay view to assisting personnel, an interactive control unit for collecting virtual information and transmitting it to the image acquisition and processing unit of the AR device module for fusion display, and a communication unit for two-way voice communication and data synchronization and transmission between the AR device module and the remote terminal module.

[0009] The AR device module also includes an eye-tracking unit, which uses an infrared light source and a high-precision image sensor to track and record the operator's eye movements in real time, and calculates the operator's current gaze range and gaze position when looking at the operation display unit.

[0010] The eye-tracking unit is used to detect whether the location of the new virtual information is within the operator's current line of sight when the interactive control unit acquires new virtual information. If it is determined that the new virtual information is outside the line of sight, a prompting instruction is sent to the image acquisition and processing unit. The image acquisition and processing unit then processes the overlay view with prompting marks and displays the prompting marks within the operator's line of sight to prompt the operator to move their gaze until the new virtual information enters the operator's line of sight.

[0011] The principle and advantages of this solution are as follows: Through 180° field-of-view image stitching technology, the system provides operators and assistants with a much wider field of view than that of a single eye. This helps assistants observe the scene from a wider perspective and obtain more effective information; the ultra-wide realistic environment display not only enhances the operator's sense of immersion but also reduces the discomfort caused by limited field of view.

[0012] Since the human eye's range of motion is approximately 150° horizontally and 120° vertically, the operator's field of vision may not cover the entire display area of ​​the curved screen. This means the operator's eyes cannot simultaneously see all the image content on the display unit. However, the assistant's screen displays a 180° superimposed image, which may cause the virtual markers placed by the assistant to be outside the operator's current field of vision. Therefore, the eye-tracking unit can monitor the operator's field of vision and position in real time, ensuring that the operator can promptly detect new virtual information added by the assistant and guiding the operator to move their gaze to the key area.

[0013] Preferably, as an improvement, after receiving the prompt instruction, the image acquisition and processing unit analyzes the location of the newly added virtual information in the overlay view and its relationship with the operator's line of sight, and displays a prompt mark at the boundary closest to the location of the newly added virtual information within the operator's line of sight. The prompt mark is a linear arrow pointing to the newly added virtual information.

[0014] The benefits of this improvement are: by displaying directional arrows at the location closest to the newly added virtual information within the operator's line of sight, operators can quickly identify and understand the direction in which they need to move their gaze; this intuitive guidance method reduces the time operators spend searching for and confirming new information, thus improving the overall efficiency of the operation; as a prompt, the linear arrows are clearly directional, helping operators to react quickly, and their simplicity reduces the cognitive burden on operators, allowing them to focus more on the current task.

[0015] Preferably, as an improvement, the AR device module further includes a focus detection unit. The focus detection unit uses a machine learning algorithm to establish a relationship model between pupil parameters and focus range, and calculates the current focus range and focus position of the operator on the operation display unit based on the real-time collected pupil parameters of the operator.

[0016] The remote terminal module also includes an operation status monitoring unit. The operation status monitoring unit merges the focus range and focus position markers output by the focus detection unit onto the overlay view to generate a monitoring view, and transmits the monitoring view to the terminal display unit. The terminal display unit displays the monitoring view to the assisting personnel through the monitor.

[0017] The beneficial effects of this improvement are as follows: Through the focus detection unit, the system can accurately determine the operator's focus position and focus range within the current field of vision; this helps assistants understand whether the operator is truly paying attention to key virtual information, thereby providing more accurate remote guidance; in training scenarios, instructors can view the trainees' focus position and range in real time through the remote terminal module, thereby determining whether the trainees have effectively acquired virtual information or performed the necessary gaze operations; this real-time monitoring and feedback mechanism helps improve trainees' attention and participation, thereby improving training effectiveness.

[0018] Preferably, as an improvement, the pupil parameters collected by the focusing detection unit include pupil size and pupil change rate.

[0019] The beneficial effects of this improvement are: although pupil size can reflect a certain visual attention state, pupil change rate can better reflect the operator's ability to respond quickly to the current visual stimulus; by collecting these two parameters simultaneously, the system can more accurately determine the operator's focusing state; different individuals may have different pupil responses when paying attention to visual attention, and adding the pupil change rate parameter can make the system more adaptable to different operators and improve the universality of detection.

[0020] Preferably, as an improvement, the focusing detection unit outputs the focusing position as the center of the line of sight.

[0021] Preferably, as an improvement, the linear arrow is a gradient color, which is a combination of two colors with a phase difference of 120° to 160°, and the linear arrow is displayed in a flashing state.

[0022] The beneficial effects of this improvement are: the gradient color and flashing state can significantly enhance the visual appeal of the linear arrow, making it easier for operators to notice and understand the information it points to; and the selection of the color can distinguish it from the real environment image to the greatest extent, making it easy to identify.

[0023] Preferably, as an improvement, the system further includes a data processing and analysis module, which uses machine learning algorithms to analyze the operator data collected by the AR device module, identify the operator's skill level, attention allocation pattern and potential error points, and generate an operation evaluation report.

[0024] The benefits of this improvement are: by analyzing operator data, the system can identify their skill level and potential errors, thereby providing personalized guidance and suggestions; the system can adjust the training content and difficulty according to the actual situation of the operators to ensure that they have the best learning experience; and by timely detection and correction of potential errors, the system can significantly reduce safety risks during operation.

[0025] Preferably, as an improvement, the system further includes a system management and configuration module, which adopts a role-based access control model, monitors the device status in real time, records operation logs, and automatically triggers an alarm mechanism when an anomaly is detected.

[0026] The benefits of this improvement are: through a role-based access control model, the system can ensure that only authorized personnel can access and modify critical configuration information; real-time monitoring of device status and recording of operation logs can help system administrators to promptly identify and resolve problems, ensuring the stable operation of the system; and the automatic alarm triggering mechanism can quickly notify relevant personnel when an anomaly is detected, so that they can take necessary countermeasures.

[0027] Preferably, as an improvement, the system further includes a training and assessment module, which uses 3D modeling and virtual reality technology to create first aid scenarios and video tutorials, provides operators with simulated operation experiences through AR head-mounted devices, and designs an assessment system to evaluate and provide feedback to operators.

[0028] The benefits of this improvement are as follows: First aid scenarios and video tutorials created using 3D modeling and virtual reality technology can simulate real operating environments, providing operators with a more realistic operating experience; the assessment system can automatically record and analyze operators' performance in simulated operations, thereby providing more objective and accurate evaluation results; through simulated operations and assessment feedback, operators can gain a clearer understanding of their skill level and areas for improvement, thus enabling them to enhance their skills in a targeted manner. Attached Figure Description

[0029] Figure 1 This is a structural block diagram of an embodiment of the present invention. Detailed Implementation

[0030] The following detailed description illustrates the specific implementation method:

[0031] Example

[0032] The basics are as follows: Figure 1 As shown, an emergency remote visual guidance and training system includes an AR device module and a remote terminal module.

[0033] The AR device module uses an AR head-mounted device to provide operators with an image display that overlays real scenes and virtual information, including an image acquisition and processing unit and an operation display unit.

[0034] The image acquisition and processing unit uses a first camera group (comprising three cameras positioned on the left, center, and right sides of the AR headset) to capture a 180° field-of-view image of the real environment in front of the operator. The images captured by the three cameras partially overlap at their boundaries. Image stitching is performed using SIFT (Scale Invariant Feature Transform) or SURF (Speed-Up Robust Feature Transform) by matching feature points at these boundaries. Specifically, for adjacent images to be stitched, feature points and descriptors are extracted, and the similarity of the descriptors is compared. Matching feature points are found using methods such as Euclidean distance. Based on these matching feature points, a homography matrix is ​​used to calculate the transformation relationship between the images. One image is then transformed to spatially align with the other, ultimately generating a seamless 180° field-of-view image of the real environment. The image acquisition and processing unit also fuses the real environment image with virtual information to form an overlay view.

[0035] The operation display unit utilizes a high-brightness, high-resolution AMOLED curved screen to present the operator with a 180° overlaid view generated by the image acquisition and processing unit. The AMOLED curved screen offers excellent color performance and contrast, while the curved design reduces visual interference, enhancing the operator's immersion and simulating the real environment as seen by the naked eye. AMOLED (Active Matrix Organic Light Emitting Diode) technology displays images through self-emissive pixels, offering advantages such as fast response time, low power consumption, and vibrant colors. The curved design further conforms to human visual habits, reducing eye strain.

[0036] The remote terminal module is used to collect virtual information and provide an overlay view of the operation site for assisting personnel. The remote terminal module includes a terminal display unit, an interactive control unit, and a communication unit.

[0037] The terminal display unit uses a high-definition screen to display the overlay view output by the AR device module, allowing assistants to understand the situation from the operator's first-person perspective in real time. In emergency situations, the remote terminal module can be deployed in remote locations such as emergency centers or hospitals. Assistants can observe the situation through the terminal display unit and provide remote guidance as needed.

[0038] The interactive control unit collects virtual information, such as annotations and instructions, by connecting to input devices like touchscreens and mice, and transmits this information to the image acquisition and processing unit for fusion display. During emergency rescue, assistants can use the interactive control unit to mark locations requiring attention or action on a real-world image, providing intuitive remote guidance.

[0039] The communication unit is used to enable two-way voice communication and data synchronization and transmission between the AR device module and the remote terminal module. Through the two-way voice communication device, real-time communication between the operator and the assistant is ensured; through the data synchronization and transmission mechanism, the overlay view generated by the AR device module is transmitted to the remote terminal module for display in real time.

[0040] The communication unit employs advanced wireless communication technologies, such as 5G and Wi-Fi 6, to support high-speed, low-latency data transmission. Simultaneously, encryption technology ensures the security of data transmission. The two-way voice communication device utilizes high-fidelity microphones and speakers to ensure clear and high-quality voice calls. The data synchronization and transmission mechanism employs efficient data compression and transmission protocols to optimize network resource utilization.

[0041] At traffic accident emergency sites, operators wearing AR headsets enter the scene to conduct rescue operations. A remote terminal module is deployed at the emergency center, allowing assistants to observe the situation through a terminal display unit and mark the locations requiring rescue or instruct operators on specific actions via an interactive control unit on a real-world image. Simultaneously, through a two-way voice communication system, assistants can communicate in real-time with the operators, providing further guidance and support. This remote visual guidance and training system significantly improves the efficiency and quality of emergency care while reducing risks to operators.

[0042] Although the AR device module's operation display unit can show a 180° overlay view, the human eye's range of vision is approximately 150° horizontally and 120° vertically. This means that the operator's field of vision may not cover the entire display area of ​​the curved screen. In other words, the operator's eyes cannot simultaneously see all the image content on the operation display unit. However, the assistant's display screen shows a 180° overlay image, which may result in the virtual markers marked by the assistant being outside the operator's current field of vision. Therefore, it is necessary to detect the range of vision that the operator's current line of sight can cover.

[0043] The AR device module also includes an eye-tracking unit. This unit tracks and records the operator's eye movements in real time, accurately calculating the operator's gaze direction and fixation point. The eye-tracking unit uses the pupil-corneal reflex (PCCR) principle, emitting near-infrared light of a specific wavelength from an infrared light source to illuminate the eye. When this infrared light reaches the eye, it is reflected. Different parts of the eye have different infrared light reflection characteristics. The cornea, as the outermost transparent tissue of the eye, has a smooth surface and produces a strong reflection; while the pupil is relatively dark, and the infrared light entering the pupil is scattered on the retina behind the iris before being reflected back out. A high-precision image sensor captures the infrared light reflected from the cornea and pupil.

[0044] Because the cornea and pupil reflect light differently, they appear differently in an image. For example, the cornea's reflection might create a bright spot, while the pupil might appear relatively dark (in dark-pupil mode) or bright (in bright-pupil mode, when the camera and infrared light source are in the same line, the pupil is illuminated). Features such as image brightness, contrast, and edges are used to determine the boundaries between the pupil and cornea, i.e., their positions in the image. Based on the relative positions of the pupil and cornea, a vector can be calculated between them, reflecting the direction and degree of eye movement.

[0045] During system initialization, a calibration process is performed, requiring the operator to focus on a specific point on the curved screen of the operation display unit to establish a correspondence between eye features and positions on the display. Through initial calibration, a mapping model is established, converting pupil and corneal vectors into the direction of gaze and the position of the fixation point on the curved screen. In actual use, based on continuously acquired eye images from the eye-tracking unit, the pupil and corneal vectors are continuously calculated, and the mapping model is used to convert them into the operator's gaze range and position on the curved screen.

[0046] The eye-tracking unit detects whether the position of the virtual sign is within the operator's current line of sight when new virtual information is input. That is, it compares the coordinate position of the newly added virtual sign on the overlay view to see if it is within the line of sight. If it is determined that the newly added virtual sign is outside the operator's line of sight, it sends a prompting command to the image acquisition and processing unit. The image acquisition and processing unit then processes the overlay view to display the prompting sign within the operator's line of sight, prompting the operator to move their gaze so that the newly added virtual sign enters the operator's line of sight.

[0047] Upon receiving a prompt, the image acquisition and processing unit analyzes the location of the newly added virtual information in the overlay view and its relationship to the operator's line of sight. A prompt marker, a linear arrow pointing towards the newly added virtual marker, is displayed at the boundary closest to the new virtual marker within the operator's line of sight. The arrow's color is a combination of two colors with a hue difference between 120° and 160°, creating a striking gradient effect. This generates a strong visual impact and effectively attracts the operator's attention.

[0048] Furthermore, the image acquisition and processing unit controls the indicator to flash at a certain frequency. The flashing mode can be adjusted according to the actual situation to ensure that it attracts attention without causing visual fatigue. As the operator's gaze moves, the image acquisition and processing unit will adjust the position of the indicator on the overlaid image and the flashing mode in real time to ensure that the operator can obtain new virtual information in a timely and effective manner.

[0049] Because the human eye has specific recognition zones for different elements—it can effectively recognize colors within a 30°–60° range, but the recognition zone for letters is reduced to only 5°–30°, and text, due to its complexity, can only be discerned by the human eye within a 10°–20° range—this means that in an overlay view, even if newly added virtual information is within the operator's line of sight, it may not be effectively processed and absorbed due to improper attention allocation. Furthermore, assistants cannot determine whether the operator has effectively acquired the virtual information, thus affecting guidance for the next step.

[0050] When applied to training scenarios, trainees wear AR device modules for practical operation, while instructors guide them via remote terminal modules, combining verbal instructions and virtual icons. This teaching model aims to deepen trainees' understanding of theoretical knowledge through practice, thereby improving teaching quality. After training, the system can also be transformed into an acceptance testing tool. Particularly in medical emergency training, trainees are required to closely monitor a specific area. However, due to the aforementioned limitations in visual recognition range, examiners find it difficult to accurately determine whether trainees have indeed completed the required gaze.

[0051] The AR device module also includes a focus detection unit, which detects dynamic changes in the pupil and calculates the operator's focus range and position. The focus detection unit uses a high-resolution, low-latency near-eye camera with infrared sensitivity, enabling it to stably capture pupil images under varying lighting conditions. Image processing algorithms extract relevant pupil parameters from the acquired images, including pupil size and pupil change rate. Pupil size can be measured by identifying the pupil's boundaries and calculating its diameter or area. The pupil change rate can be calculated by the difference in pupil size between consecutive frames.

[0052] During the training phase of the focus detection unit, a large amount of pupil data and corresponding actual focusing distance data of operators under different conditions are collected. This data can come from different operators, different environmental conditions, and different task scenarios. Machine learning algorithms such as neural networks and support vector machines are used to establish a model relating pupil conditions to the focusing range at a specific distance. The input to this module is parameters such as pupil size and pupil change rate, and the output is the focusing distance or focusing range. The model is trained using the collected large amount of data, and by continuously adjusting the model's parameters, the model can predict the focusing range and focusing position as accurately as possible. During training, techniques such as cross-validation can be used to evaluate the model's performance and optimize it.

[0053] In practical use, pupil parameters collected in real time by the camera are input into a trained model to calculate the operator's current focus range and focus position. The focus range indicates the area within which the operator can effectively acquire complex information. Typically, the focus range is a region that expands outward from the focus position. The focus position can be considered the central area of ​​the field of vision calculated by the eye-tracking unit, but since the human eye's focus is not perfectly precise, there exists a certain range of focus areas. The focus detection unit outputs the operator's current focus range and focus position.

[0054] The remote terminal module also includes an operation status monitoring unit. This unit analyzes the focusing results output by the focusing detection unit and integrates these results onto a superimposed view to generate a monitoring view. Specifically, it marks the current operator's focusing range and position on the superimposed image. Finally, the monitoring view is transmitted to the terminal display unit. Assistants can use the monitoring view displayed on the terminal display unit to understand the operator's actions and the duration of virtual information within the focusing range, thereby determining whether the operator has effectively acquired the virtual information or performed a gaze operation.

[0055] The emergency remote visual guidance and training system also includes a data processing and analysis module, a system management and configuration module, and a training and assessment module.

[0056] The data processing and analysis module is used to collect, process, analyze, and evaluate various data during first aid and training processes. It stores data collected by the eye-tracking unit and focus detection unit, and collects operator eye-tracking data, including pupil diameter, fixation point trajectory, and blink frequency. This data is combined with operational behavior data, such as operation time, path selection, and number of errors, for comprehensive analysis. Machine learning algorithms, such as cluster analysis and regression analysis, are used to deeply mine the collected data, identifying operator skill levels, attention allocation patterns, and potential error points. Based on the data analysis results, an operational evaluation report is automatically generated, using statistical indicators such as operational efficiency index and accuracy percentage to quantify training effectiveness and providing personalized improvement suggestions.

[0057] The system management and configuration module is used for overall system operation and maintenance management, including user permission management, device monitoring and configuration, logging and anomaly handling, ensuring stable and secure system operation. It adopts a role-based access control model, assigning different access permissions according to user roles to ensure secure data isolation. Utilizing IoT technology, it monitors the status information of hardware such as AR headsets and remote terminals in real time, such as battery level, connection status, and calibration status. An integrated log management system records key events and error logs during system operation. When an anomaly is detected, an alarm mechanism is automatically triggered, notifying relevant personnel via email, SMS, etc.

[0058] The training and assessment module provides abundant training resources and online assessment functions, aiming to improve operators' first aid skills and response capabilities through simulated real-world environments. Utilizing 3D modeling and virtual reality technology, it creates highly realistic first aid scenarios and video tutorials, simulating operational steps and decision-making processes in various emergency situations. Through AR headsets, the virtual scenes are integrated with the real environment, allowing operators to perform simulated operations in a realistic setting, enhancing immersion and practical experience. An automated assessment system is designed to randomly generate assessment scenarios, automatically score and record the operational process, and use AI algorithms to analyze operational details, providing precise feedback.

[0059] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A remote visual guidance and training system for emergency medical care, characterized in that: This includes an AR device module for providing operators with an overlay image display of real scenes and virtual information, and a remote terminal module for collecting virtual information and providing assistants with an overlay view of the operation site. The AR device module includes an image acquisition and processing unit and an operation display unit. The image acquisition and processing unit uses multiple cameras located in different directions to acquire real-world environmental images in a 180° field of view in front of the operator, and generates seamless 180° field-of-view images through image stitching technology. The real-world environmental images are then fused with virtual information to form an overlay view. The operation display unit uses a curved screen to display the overlay view. The remote terminal module includes a terminal display unit for displaying an overlay view to assisting personnel, an interactive control unit for collecting virtual information and transmitting it to the image acquisition and processing unit of the AR device module for fusion display, and a communication unit for two-way voice communication and data synchronization and transmission between the AR device module and the remote terminal module. The AR device module also includes an eye-tracking unit, which uses an infrared light source and a high-precision image sensor to track and record the operator's eye movements in real time, and calculates the operator's current gaze range and gaze position when looking at the operation display unit. The eye-tracking unit is used to detect whether the location of the new virtual information is within the operator's current line of sight when the interactive control unit acquires new virtual information. If it is determined that the new virtual information is outside the line of sight, a prompting instruction is sent to the image acquisition and processing unit. The image acquisition and processing unit then processes the overlay view with prompting marks and displays the prompting marks within the operator's line of sight to prompt the operator to move their gaze until the new virtual information enters the operator's line of sight.

2. The emergency remote visual guidance and training system according to claim 1, characterized in that: After receiving the prompt instruction, the image acquisition and processing unit analyzes the location of the newly added virtual information in the overlay view and its relationship with the operator's line of sight. The prompt mark is displayed at the boundary closest to the location of the newly added virtual information within the operator's line of sight. The prompt mark is a linear arrow pointing to the newly added virtual information.

3. The emergency remote visual guidance and training system according to claim 2, characterized in that: The AR device module also includes a focus detection unit. The focus detection unit uses machine learning algorithms to establish a relationship model between pupil parameters and focus range, and calculates the current focus range and focus position of the operator on the operation display unit based on the real-time collected pupil parameters of the operator. The remote terminal module also includes an operation status monitoring unit, which merges the focus range and focus position markers output by the focus detection unit onto the overlay view to generate a monitoring view, and transmits the monitoring view to the terminal display unit. The terminal display unit presents the monitoring view to the assisting personnel via a monitor.

4. The emergency remote visual guidance and training system according to claim 3, characterized in that: The pupil parameters collected by the focusing detection unit include pupil size and pupil change rate.

5. The emergency remote visual guidance and training system according to claim 4, characterized in that: The focusing detection unit outputs the focusing position as the center of the line of sight.

6. The emergency remote visual guidance and training system according to claim 5, characterized in that: The linear arrow is a gradient color, consisting of two colors with a phase difference between 120° and 160°. The linear arrow is displayed in a flashing state.

7. The emergency remote visual guidance and training system according to claim 6, characterized in that: The system also includes a data processing and analysis module, which uses machine learning algorithms to analyze the operator data collected by the AR device module, identify the operator's skill level, attention allocation pattern and potential error points, and generate an operation evaluation report.

8. The emergency remote visual guidance and training system according to claim 7, characterized in that: The system also includes a system management and configuration module, which adopts a role-based access control model, monitors the device status in real time, records operation logs, and automatically triggers an alarm mechanism when an anomaly is detected.

9. The emergency remote visual guidance and training system according to claim 8, characterized in that: The system also includes a training and assessment module, which uses 3D modeling and virtual reality technology to create first aid scenarios and video tutorials, provides operators with simulated operation experiences through AR head-mounted devices, and designs an assessment system to evaluate and provide feedback to operators.

Images