Interactive laboratory welding safety monitoring system and method based on multi-mode perception

Abstract

The invention discloses an interactive laboratory welding safety monitoring system and method based on multi-mode sensing, and belongs to the field of laboratory intelligent safety monitoring, and the system comprises a hardware sensing layer, an intelligent control layer, an execution and interaction layer and a communication and remote layer. The system takes an embedded processor as a core, and integrates a camera, a state sensor, an environment sensor, an intelligent socket and an applet. The use states of the electric soldering iron are recognized in real time through the target detection model, including handheld, on-rack and illegal placement, and alarming and automatic power-off graded response are achieved in combination with sensor data and the use duration set by a user. And meanwhile, the public network remote real-time access of a monitoring picture is realized by utilizing an intranet penetration technology. The problems that supervision lags behind and the means is single in traditional laboratory electric soldering iron safety management are solved, whole-process and systematic safety protection from user identity binding to remote intelligent intervention is achieved, and the management efficiency and safety are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent laboratory safety monitoring, specifically to an interactive laboratory welding safety monitoring system and method based on multimodal perception. Background Technology

[0002] In university electronics labs and various research and development sites, soldering irons are basic and essential tools, operating at extremely high temperatures, typically exceeding 300°C. However, students often neglect or operate them improperly during hands-on practice, leaving the soldering iron powered on after use, failing to return it to its stand, or leaving it unattended for extended periods. This can easily lead to serious safety accidents such as burns and fires. Traditional safety management models rely heavily on the self-discipline of personnel and on-site inspections by administrators, resulting in problems such as delayed supervision, low efficiency, and inability to intervene in real time.

[0003] In existing technologies, most improvements focus on the soldering iron itself. For example, some designs use built-in accelerometers to achieve automatic shutdown when not in use, or integrate automatic temperature control and power-off functions into the soldering iron. While these solutions have some effect, they do not form a systematic monitoring system for the "operator-equipment-environment". In addition, single smoke and fire alarm systems have a high false alarm rate in scenarios where welding smoke is generated, and lack specificity.

[0004] Therefore, there is an urgent need in this field for an intelligent system that can deeply integrate user management, behavior recognition, environmental monitoring and remote control to achieve full-process, proactive protection of laboratory welding safety. Summary of the Invention

[0005] The technical problem to be solved by this invention is to address the shortcomings of the prior art by proposing an interactive laboratory welding safety monitoring system and method based on multimodal perception that integrates identity authentication, state perception, behavior recognition, environmental monitoring and remote linkage, thereby achieving a safety management upgrade from passive response to proactive prevention. Specifically, it includes: a hardware perception layer, an intelligent control layer, an execution and interaction layer and a communication and remote layer. The hardware perception layer includes an image acquisition module, a status perception module, and an environmental safety sensor group; the image acquisition module is used to capture image data of the experimental workbench in real time; the status perception module includes a gyroscope for detecting the movement of the soldering iron; the environmental safety sensor group includes a temperature sensor and a smoke sensor. The intelligent control layer includes a main control module, a vision processing module, and a data fusion decision module. The main control module uses an embedded processor, which is electrically connected to each module of the hardware perception layer. The vision processing module is equipped with a pre-trained target detection model, which identifies the state of the soldering iron based on the image data captured by the image acquisition module. The data fusion decision module fuses the sensor data from the environmental safety sensor group with the identified state of the soldering iron and executes the core logic judgment. The execution and interaction layer includes an execution module and a human-machine interaction module; the execution module is electrically connected to the intelligent control layer and is used to control the power supply and alarm device of the soldering iron; the human-machine interaction and communication module is used to interact with the user terminal for data. The communication and remote layer uploads the monitoring video stream from the main control module to the public network.

[0006] Furthermore, the main control module of the intelligent control layer is equipped with safety thresholds corresponding to temperature sensors and smoke sensors; when the data of any sensor exceeds its safety threshold, the intelligent control layer determines that the environment is dangerous and triggers the corresponding operation in the execution and interaction layer.

[0007] Furthermore, the intelligent control layer conducts a comprehensive safety risk assessment based on the real-time position status of the soldering iron, the estimated usage time, and data from the environmental safety sensor group, and controls the execution module to perform corresponding safety operations.

[0008] Furthermore, the process of executing core logical judgments includes: When the soldering iron is detected to be in an "improperly placed" state, or when the environmental safety sensor group detects data exceeding a preset threshold, it is determined that an alarm is required. The control execution module triggers a local alarm, and if no manual confirmation signal is received within a preset time, the control execution module cuts off the power to the smart socket. When the soldering iron has been used for the time set by the user and no extension request has been received, the power will be cut off after the preset grace period has expired.

[0009] Furthermore, user registration is completed by scanning a unique identifier code bound to the experimental platform. The registration information includes at least the name and student ID or the name and employee ID.

[0010] Furthermore, the main control module uses an embedded processor called Orange Pie.

[0011] An interactive laboratory welding safety monitoring method based on multimodal perception is applied to the aforementioned interactive laboratory welding safety monitoring system based on multimodal perception. The method includes the following steps: Step 1: User identity binding and device activation. The user completes identity registration and sets the expected usage time through the user terminal. The intelligent control layer controls the power supply to the soldering iron and starts multimodal data acquisition accordingly. Step 2: Multimodal data synchronous acquisition. The images, physical status and environmental parameters of the experimental workbench are acquired in real time through the image acquisition module, the status perception module and the environmental safety sensor group. Step 3: Intelligent judgment of safety status. Based on the collected image data, the intelligent control layer identifies the status of the soldering iron through a pre-trained target detection model and integrates data from multiple sensors for a comprehensive safety assessment. Step 4: Tiered response and remote linkage. Based on the judgment result, tiered security operations are performed at the interaction layer, including local alarms, remote notifications, and automatic power-off, and all status information and real-time images are synchronized to the user terminal.

[0012] Furthermore, tiered response and remote linkage include: If the soldering iron is detected to be in an "improperly placed" state, the system will immediately activate a local alarm and report it to the management terminal. If no manual intervention confirmation is received within the set time, the power will be automatically cut off. If the soldering iron is detected to be in "handheld soldering iron" mode, it is determined to be safe to use, and the user can manually end the use. If the environmental safety sensor array detects fire characteristics, it will immediately trigger the highest level alarm and execute a power outage.

[0013] Compared with the prior art, the present invention achieves the following technical effects: 1. A closed-loop security management system has been established: a complete management loop is formed from "user identity registration - device power supply - real-time monitoring - behavior judgment - automatic intervention - remote notification", and security rules are solidified into system processes.

[0014] 2. High recognition accuracy and reliability: The YOLOv5 model is trained using a large-scale dataset that has been specially collected and labeled, and sensor redundancy verification is introduced, which effectively solves the recognition problem in complex scenarios and significantly reduces the false alarm rate.

[0015] 3. Revolutionary improvement in management efficiency: Administrators can remotely monitor multiple workstations through a mini-program, eliminating the need for frequent on-site inspections and realizing the "digitalization, remote operation, and intelligentization" of laboratory safety management.

[0016] 4. High system integration and scalability: Centered on Orange Pie, it integrates general-purpose sensors and smart hardware, and achieves interaction through mini-programs and cloud services. The cost is controllable, it is easy to deploy on a large scale in universities and enterprise laboratories, and it can be extended to the management of other high-risk devices. Attached Figure Description

[0017] For ease of explanation, the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0018] Figure 1A schematic diagram of the structure of an interactive laboratory welding safety monitoring system based on multimodal perception provided in an embodiment of the present invention; Figure 2 This is a visualization of the confusion matrix of the YOLOv5 model after training, as shown in this embodiment of the invention. Figure 3 This is a visualization of the recall-confidence curve training results of the YOLOv5 model after training, as shown in this embodiment of the invention. Figure 4 This is a visualization of the training process of the YOLOv5 model implemented in this invention; Figure 5 This is a visualization of the label distribution and bounding box size statistics of the YOLOv5 model after training, as shown in this embodiment of the invention. Figure 6 A schematic diagram of the user interface of a mini-program for an interactive laboratory welding safety monitoring system and method based on multimodal perception, provided in an embodiment of the present invention; Figure 7 A schematic diagram of a public network remote monitoring screen for an interactive laboratory welding safety monitoring system and method based on multimodal perception, provided in an embodiment of the present invention; Figure 8 The overall architecture and workflow diagram of an interactive laboratory welding safety monitoring system and method based on multimodal perception provided in this embodiment of the invention; Figure 9 This is a schematic diagram illustrating a specific application scenario of an interactive laboratory welding safety monitoring system and method based on multimodal perception, provided in an embodiment of the present invention. Detailed Implementation

[0019] The following are specific embodiments of the present invention, described in conjunction with the accompanying drawings, to further illustrate the technical solutions of the present invention. However, the present invention is not limited to these embodiments. Specific details, such as particular configurations, are provided in the following description merely to aid in a comprehensive understanding of the embodiments of the present invention. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention.

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.

[0021] Example 1:

[0022] This invention provides an interactive laboratory welding safety monitoring system based on multimodal perception, which constructs a complete monitoring closed loop through "hardware and software integration" and "cloud collaboration".

[0023] like Figure 1As shown, an embodiment of an interactive laboratory welding safety monitoring system based on multimodal perception is presented. The hardware perception layer includes an image acquisition module, a status perception module, and an environmental safety sensor group.

[0024] The image acquisition module can be a USB camera for capturing real-time image data of the experimental workbench; the status perception module includes a gyroscope for detecting the movement of the soldering iron; the environmental safety sensor group includes a temperature sensor, a smoke sensor, etc.

[0025] The intelligent control layer comprises a main control module, a vision processing module, and a data fusion decision module. The main control module uses an embedded processor (Orange Pi) and is responsible for fusing multi-source data, including visual and environmental data, and executing core decision-making logic. The vision processing module is equipped with a pre-trained target detection model. This model identifies the soldering iron's state based on image data captured by the image acquisition module. The vision processing module in the intelligent control layer utilizes a target detection model trained using the YOLOv5 algorithm. The data fusion decision module fuses sensor data from the environmental safety sensor group with the identified soldering iron state and executes core logical judgments.

[0026] LabelImg was used to perform detailed annotations on over 1500 images, including those showing empty soldering iron stands, soldering iron stands with soldering irons, handheld soldering irons, and improperly placed soldering irons. The annotated images were then used to train a YOLOv5 model. By adjusting hyperparameters and adding data augmentation, a high-precision recognition model was obtained and deployed to Xiangcheng Pi. The YOLOv5 model training results are shown below. Figure 2-5 As shown. Figure 2 In the diagram, the horizontal axis (True) represents the true class, and the vertical axis (Predicted) represents the class predicted by the model. The value of each cell represents the proportion of the true class that is predicted as a certain class (normalized). The diagonal line indicates a correct prediction, and the off-diagonal line indicates a misclassification. Figure 3 In the diagram, the horizontal axis represents the confidence threshold, the vertical axis represents the recall rate, and the curve represents how many true targets the model can still retrieve when the prediction confidence requirement is increased. Figure 4In the dataset, train / box_loss represents the bounding box regression error during the training phase, train / obj_loss represents the target existence judgment error during the training phase, train / cls_loss represents the target existence judgment error during the training phase, val / box_loss represents the bounding box regression error during the validation phase, val / obj_loss represents the target existence judgment error during the validation phase, val / cls_loss represents the target existence judgment error during the validation phase, metrics / precision represents the percentage of true positive samples that are actually true, metrics / recall represents the proportion of true targets that were successfully detected, metrics / mAP@0.5 represents the average detection precision under IoU=0.5, and metrics / mAP@0.5:0.95 represents the comprehensive metric under more stringent IoU conditions. Figure 5 In the diagram, x and y represent the normalized positions of the target center point in the image, width and height represent the normalized width and height of the target box, the diagonal lines represent the histogram distribution of each variable, and the off-diagonal lines represent the joint distribution among the variables.

[0027] The trained YOLOv5 model can accurately identify the status of soldering irons, such as an empty soldering iron stand, a soldering iron stand with a soldering iron, a handheld soldering iron, and improper placement of the soldering iron. The data fusion decision module fuses sensor data from the environmental safety sensor group to determine subsequent actions, such as power outages and alarms.

[0028] The execution and interaction layer includes an execution module and a human-machine interaction module. The execution module is electrically connected to the intelligent control layer and is used to control the power supply to the soldering iron and the alarm device. The human-machine interaction and communication module is used for data interaction with the user terminal. The execution module may include a Xiaomi smart socket for unlocking and powering on via QR code and for remote power off; and an alarm device that provides audible and visual alarms via a buzzer and LED. The human-machine interaction and communication module includes a WeChat mini-program, which provides both user-end and management functions, such as... Figure 6 The user terminal provides functions such as scanning codes, registration, usage renewal, and manual shutdown, while the management terminal provides functions such as remote real-time monitoring, alarm information reception, and issuing commands.

[0029] The communication and remote layer utilizes "FRP intranet penetration technology" to upload the monitoring video stream from the local Orange Pi to the public network (such as Alibaba Cloud), and uses mjpg-streamer to start the video stream server; control scripts are written to implement sensor data reading, model inference, logical judgment, and socket control functions. This enables real-time public network visualization access to the laboratory screen and facilitates human-computer interaction, such as... Figure 7 As shown.

[0030] Example 2:

[0031] Combination Figure 8 The overall architecture and workflow diagram shown illustrates how this invention achieves a shift from passive response to proactive prevention in a complete use case, including the following core steps: S1. User Identity Binding and Device Activation: User identity binding and device activation: Users complete identity registration and set the expected usage time through the user terminal. The intelligent control layer then controls the power supply to the soldering iron and starts multimodal data acquisition.

[0032] Users scan a dedicated QR code using their personal WeChat account at their workstation. This QR code is linked to a specific workstation and hardware system, automatically redirecting to the corresponding WeChat mini-program interface. Users complete necessary personal information registration on the mini-program, typically including at least their name and student ID. This associates the welding operation with a specific person in charge, shifting from anonymous to real-name operation and enhancing users' sense of responsibility. Users manually set the estimated duration of the operation (e.g., 30 minutes, 1 hour) and power outage intervals on the mini-program, introducing a time management dimension. The system uses this as an important benchmark for judging whether the operation is abnormal (e.g., prolonged inactivity). The mini-program sends the user's identity information and the set usage duration to the backend server. After server verification, it sends a command to the main controller (Orange Pie) at the corresponding workstation. The main controller then sends a command to the Xiaomi smart socket, driving the smart socket to power the soldering iron. Simultaneously, the camera and all sensors at that workstation begin operating, and the system enters real-time monitoring mode.

[0033] User identity binding and device activation have changed the traditional "ready-to-use" extensive model of laboratories, and built a closed loop of responsibility for safety management through simple scanning and timing.

[0034] S2. Synchronous acquisition of modal data: Multimodal data is acquired synchronously, and images, physical status and environmental parameters of the experimental workbench are collected in real time through cameras, status sensors and environmental sensors.

[0035] After the equipment is activated, the system relies on the environmental safety sensor array to enter a continuous multi-source information sensing state. In terms of visual perception, the USB camera continuously captures real-time images of the experimental workbench at a specific frame rate (e.g., 15fps), containing two key pieces of information: The system monitors two key aspects: first, the state of the soldering iron itself, including its location, whether it's being held, on a stand, or improperly placed on the table; second, the operator's status, specifically whether they are at the workbench. For state awareness, a gyroscope detects whether the soldering iron has been moved, picked up, or dropped, while a YOLOv5 model visually identifies the iron's state. For environmental awareness, a temperature sensor monitors the temperature near the soldering iron tip or in the surrounding environment to prevent abnormal overheating. A smoke sensor detects the concentration of smoke generated during soldering, serving as an early warning system for fires. A red and ultraviolet flame sensor actively detects open flames to confirm the occurrence of a fire.

[0036] A three-dimensional, redundant sensing network was constructed, overcoming the limitations of a single sensor (such as smoke alarms which are prone to false alarms) or a single vision (which is susceptible to obstruction and light).

[0037] S3. Security status assessment based on model training and sensor fusion: Intelligent safety status assessment: Based on the collected image data, the system identifies the status of the soldering iron through a pre-trained target detection model and integrates data from multiple sensors for a comprehensive safety assessment.

[0038] Visual intelligent recognition: The main controller inputs the images captured by the camera into a pre-trained YOLOv5 model. This model is a key achievement of the project. Trained on a self-built dataset of 1500+ images, it can accurately recognize and output the following labels: The handheld soldering iron is set to "Safe to use".

[0039] The soldering iron stand with the soldering iron is set to "compliant standby".

[0040] The soldering iron stand is not placed properly (i.e., the soldering iron is placed on a non-support area such as a desktop), and the status is judged as "dangerous".

[0041] Meanwhile, the system employs multi-source data fusion and redundant verification, not simply relying on visual recognition results, but performing cross-validation. For example, in consistency verification, when YOLO identifies a "soldering iron stand with a soldering iron" and the gyroscope detects the soldering iron placed on the stand, the system is highly confident that the state is "compliant standby." In conflict handling, if YOLO identifies an "empty stand," but the gyroscope detects the soldering iron placed on the stand, the system will mark it as "abnormal," potentially triggering a review mechanism or logging, indicating possible visual occlusion or other issues.

[0042] Timing and logical judgment: The system combines "estimated usage time" for judgment. For example, even if the soldering iron is always on the soldering iron stand, if it is not turned off at the set time interval, the system will judge it as "the operator may have forgotten" and trigger an alarm. Thresholds are set for environmental sensor data. When the smoke concentration or temperature exceeds the safety threshold, or when the flame sensor is triggered, it is directly judged as the highest level "environmental hazard" and an alarm is triggered.

[0043] By using AI models, sensor redundancy verification, and timing logic, the accuracy and reliability of state judgment have been greatly improved, achieving a leap from perception to cognition and significantly reducing the probability of false alarms and missed alarms.

[0044] S4. Remote monitoring linkage: Key statuses (equipment status, alarm information) throughout the process are synchronized to the mini-program management terminal in real time.

[0045] Tiered response and remote linkage: Based on the judgment results, tiered safety operations are executed, including local alarms, remote notifications, and automatic power-off, and all status information and real-time images are synchronized to the mini-program.

[0046] The main controller, Orange Pie, uses a USB camera to capture video streams, which are then uploaded to the public network via FRP intranet penetration and mjpg-streamer technology. The public network then sends the video stream to a mini-program platform, allowing administrators to view the laboratory's monitoring screen in real time from anywhere with internet access, enabling remote visual inspection. Simultaneously, user and administrator actions performed on the mini-program transmit unlocking information, usage time information, and other data to the public network. The public network then transmits this information to the main controller, Orange Pie, to trigger actions such as unlocking, powering on, and alarm activation. All user operations, system alarms, and power outage events are logged, creating a traceable security audit trail.

[0047] A dual-protection mechanism combining on-site automated response and remote manual supervision has been established. The hierarchical response strategy reflects intelligence and humanization, while remote linkage completely breaks down the spatial limitations of safety management.

[0048] Specifically, users scan the workstation QR code using WeChat to access the mini-program, register their student ID, and set the expected usage time. After verifying the information, the backend server sends an authorization command to the corresponding smart socket. The smart socket then powers on and simultaneously activates the camera and all sensors, putting the system into monitoring mode. While the user is soldering, the YOLOv5 model identifies the soldering iron's position. If the soldering iron is on the stand and not powered off within the set time interval, an alarm is triggered, and the information is sent to the mini-program management terminal. If the soldering iron is not on the stand, the system checks if it is in the user's hand. If not, an alarm is triggered, and the information is sent to the mini-program management terminal. If it is in the user's hand, the user can power it off via the mini-program or manually after use. If the YOLOv5 model detects a flame, an alarm is triggered directly, and the information is sent to the mini-program management terminal. After receiving the alarm notification, the administrator can view the real-time footage through a remote monitoring platform to confirm the situation. The system starts with a timer. If no "confirmation" instruction is received from the administrator or user via the mini-program before the timer expires, the system will automatically cut off the power and complete the automatic intervention.

[0049] Example 3:

[0050] like Figure 9 As shown, this embodiment of the invention provides a specific application scenario for an interactive laboratory welding safety monitoring system and method based on multimodal perception, which specifically includes the following steps: Step 1: Identity Binding: Student Zhang San enters the lab and scans the workstation's QR code using WeChat. The mini-program prompts him to enter his name and student ID, and sets the estimated time to 40 minutes. After submission, the cloud server verifies the information and sends a "power authorization" command to the Orange Pie smart socket at that workstation. The Orange Pie then activates the smart socket to power the soldering iron.

[0051] Step 2, Data Acquisition: After the system starts up, the camera, gyroscope, temperature and smoke sensors begin to work continuously, sending multimodal data to Orange Pie in real time.

[0052] Step 3, Risk Identification: While Zhang San was soldering, the system stably identified it as "Safe to Use." Subsequently, Zhang San temporarily left to answer a phone call, placing the heated soldering iron flat on the wooden table. At this point: The YOLO model identified "improper placement of the soldering iron".

[0053] The gyroscope signal indicates "stationary".

[0054] The system confirmed the "dangerous state" through a redundancy check mechanism.

[0055] Step 4, Intervention and Handling: The system triggers a local audible and visual alarm and pushes an alarm message, "Improper placement of soldering iron at workstation," to the administrator, Mr. Li's, mini-program. Administrator Mr. Li views the remote real-time video stream through the management terminal, confirms the situation, and performs operations such as powering off the power. If Mr. Li does not take any action within 30 seconds, the system automatically cuts off the power to the smart socket. The entire event (scanner, alarm time, and handling result) is fully recorded, forming a safety audit log.

[0056] Step 5, Process Closure and Behavior Correction: Upon returning, Zhang San finds the device powered off and must re-authorize by scanning the code to continue using it. If a student's usage time expires, they can apply for an extension; otherwise, the system will automatically power off after the grace period.

[0057] This invention requires users to register by scanning a QR code to activate the device, binding operational behavior to personal identity and achieving traceability of responsibility. By using the YOLOv5 model to identify the soldering iron's status and cross-checking it with the physical signals from the gyroscope, the accuracy of status judgment is greatly improved. The concept of "estimated usage time" is introduced, combined with visual recognition of the "handheld" safety status, enabling time-based automatic power-off and behavior-based flexible management. From local alarms and delayed power-offs for "improper placement" to immediate power-offs in hazardous environments, and real-time push notifications to administrators via a mini-program, a tiered response and remote monitoring mechanism is formed. This invention integrates discrete technical modules (vision, sensors, mini-programs) into a closed-loop, proactive, and traceable intelligent safety management system through a rigorous logical process.

[0058] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0059] Those skilled in the art to which this application pertains may make various modifications or additions to the specific embodiments described, or adopt similar methods to replace them, without departing from the inventive concept of this application or exceeding the scope defined by the appended claims.

Claims

1. An interactive laboratory welding safety monitoring system based on multimodal perception, characterized in that, include: Hardware perception layer, intelligent control layer, execution and interaction layer, and communication and remote layer; The hardware sensing layer includes an image acquisition module, a state sensing module, and an environmental safety sensor group; the image acquisition module is used to capture image data of the experimental workbench in real time; the state sensing module includes a gyroscope for detecting the movement of the soldering iron; the environmental safety sensor group includes a temperature sensor and a smoke sensor. The intelligent control layer includes a main control module, a vision processing module, and a data fusion decision module. The main control module uses an embedded processor, which is electrically connected to each module of the hardware perception layer. The vision processing module is equipped with a pre-trained target detection model, which identifies the state of the soldering iron based on the image data captured by the image acquisition module. The data fusion decision module fuses the sensor data from the environmental safety sensor group with the identified state of the soldering iron and performs core logic judgment. The execution and interaction layer includes an execution module and a human-computer interaction module; the execution module is electrically connected to the intelligent control layer and is used to control the power supply of the soldering iron and the alarm device; the human-computer interaction and communication module is used to interact with the user terminal for data. The communication and remote layer uploads the monitoring video stream of the main control module to the public network.

2. The interactive laboratory welding safety monitoring system based on multimodal perception according to claim 1, characterized in that, The main control module of the intelligent control layer is equipped with safety thresholds corresponding to temperature sensors and smoke sensors; when the data of any sensor exceeds its safety threshold, the intelligent control layer determines that the environment is dangerous and triggers the corresponding operation in the execution and interaction layer.

3. The interactive laboratory welding safety monitoring system based on multimodal perception according to claim 1, characterized in that, The intelligent control layer performs a comprehensive safety risk assessment based on the real-time position status of the soldering iron, the estimated usage time, and the data from the environmental safety sensor group, and controls the execution module to perform corresponding safety operations.

4. The interactive laboratory welding safety monitoring system based on multimodal perception according to claim 3, characterized in that, The process of executing the core logic judgment includes: When the soldering iron is detected to be in an "improperly placed" state, or when the environmental safety sensor group detects data exceeding a preset threshold, it is determined that an alarm is required. The control execution module is then activated to trigger a local alarm. If no manual confirmation signal is received within a preset time, the control execution module will cut off the power to the smart socket. When the soldering iron has been used for the time set by the user and no extension request has been received, the power will be cut off after the preset grace period has expired.

5. The interactive laboratory welding safety monitoring system based on multimodal perception according to claim 1, characterized in that, The user registration is completed by scanning the unique identification code bound to the experimental platform. The registration information includes at least the name and student ID or the name and employee ID.

6. The interactive laboratory welding safety monitoring system based on multimodal perception according to claim 1, characterized in that, The main control module uses an embedded processor called Orange Pie.

7. An interactive laboratory welding safety monitoring method based on multimodal perception, characterized in that, Applied to the system as described in any one of claims 1-6, the method comprises the following steps: Step 1: User identity binding and device activation. The user completes identity registration and sets the expected usage time through the user terminal. The intelligent control layer controls the power supply to the soldering iron and starts multimodal data acquisition accordingly. Step 2: Multimodal data synchronous acquisition. The images, physical status and environmental parameters of the experimental workbench are acquired in real time through the image acquisition module, the status perception module and the environmental safety sensor group. Step 3: Intelligent judgment of safety status. Based on the collected image data, the intelligent control layer identifies the status of the soldering iron through a pre-trained target detection model and integrates data from multiple sensors for a comprehensive safety assessment. Step 4: Tiered response and remote linkage. Based on the judgment result, tiered security operations are performed at the interaction layer, including local alarms, remote notifications, and automatic power-off, and all status information and real-time images are synchronized to the user terminal.

8. The interactive laboratory welding safety monitoring method based on multimodal perception according to claim 7, characterized in that, The hierarchical response and remote linkage include: If the soldering iron is detected to be in an "improperly placed" state, the system will immediately activate a local alarm and report it to the management terminal. If no manual intervention confirmation is received within the set time, the power will be automatically cut off. If the soldering iron is detected to be in "handheld soldering iron" mode, it is determined to be safe to use, and the user can manually end the use. If the environmental safety sensor array detects fire characteristics, it will immediately trigger the highest level alarm and execute a power outage.

Images