Routing inspection and safety training support method and system based on augmented reality and risk analysis
By combining HAZOP and FMEA risk analysis methods with RFID technology and image recognition, the problem of insufficient structured risk analysis in existing AR inspection and safety training systems is solved, and high-accuracy risk visualization training with low dependence on human resources is achieved.
Patent Information
- Application Number
- CN202510956576.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-11
- Publication Date
- 2025-10-17
AI Technical Summary
Existing AR-based inspection and safety training systems lack structured risk analysis methods and cannot effectively combine data collection and image algorithms, resulting in high reliance on human resources for training and low object recognition accuracy.
Using HAZOP and FMEA risk analysis methods, combined with RFID technology and image recognition, we visualize the risks in production scenarios. We also conduct 3D modeling and human-computer interaction in an augmented reality environment, design inspection processes and record sheets, and achieve preliminary visualization of risk analysis.
It reduces dependence on human resources, improves the professionalism of training and the accuracy of object recognition, enhances the interpretability of risk analysis and the effectiveness of training, and reduces training errors.
Smart Images
Figure CN120806932A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of augmented reality and risk analysis, and in particular to an inspection and safety training support method and system based on augmented reality and risk analysis. Background Art
[0002] Safety training and inspections, as key components of safety management, are essential for preventing accidents and improving risk management capabilities. Traditional inspections and safety training are often conducted through written or on-site instruction, requiring high levels of experience and labor costs. Augmented reality (AR) technology can assist in production scenarios and enhance visualization, finding widespread application in safety training.
[0003] Existing AR-based inspection and safety training methods or systems mostly rely on data collection and image algorithm comparison, without adopting structured risk analysis methods, and lack the ability to combine the two to achieve better safety inspections and training.
[0004] Therefore, how to provide an inspection and safety training support method and system based on augmented reality and risk analysis methods is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of this, the present invention provides an inspection and safety training support method and system based on augmented reality and risk analysis, which can provide inspection and safety training support and assistance. Through risk visualization and safety training support, it is helpful to improve the technical level of practitioners and reduce the probability of accidents.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] Inspection and safety training support methods based on augmented reality and risk analysis, including:
[0008] Conduct risk analysis on production scenarios, formulate risk analysis prompts based on risk analysis methods and domain knowledge, design inspection processes, retain inspection content, and generate inspection records;
[0009] Conduct 3D modeling of production process equipment and construct AR scenes in an augmented reality environment. Add risk analysis prompts and inspection processes to the AR scenes to form AR projects. Combine the AR projects with RFID technology to complete preliminary visualization of risk analysis.
[0010] After preliminary visualization, image recognition and data analysis technology are used to identify appearance defects and hidden dangers of production process equipment and display them in real time on the AR interface.
[0011] Design the human-computer interaction interface and complete the human-computer interaction functions.
[0012] Preferably, the risk analysis method comprises HAZOP risk analysis and FMEA risk analysis.
[0013] Preferably, the HAZOP risk analysis comprises:
[0014] The HAZOP risk analysis comprises: dividing nodes, selecting guide words, determining deviations, analyzing reasons, evaluating consequences, proposing suggestions and emergency measures;
[0015] The HAZOP risk analysis result is judged in combination with domain knowledge to form a HAZOP analysis result table;
[0016] The production process equipment introduction, function and operation are combined with the deviations, evaluated consequences, analyzed reasons, suggestions and emergency measures in the HAZOP analysis result table to formulate risk analysis prompt content.
[0017] Preferably, the FMEA risk analysis comprises:
[0018] The FMEA risk analysis comprises: determining the analysis range, identifying potential failure modes, analyzing failure impacts, determining failure causes, evaluating existing control measures, calculating risk priority numbers and formulating improvement measures;
[0019] The FMEA risk analysis result is judged in combination with domain knowledge to form a FMEA analysis result table;
[0020] The production process equipment introduction, function and operation are combined with the potential failure modes, failure causes, risk priority numbers and improvement measures in the FMEA analysis result table to formulate risk analysis prompt content.
[0021] Preferably, a design inspection process, inspection content retention and generation of an inspection record table are retained, comprising:
[0022] Designing an inspection process: combining domain knowledge, risk analysis method results and process guidelines to design inspection steps and standards;
[0023] Retaining inspection content: based on augmented reality and according to the inspection process, performing work, and through an AR mobile terminal, collecting pictures of inspection operations, and transmitting the pictures of inspection operations to a background server for retention;
[0024] Generating an inspection record table: based on the inspection content, generating an inspection record table on a background server to form an inspection database.
[0025] Preferably, in an augmented reality environment, a three-dimensional model of equipment in a production scene is established and an AR scene is constructed, risk analysis prompt content and an inspection process are added to the AR scene to form an AR project, comprising:
[0026] Use the basic geometry of Unity3D or use third-party modeling software to create a three-dimensional model of the equipment in the production scene, and import Unity3D. For parts that are not easy to label or find on the device body, label them on the three-dimensional model for easy operation.
[0027] Complete the AR scene configuration in Unity3D, drag the created three-dimensional model from the Assets folder to the Hierarchy panel to make it part of the AR scene;
[0028] Create a UI element in the Hierarchy panel, add risk analysis prompt content and inspection process to the AR scene to form an AR project.
[0029] Preferably, the AR project is combined with RFID technology, including:
[0030] Save the AR project to the local database and index the AR project;
[0031] Design an RFID tag for the AR project;
[0032] In the local database, establish a correspondence table between the RFID tag ID and the AR project index number;
[0033] Transmit the read RFID tag data to the AR mobile terminal and extract the RFID tag ID;
[0034] According to the parsed RFID tag ID, obtain the detailed information of the corresponding AR project from the local database.
[0035] Preferably, based on image recognition and data analysis technology, judge the appearance defects and hidden dangers of process equipment, and display them in real time on the AR interface, including:
[0036] Collect equipment images through the AR mobile terminal, and judge the appearance defects of the equipment by comparing them with normal images through image recognition algorithms;
[0037] Transmit the parameters of the device sensor to the background server, and obtain the hidden points of the device through data analysis;
[0038] Transmit the appearance defects identified by the image recognition and the hidden points obtained by data analysis to the display interface of the AR mobile terminal for real-time display and marking of the hidden points;
[0039] Preferably, design a human-computer interaction interface and complete the human-computer interaction function, including:
[0040] Design a human-computer interaction interface, which includes device name, three-dimensional model, inspection process and risk analysis prompt content;
[0041] Designing interaction rules, i.e. action and posture patterns, on the human-computer interaction interface, processing and fusing data of features reflecting hand action and posture in the data glove, extracting features reflecting hand action and posture therefrom and matching them with pre-defined action and posture patterns.
[0042] The inspection and safety training support system based on augmented reality and risk analysis comprises:
[0043] The risk analysis and inspection design module is used for risk analysis on a production scene and formulating risk analysis prompt content based on risk analysis methods, designing an inspection process, retaining inspection content and generating an inspection record table;
[0044] The AR visualization module is used for three-dimensional modeling and AR scene construction of production process equipment in an augmented reality environment, adding risk analysis prompt content and an inspection process in the AR scene to form an AR project, and combining the AR project with RFID technology to complete preliminary visualization of risk analysis;
[0045] The hidden danger judgment module is used for judging appearance defects and hidden dangers of production process equipment based on image recognition and data analysis technology after preliminary visualization and displaying them in real time on an AR interface;
[0046] The interaction module is used for designing a human-computer interaction interface and completing human-computer interaction functions.
[0047] According to the technical solution, compared with the prior art, the inspection and safety training support method and system based on augmented reality and risk analysis have the following advantages:
[0048] 1) The existing AR-based inspection and safety training system only compares data acquisition and image algorithms, does not use a structured risk analysis method, and only compares data and images with correct states, lacks an understandable and explainable analysis process. The present application uses a safety field risk analysis method to analyze and investigate hidden dangers and risks, which is convenient for employees to understand risks and investigate hidden dangers.
[0049] 2) The existing safety training system requires high human resources, mostly relies on old workers to train new workers, or collects images of new workers' operations, records them in the cloud and judges and feeds back from experienced old workers or experts. The present application visualizes experience by using AR technology and a risk analysis method, reduces the dependence on human resources and improves professional skills.
[0050] 3) Existing AR inspection and safety training systems mostly use machine vision for object recognition, which has accuracy issues. RFID technology is often used in AR for positioning rather than object recognition. This invention uses RFID technology for object recognition. By indexing and mapping item information, including object information, to RFID tags, object recognition is achieved, improving accuracy and preventing training errors. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0052] Figure 1 This is a flow chart of the inspection and safety training support method based on augmented reality and risk analysis provided by the present invention.
[0053] Figure 2 This is a structural block diagram of the inspection and safety training support system based on augmented reality and risk analysis provided by the present invention. DETAILED DESCRIPTION
[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0055] The embodiment of the present invention discloses an inspection and safety training support method based on augmented reality and risk analysis, such as Figure 1 As shown, including:
[0056] Conduct risk analysis on production scenarios, formulate risk analysis prompts based on risk analysis methods and domain knowledge, design inspection processes, retain inspection content, and generate inspection records;
[0057] Conduct 3D modeling of production process equipment and construct AR scenes in an augmented reality environment. Add risk analysis prompts and inspection processes to the AR scenes to form AR projects. Combine the AR projects with RFID technology to complete preliminary visualization of risk analysis.
[0058] After preliminary visualization, image recognition and data analysis technology are used to identify appearance defects and hidden dangers of production process equipment and display them in real time on the AR interface.
[0059] Designing a human-computer interaction interface and completing a human-computer interaction function.
[0060] In this embodiment, the risk analysis method for the production scene is taken as an example, including but not limited to the following methods:
[0061] (1) HAZOP risk analysis: dividing nodes, selecting guide words, determining deviations, analyzing reasons, evaluating consequences, making suggestions and emergency measures;
[0062] Combining domain knowledge to obtain a HAZOP analysis result table: combining domain knowledge to judge the HAZOP risk analysis result to form a HAZOP analysis result table;
[0063] Developing risk analysis prompt content: combining the important equipment introduction, function and operation of the production process with the deviations, evaluated consequences, analyzed reasons, suggestions and emergency measures in the HAZOP analysis result table to make risk analysis prompt content.
[0064] Among them, the HAZOP analysis steps are divided into dividing nodes, selecting guide words, determining deviations, analyzing reasons, evaluating consequences, making suggestions and emergency measures, and specifically:
[0065] Dividing nodes: decomposing the process system into a plurality of "nodes" (such as a single device, a pipe section, a control loop), each node corresponding to an explicit process unit. List the key process parameters of the node: such as temperature (T), pressure (P), flow (F), liquid level (L), component (C), reaction rate, etc.
[0066] Selecting guide words: HAZOP analysis usually uses a set of standard guide words, such as "none", "too much", "too little", "part", "opposite", "other", etc. These guide words are used to modify the key process parameters to help identify deviations. If the process system has special properties, appropriate guide words can be added or adjusted.
[0067] After determining the nodes, the selection of guide words will affect the scientificity of the analysis result. As shown in Table 1, the common guide words and deviations are as follows:
[0068] Table 1
[0069]
[0070] Determining deviations: using standard guide words and key process parameters to generate "deviations" (guide words + parameters = deviations).
[0071] Analyzing reasons: analyzing possible reasons for deviations (such as pump failure, valve misoperation, sensor failure), which need to distinguish between "equipment failure", "human error" and "control failure".
[0072] Consequences: Risks that may be induced by the deviation (e.g. overpressure leading to equipment rupture, reaction runaway leading to fire, toxic substance leakage), taking into account safety, environmental, and operational efficiency impacts.
[0073] Recommendations and emergency measures: Propose improvements and emergency measures for the risks that may be induced.
[0074] Example analysis using a HAZOP analysis of a chemical process equipment unit.
[0075] 1. Node division
[0076] Analysis node: Motor drive system, including motor, frequency converter, speed sensor, coupling, and load equipment.
[0077] Key process parameters:
[0078] Speed (N): Spindle rotation speed, directly affecting equipment operation status.
[0079] Current (I): Reflects motor load condition, abnormal current may indicate mechanical failure.
[0080] Vibration (V): Abnormal speed may cause mechanical vibration to intensify.
[0081] Temperature (T): Bearing or winding overheating may be caused by abnormal speed.
[0082] 2. Guidewords and deviation generation
[0083] Generate possible deviations by combining guidewords and parameters:
[0084] None (NO) + Speed → Motor stall (no speed output).
[0085] High (HIGH) + Speed → Speed too high (exceeding safety range).
[0086] Low (LOW) + Speed → Speed insufficient (below set value).
[0087] Reverse (REVERSE) + Speed → Reverse rotation (wrong rotation direction).
[0088] Fluctuate (FLUCTUATE) + Speed → Unstable speed (periodic fluctuation).
[0089] 3. Cause analysis (using "speed too high" as an example)
[0090] Deviation: Speed too high (HIGH + N)
[0091] Potential cause classification:
[0092] Equipment Failure: Abnormal frequency output from the frequency converter, causing the motor to overspeed; distorted signal from the speed sensor, resulting in ineffective feedback; slip in the coupling, sudden reduction in load, and a sudden increase in speed.
[0093] Human Error: Incorrect parameter settings for the frequency converter (e.g., setting the upper limit frequency too high); failure to perform speed calibration after maintenance.
[0094] Control Failure: Malfunctioning of the PID control loop, unable to stabilize the speed; failure of the overspeed protection function (e.g., relay).
[0095] External Factors: Sudden loss of load (e.g., belt breakage), motor running at no load and accelerating.
[0096] 4. Consequence Assessment
[0097] Safety Risk: Mechanical overspeed may cause bearing overheating, rotor breakage, leading to equipment damage or flying debris injuring personnel.
[0098] Production Impact: Decreased processing precision (e.g., spindle overspeed affecting part quality); forced shutdown of production line, resulting in economic losses.
[0099] Environmental Risk: If the motor drives a pump or compressor, overspeed may cause seal failure and medium leakage.
[0100] 5. Recommendations and Emergency Measures
[0101] Engineering Improvement: Install independent overspeed protection devices (e.g., mechanical centrifugal switch or electronic overspeed relay) to forcibly cut off power when the speed exceeds the limit; regularly check the speed sensor and frequency converter output accuracy to ensure feedback and control match.
[0102] Management Optimization: Clearly define the speed limit setting process in the operation manual and set up permission management (e.g., password protection for parameter modification); add an overspeed alarm interlocking system to trigger audible and visual alarms and automatically cut off power.
[0103] Emergency Response: Train operators to recognize overspeed signs (abnormal noise, increased vibration) and execute emergency shutdown procedures.
[0104] Parameter Analysis Table for Spindle Speed, as shown in Table 2. A represents a minor impact safety accident with no personnel injury; B represents a minor impact health / safety accident that does not cause long-term health effects; C represents a moderate impact health / safety accident; and D represents a greater impact health / safety accident.
[0105] Table 2
[0106]
[0107] (2) FMEA risk analysis: determine the analysis scope, identify potential failure modes, analyze failure impacts, determine failure causes, evaluate existing control measures, calculate risk priority number (RPN), and develop improvement measures;
[0108] Combine domain knowledge to get FMEA analysis result table: combine domain knowledge to judge FMEA risk analysis results and form FMEA analysis result table;
[0109] Develop risk analysis prompt content: combine the introduction of important equipment in production process, function and operation with potential failure modes, failure causes, risk priority number and improvement measures in FMEA risk analysis table to develop risk analysis prompt content.
[0110] Among them, the FMEA risk analysis steps include: determining the analysis scope, identifying potential failure modes, analyzing failure impacts, determining failure causes, evaluating existing control measures, calculating risk priority number (RPN), and developing improvement measures, specifically:
[0111] Determine the analysis scope: clearly define the object to be analyzed (such as product, process or system), define its function boundary and key characteristics. This step needs to clarify the analysis purpose (design improvement, process optimization or risk assessment) and determine the detailed degree of analysis.
[0112] Identify potential failure modes: based on historical data of the determined analysis object, experience of similar products or theoretical analysis, list all possible failure modes (such as mechanical fracture, electrical short circuit, sealing leakage, software failure, etc.).
[0113] Analyze failure impact: evaluate the impact of each potential failure mode on the system, downstream process or end user, including loss of function, performance degradation, safety risk, etc. Usually use severity (Severity, S) 1-10 rating, the higher the score, the more serious the impact.
[0114] Determine the failure cause: for each failure mode, analyze its root cause (such as material defect, assembly error, environmental factor, design deficiency, etc.). Evaluate the frequency of occurrence (Occurrence, O) 1-10 points, the higher the frequency, the higher the score.
[0115] Evaluate existing control measures: check existing prevention or detection means (such as design review, process mistake-proofing, online detection, etc.), judge its detection ability (Detection, D) 1-10 points, the more difficult to detect, the higher the score.
[0116] Calculate risk priority number (RPN): quantify the risk (range 1-1000) by the formula RPN=SxOxD, the higher the RPN, the greater the risk. High RPN items are prioritized, but combined with S single high score items (such as safety critical failure), the potential failure mode is sorted by this value.
[0117] Develop improvement measures: for high-risk failure sorted by risk priority number, propose optimization schemes (such as design redundancy, process error-proofing, enhance detection frequency, etc.).
[0118] The application adopts the risk analysis method in the safety field to provide detailed guidance and analysis for operators or new employees, assist and support employees in skill training and safety inspection, help to improve the professional skills of employees, and reduce the probability and harm degree of risk occurrence.
[0119] In the embodiment, the inspection process is designed, the inspection content is retained, and the inspection record table is formed, including:
[0120] Design inspection process: collect process equipment layout diagram, clearly define the distribution, connection relationship and key area of the equipment; combine the results of risk analysis method, and refer to process guide to master the normal operation parameter range, operation requirement and maintenance matters needing attention of the equipment, so as to design safety inspection link and standard;
[0121] Retain inspection content: through AR assistance, the operation is carried out according to the inspection process, the picture of inspection operation is collected through AR mobile terminal, and the picture of inspection operation is transmitted to the background server for retention, in order to ensure the accuracy of data, the picture can be transmitted to the background for retention after being confirmed by the inspector;
[0122] Generate inspection record table: generate inspection record table based on inspection content in background server, which is convenient for accident tracing, forms inspection database, and the results obtained by risk analysis method can also be updated according to the inspection database to obtain the risk situation of the equipment, which is convenient for better supporting the inspection operation.
[0123] The application combines field knowledge, HAZOP analysis results and process guide to design the inspection program suitable for the target process, provides clear inspection steps for the inspector, avoids the omission of part of the inspection due to insufficient experience or human negligence, and improves the reliability of the process from the inspection design.
[0124] In the embodiment, the equipment in the production scene is three-dimensionally modeled and the AR scene is constructed in the augmented reality environment, specifically including:
[0125] Three-dimensional modeling of production equipment: use the basic geometric bodies provided by Unity3D (such as cubes, spheres, cylinders, etc.) to create simple three-dimensional models, or use third-party modeling software (such as Blender, Maya, etc.) to create complex models, and then import them into Unity3D, thereby enhancing the visualization of risk analysis results, and facilitating the labeling of some parts that are not easy to label or discover on the equipment body on the three-dimensional model;
[0126] Building an AR scene: complete the AR environment configuration in Unity3D, and drag and drop the imported or created three-dimensional models from the "Assets" folder into the Hierarchy panel to make them part of the scene;
[0127] Adding prompt content in the AR scene: right-click "GameObject" in the Hierarchy panel to create UI elements (such as Text, Panel, etc.), and import the inspection process and risk analysis prompt content into them;
[0128] More preferably, it can also include lightweight and effect optimization: reducing the complexity and file size of the model to ensure smooth operation on the AR device, adjusting the position, rotation and scaling of the model to make it suitable for display in the AR scene, and performing text layout and format design to make it easy to read in AR display.
[0129] The present application uses Unity3D to build three-dimensional models and AR scenes, and adds prompt content, enhancing the visualization of risk analysis, providing detailed guidance and analysis for operators or new employees, assisting and supporting employees in skill training and safety inspection, helping to improve the professional skills of employees and reduce the probability and harm of risk occurrence.
[0130] In this embodiment, the AR project is combined with RFID technology to complete the preliminary visualization of risk analysis, including:
[0131] Establishing a local database: storing the optimized AR project including three-dimensional models and risk analysis prompt content in the local database, and indexing the AR project;
[0132] Designing RFID tags: designing RFID tags for AR project data (including three-dimensional models, risk analysis prompt content, etc.);
[0133] Establishing a mapping: in the local database, establish a correspondence table between RFID tag ID and AR project data. When the RFID reader reads the tag ID, the corresponding project information is obtained by querying the relationship table.
[0134] Data transmission and analysis: The RFID reader transmits the read RFID tag data to the AR mobile terminal or the processing device connected thereto. After the AR application receives the data, it parses it according to the established protocol and format, and extracts the RFID tag ID.
[0135] Implement AR visualization: According to the parsed RFID tag ID, obtain the detailed information of the corresponding AR project data from the local database, such as 3D model data, text description, pictures, etc. Then, the AR application renders the project in the AR scene according to these information, and displays it to the user in the form of virtual objects, risk analysis prompt content, etc.
[0136] The present application realizes the fusion of RFID tags and AR project index by mapping them, enhances the visualization of risks and hidden dangers in the production process, and reduces the possibility of accidents caused by negligence or lack of professional knowledge in safety inspection. At the same time, the consequences caused by each deviation in the prompt content and the emergency suggestions are helpful to timely control the risk and help them focus on information when inspecting in complex equipment environment, improving the efficiency.
[0137] In this embodiment, based on industrial equipment sensors and image recognition for hidden danger judgment, after completing object recognition and AR project content mapping, hidden danger judgment is added to complement the above steps, avoiding human error leading to misjudgment risk. Specifically, it includes:
[0138] Image acquisition and comparison: Collect equipment images through the AR mobile terminal, and judge the differences from normal images through image recognition algorithms to judge equipment appearance defects;
[0139] Sensor data transmission and hidden danger judgment: Transmit the parameters of the device sensors (such as voltage, current, speed, flow, etc., different devices have different specific parameters) to the background server, and obtain the hidden danger points of the device through data analysis;
[0140] AR real-time display and marking: The appearance defects identified by image recognition and the hidden danger points obtained by data analysis are transmitted to the display interface of the AR mobile terminal for real-time display and marking of hidden danger points;
[0141] More preferably, it can also include AR video communication and expert support: Use the web real-time communication (WebRTC) interface provided by the mixed reality toolkit (MRTK) to realize AR video communication, and the remotely guided experts can provide guidance through video calls with the inspectors through the web end of the system.
[0142] The application judges process equipment hidden dangers through image recognition and data analysis, and combines the results obtained by the foregoing risk analysis, so that the hidden danger information provided in the safety inspection and training system has an explanation and prior knowledge based on the safety field, improves the explainability of the training, and facilitates learning of new inspectors.
[0143] In the embodiment, a man-machine interaction interface is designed, and sensors are used to complete click interaction, drag interaction, zoom interaction and rotation interaction and other operations to complete man-machine interaction functions, including:
[0144] Man-machine interaction interface design: the man-machine interaction interface includes device name, three-dimensional model, inspection process and designed risk analysis prompt content, under the premise of ensuring the above content, interface beautification is performed to ensure the experience of the operator;
[0145] Interaction function implementation: design rules, i.e., action and posture modes, process and fuse data in the data glove that can reflect the features of hand action and posture, extract features that can reflect hand action and posture from the data, and match the features with the pre-defined action and posture modes.
[0146] The application enhances the visualization of risks and hidden dangers in the production process by designing an AR interaction interface and interaction function, certain man-machine interaction can help the operator better understand and master the inspection knowledge, improve the professional skill, and help to control the risk in time.
[0147] In summary, the application uses the analysis results of the risk analysis method to design the inspection process and the risk analysis prompt content, and imports them into the AR environment; inspection records in the inspection process are formed into an inspection record table, risk tracing is facilitated, object recognition is performed in the inspection process through RFID technology to confirm the risk analysis prompt content of the equipment in the inspection, image comparison and data analysis technology are used to assist the appearance and internal hidden dangers of the equipment, human errors are avoided, and the MRTK development kit WebRTC is added to realize AR video communication and guide hidden danger control. Finally, the AR interaction interface is designed to realize the AR interaction function.
[0148] The embodiment provides a kind of inspection and safety training support system based on augmented reality and risk analysis, as shown in Figure 2 It includes:
[0149] Risk analysis and inspection design module: for risk analysis to production scene, and based on risk analysis method fusion field knowledge to formulate risk analysis prompt content, design inspection process, retain inspection content and generate inspection record table;
[0150] AR visualization module: used for three-dimensional modeling and constructing AR scene of production process equipment under augmented reality environment, adding risk analysis prompt content in AR scene, forming AR project with inspection process, combining AR project with RFID technology, and completing preliminary visualization of risk analysis;
[0151] Hidden danger judgment module: used for judging appearance defects and hidden dangers of production process equipment based on image recognition and data analysis technology after preliminary visualization, and displaying in AR interface in real time;
[0152] Interaction module: used for designing man-machine interaction interface and completing man-machine interaction function.
[0153] The specific implementation process and method in the system are consistent, and will not be repeated here. Please refer to the method part.
[0154] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.
[0155] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. Inspection and safety training support method based on augmented reality and risk analysis, characterized by: include: Conduct risk analysis on production scenarios, formulate risk analysis prompts based on risk analysis methods and domain knowledge, design inspection processes, retain inspection content, and generate inspection records; Conduct 3D modeling of production process equipment and construct AR scenes in an augmented reality environment. Add risk analysis prompts and inspection processes to the AR scenes to form AR projects. Combine the AR projects with RFID technology to complete preliminary visualization of risk analysis. After preliminary visualization, image recognition and data analysis technology are used to identify appearance defects and hidden dangers of production process equipment and display them in real time on the AR interface. Design the human-computer interaction interface and complete the human-computer interaction functions.
2. The inspection and safety training support method based on augmented reality and risk analysis according to claim 1 is characterized in that: Risk analysis methods include HAZOP risk analysis and FMEA risk analysis.
3. The inspection and safety training support method based on augmented reality and risk analysis according to claim 2 is characterized in that: HAZOP risk analysis includes: Conduct HAZOP risk analysis, including: dividing nodes, selecting guide words, identifying deviations, analyzing causes, evaluating consequences, and making recommendations and emergency measures; Evaluate the HAZOP risk analysis results based on domain knowledge and form a HAZOP analysis results table; Combine the introduction, function and operation of production process equipment with the deviations, consequences, causes, suggestions and emergency measures in the HAZOP analysis results table to formulate risk analysis prompts.
4. The inspection and safety training support method based on augmented reality and risk analysis according to claim 2 is characterized in that: FMEA risk analysis includes: Conduct FMEA risk analysis, including: determining the scope of analysis, identifying potential failure modes, analyzing failure impacts, determining failure causes, evaluating current control measures, calculating risk priority numbers, and developing improvement measures; Evaluate the FMEA risk analysis results based on domain knowledge and form a FMEA analysis result table; Combine the introduction, function and operation of production process equipment with the potential failure modes, failure causes, risk priority numbers and improvement measures in the FMEA analysis results table to formulate risk analysis prompts.
5. The inspection and safety training support method based on augmented reality and risk analysis according to claim 1 is characterized in that: Design inspection process, retain inspection content and generate inspection record sheets, including: Design inspection process: Combine domain knowledge, risk analysis results, and process guidelines to design inspection steps and standards; Preserve inspection content: Based on augmented reality and according to the inspection process, the inspection operation pictures are collected through AR mobile terminals and transmitted to the backend server for preservation; Generate inspection record table: Generate inspection record table on the background server based on the inspection content to form an inspection database.
6. The inspection and safety training support method based on augmented reality and risk analysis according to claim 1 is characterized in that: In the augmented reality environment, 3D modeling of equipment in production scenarios is performed and AR scenes are constructed. Risk analysis prompts and inspection processes are added to the AR scenes to form AR projects, including: Use Unity3D's built-in basic geometry or third-party modeling software to create a 3D model of the equipment in the production scene, and import it into Unity3D. Parameters of parts that are difficult to mark or find on the equipment itself can be marked on the 3D model for easier operation. Complete the AR scene configuration in Unity3D, drag and drop the created 3D model from the Assets folder to the Hierarchy panel to make it part of the AR scene; Create UI elements in the Hierarchy panel and add risk analysis prompts and inspection processes to the AR scene to form an AR project.
7. The inspection and safety training support method based on augmented reality and risk analysis according to claim 6 is characterized in that: Combining AR projects with RFID technology includes: Save the AR project to the local database and index the AR project; Design RFID tags for AR projects; In the local database, a correspondence table between RFID tag ID and AR project index number is established; Transmit the read RFID tag data to the AR mobile terminal and extract the RFID tag ID; According to the parsed RFID tag ID, the detailed information of the corresponding AR project is obtained from the local database.
8. The inspection and safety training support method based on augmented reality and risk analysis according to claim 1 is characterized in that: Based on image recognition and data analysis technology, the appearance defects and hidden dangers of process equipment are judged and displayed in real time on the AR interface, including: Capture device images through AR mobile terminals, use image recognition algorithms to determine the difference from normal images, and identify device appearance defects; Transmit the parameters of the equipment sensors to the backend server, and analyze the data to find out the potential dangers of the equipment; The appearance defects identified by image recognition and the potential danger points from data analysis results are transmitted to the display interface of the AR mobile terminal for real-time display and marking of potential danger points.
9. The inspection and safety training support method based on augmented reality and risk analysis according to claim 1, characterized in that: Design the human-computer interaction interface and complete the human-computer interaction functions, including: Design the human-computer interaction interface, which includes the equipment name, 3D model, inspection process, and risk analysis prompts; Interaction rules, namely action and posture patterns, are designed on the human-computer interaction interface. The data in the data glove that can reflect the characteristics of hand movements and postures are processed and integrated, and the characteristics that can reflect hand movements and postures are extracted and matched with pre-defined action and posture patterns.
10. Inspection and safety training support system based on augmented reality and risk analysis, characterized by: include: Risk Analysis and Inspection Design Module: This module is used to conduct risk analysis on production scenarios, formulate risk analysis prompts based on risk analysis methods and domain knowledge, design inspection processes, retain inspection content, and generate inspection records. AR visualization module: This module is used to perform 3D modeling of production process equipment and construct AR scenes in an augmented reality environment. Risk analysis prompts and inspection processes are added to the AR scenes to form AR projects. The AR projects are then combined with RFID technology to complete preliminary visualization of risk analysis. Hidden danger judgment module: After preliminary visualization, it uses image recognition and data analysis technology to judge the appearance defects and hidden dangers of production process equipment and display them in real time on the AR interface; Interaction module: used to design the human-computer interaction interface and complete the human-computer interaction function.