An augmented reality based engineering technology training system

Abstract

The application provides an engineering technology training system based on augmented reality, which comprises a central control system, an overall framework with an operation position, a simulation module and an auxiliary system; the simulation module comprises simulated physical operation equipment and monitoring and interactive equipment; the monitoring and interactive equipment comprises a camera and augmented reality glasses; the simulated physical operation equipment comprises a lifting conveyor belt, a cutting model of a grinding machine and a wired touch tool arranged on the overall framework; an operator at the operation position sees virtual simulated engineering materials through the augmented reality glasses, operates through physical tools, sees the operation effect and timely adjusts the operation; the projection effect of a projection device is synchronized with the effect in the augmented reality glasses, the operation result of the operator is presented, and reference and basis are provided for other onlookers; a broadcast sound is used for playing prompt sounds and performing human-computer interaction sounds.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of augmented reality, and particularly relates to an engineering technology training system based on augmented reality. BACKGROUND

[0002] In recent years, China has attached great importance to the cultivation of engineering technology practical operation ability, and various colleges and universities have actively participated in the cultivation and training of skilled personnel and actively organized students to participate in engineering technology related competitions, such as masonry workers, decoration workers and other competitions. For a long time, they have occupied an important position in the national and world competitions and have been highly valued by various colleges and universities.

[0003] From the perspective of teaching requirements, the knowledge content of masonry plastering, steel bar processing and binding, wood processing and wall and roof decoration in engineering construction technology all belong to difficult and important content. Therefore, as a knowledge that must be learned and learned well, the combination of theory and practice is of great significance.

[0004] From the analysis of related content requirements, engineering technology professional ability training, safety education, emergency problem disposal, competition training and pre-service training have high specific operation content requirements. Therefore, real training needs to invest a lot of resources, including but not limited to cement, sandstone, bricks, brick cutting grinding machines, spades, line hangers, levels, trowels, rollers, paints, brushes and other tools; and professional personnel guidance is required, and the working environment requirements are also relatively high, which needs to consider weather, transportation, site size, cleaning convenience and many other requirements.

[0005] From the analysis of education resources conditions, masonry, especially cement with a shelf life of only three months, standard block loading and unloading easily damaged, mixing water transportation, tool wear and tear, dust pollution on physical health, masonry products cannot be reused, directly become construction waste, and the volume and quality are very huge, loading and unloading transportation difficulty is similar to relocation. A school trained 6-8 masonry workers to participate in the vocational skill competition, and created more than 20 cubic meters of construction waste in a month. For example, the decoration workers, wall tile laying, paint brushing and other operation training, need to consider the fragility of materials, material odor, formaldehyde pollution and other serious problems. Woodworking needs to consider wood processing, electric saw and planer operation danger, and steel worker needs material processing, but such as steel cutting machine, straightening machine, etc. The cost is high, and the steel reinforcement needs a larger space and has the requirements of waterproof and rust prevention.

[0006] From a job requirement perspective, although plastering robots have emerged, their efficiency is limited by usage costs, especially operating conditions. They are typically used for simple, continuous wall work on large, uncomplicated surfaces, such as shear walls. Their efficiency drops significantly when encountering corners, door and window openings, etc. Furthermore, efficiency is also affected by factors such as site leveling, working environment, power supply, and mechanical malfunctions. Upgrading the technology and improving the equipment's artificial intelligence level would drastically increase costs, significantly impacting construction costs. Currently, high-tech equipment for related engineering technologies is relatively rare and can only solve relatively simple problems. Therefore, manual labor for masonry, plastering, steel and wood processing, and decoration remains the primary demand. Even with the addition of prefabricated buildings, the need for human assistance and manual operation at complex nodes still requires consideration.

[0007] From the perspective of job management and accident prevention, this approach helps operators clearly understand that accidents occur during actual work processes, not during specially prepared drills. As the saying goes, prevention is better than cure; safety and risk control need to start with beginners. Augmented reality simulations of safety issues encountered in actual operations, such as accidentally falling bricks hitting feet, personnel misoperation causing tools to fall from the air and hit people, or people suddenly losing their footing, constantly remind operators of the importance of safety risk control.

[0008] In summary, existing education, vocational skills training, and safety emergency drills in masonry, plastering, and wall construction suffer from problems such as high material costs, dust pollution, large space requirements for materials, and susceptibility to weather conditions. Therefore, it is necessary to provide an augmented reality-based engineering technology training system to address these issues.

[0009] A search revealed no publicly available technical solutions similar to this invention. Summary of the Invention

[0010] The present invention aims to solve the technical problems existing in the prior art, and the purpose of the present invention is to provide an engineering technology training system based on augmented reality.

[0011] In order to achieve the above object, the application adopts the following technical scheme: an engineering technology training system based on augmented reality, comprising a central control system, an overall framework with an operation position, a simulation module and an auxiliary system, the simulation module and the auxiliary system being connected with the central control system respectively; the simulation module comprises simulated physical operation equipment and monitoring and interactive equipment; the monitoring and interactive equipment comprises a camera for obtaining the operation process of an operator and augmented reality glasses for augmented reality performance; the simulated physical operation equipment comprises a conveying belt arranged on the overall framework, a grinding machine cutting model arranged on the overall framework, and a wired touch tool for the operator to perform engineering technology operation, the conveying belt being a lifting type conveying belt, the conveying belt serving as a workbench, the operator at the operation position wearing the augmented reality glasses, picking up the touch tool and operating the virtual simulated engineering material on the conveying belt to simulate the engineering technology operation, the operator being able to simulate cutting work at the grinding machine cutting model through the virtual simulated engineering material, the surface of the conveying belt being provided with a sensor for conducting the pressing effect of the touch tool, and the conveying belt being able to be lifted to simulate the effect of continuously increasing masonry brick walls; the central control system comprises a central control computer, a display and a projection device; the camera and the sensor transmit information to the central control computer of the central control system, and the engineering technology operation picture is presented on the augmented reality glasses and the projection screen of the projection device, and the operator adjusts the operation according to the picture in the augmented reality glasses; the auxiliary system comprises a broadcast speaker connected with the central control system for emitting sound.

[0012] The above technical scheme, the overall framework constitutes the external contour of the training system, is used for mounting the central control system, the simulation module and the auxiliary system, and provides an operation space for the operator; the touch tool is an entity tool used for actual building construction work, the central control system adjusts the height of the conveying belt in combination with the operation information collected by the camera and the sensor information, and after the conveying belt is lifted, the conveying belt can be used as an auxiliary prop for augmented reality simulation to realize masonry, steel cutting, woodworking and other operations; the operator sees the virtual simulated engineering material through the augmented reality glasses and sees the effect of his own operation through the entity tool and timely operation adjustment; the projection effect of the projection device is synchronous with the effect in the augmented reality glasses, and the operation result of the operator is presented to provide a reference and basis for other onlookers; the broadcast speaker is used for playing prompt sound and human-computer interaction sound.

[0013] In a preferred embodiment of the application, the touch tool comprises a plurality of tool models hung on an augmented reality hanger, the augmented reality hanger is an annular hanger, the tool models are hung on the annular hanger through telescopic ropes and can rotate circumferentially on the annular hanger; the tool models are provided with touch sensors at the ends, the conveying belt and the grinding machine cutting model are auxiliary props, and the auxiliary props are provided with touch pieces capable of interacting with the touch sensors to generate touch feeling.

[0014] The technical scheme, the tool models are hung on the augmented reality hanging frame through the telescopic hanging ropes, so that the operator can pick up various tool models, the augmented reality effect is assisted, and simulation is maximally realized; the end of the tool model is provided with a touch sensor, the tool is provided with a touch piece, the touch of the tool model is simulated, when the tool model touches the designated area such as the conveying belt and the abrasive wheel cutting model, the stress feeling is simulated and information is fed back.

[0015] In a preferred embodiment of the present application, a telescopic frame is arranged in the middle of the augmented reality hanging frame, telescopic top plates capable of being pulled horizontally are arranged on the left and right sides of the telescopic frame, and a downward pulling plate capable of being pulled vertically downward is arranged at the lower end of the telescopic frame, the telescopic top plates and the downward pulling plate are used to assist virtual plastering and wall painting.

[0016] The telescopic top plates and the downward pulling plate are retracted into the telescopic frame when not in use, and are pulled out of the telescopic frame when in use, and are used to assist the augmented reality effect of virtual plastering and wall painting.

[0017] In a preferred embodiment of the present application, a plurality of falling blocks for simulating high-altitude falling objects are hung on the augmented reality hanging frame, the falling blocks are falling line type falling blocks hung by falling lines and capable of falling and rising.

[0018] The falling blocks are used to simulate high-altitude falling objects, the falling blocks fall randomly according to the operation of the operator during the operation, when the tool model in the hand of the operator is dropped, the falling line of the upper falling block is loosened and falls randomly, which is used to simulate high-altitude falling objects and also reminds the operator of the damage of the falling tool or material.

[0019] In a preferred embodiment of the present application, the training system further comprises a lifting foot pedal for simulating the operation feeling of high-altitude construction, a pedal groove is arranged at the operation position of the overall frame, a plurality of groups of lifting springs are arranged in the pedal groove, the lower end of the lifting foot pedal is connected with and supported by the upper end of the lifting springs, and the lifting foot pedal can be lifted and swung left and right by telescoping the lifting springs.

[0020] The lifting foot pedal is lifted by the lifting springs, the augmented reality simulation of high-altitude construction such as masonry, scaffold erection and formwork installation can be enhanced, the lifting foot pedal can swing left and right, the feeling of shaking under the feet can be created, and the safety problems in the operation process are simulated to give the operator a close-to-real experience.

[0021] In another preferred embodiment of the present application, each group of lifting springs comprises a spring and a jack in the spring, the upper end of the spring is fixedly connected with the lower end of the lifting foot pedal, the jack is in contact with or not in contact with the lower end of the lifting foot pedal, and the spring can be telescoped to make the lifting spring telescoped by lifting the jack.

[0022] The above technical scheme, the spring and the jack constitute a lifting spring, the upper end of the spring is fixedly connected with the lower end of the lifting foot pedal, the jack can make the lifting foot pedal lift when the jack is in contact with the lower end of the lifting foot pedal, the spring is slightly higher than the jack when the jack is not in contact with the lower end of the lifting foot pedal, and the lifting foot pedal can swing left and right to generate a shaking feeling under the feet.

[0023] In another preferred embodiment of the present application, the conveying belt is installed in a support frame fixed to the overall frame, a lifting mechanism is installed on the support frame to drive the conveying belt to lift in the support frame, and a plurality of cameras are installed on the top of the support frame.

[0024] The above technical scheme uses the support of the support frame to assist in the operation of augmented reality, and a plurality of cameras are installed on the top of the support frame to monitor all operations in the entire site without dead angles.

[0025] In another preferred embodiment of the present application, the inner side of the support frame is provided with a vertically extending lifting rail, the conveying belt has a sliding column clamped in the lifting rail and sliding in the rail, and the lifting rail is provided with upper and lower barriers limiting the upward and downward movement range of the conveying belt.

[0026] The above technical scheme guides the lifting movement of the conveying belt by the lifting rail and the sliding column, so that the lifting of the conveying belt is more stable, and the upper and lower barriers are arranged to limit the upward and downward movement range of the conveying belt, so that the conveying belt can be prevented from being separated from the support frame.

[0027] In another preferred embodiment of the present application, the overall frame is provided with a downward recessed lower groove, the conveying belt can be embedded into the lower groove, and the upper surface of the conveying belt can be flush with the top of the lower groove, and a plurality of auxiliary bricks are placed in the lower groove below the conveying belt, and the auxiliary bricks can slide into the lower groove along with the operation of the conveying belt.

[0028] The above technical scheme embeds the conveying belt into the lower groove and flushes the top of the conveying belt with the surrounding, so that the laying of the ground, the paving of the tiles at a low position, and the masonry construction can be simulated, the lower groove serves as a space for storing the auxiliary bricks, the auxiliary bricks are arranged as the bricks or the auxiliary props simulating the masonry process, and are placed on the conveying belt, and the masonry action is simulated by the touch tool, so that the real feeling is stronger.

[0029] In another preferred embodiment of the present application, the overall frame is composed of a metal framework and a plurality of cover plates covering the metal framework, the overall frame has a high area located in the middle and a low area located around the high area, the operation position, the conveying belt and the touch tool are arranged in the high area, and the abrasive cutting model, the central control computer, the display and the projector are arranged in the low area.

[0030] The technical scheme has the advantages that the whole frame is divided into a high area and a low area, the high area is the main operation area of the operator, and other personnel can operate the central control computer in the low area without interfering with each other.

[0031] Compared with the prior art, the preferred technical scheme of the present application has the following beneficial effects: the present application is a man-machine interaction simulation teaching system capable of enhancing reality, through enhanced reality operation, especially in operation, specific safety warning and safety accident virtual scene are set, so that participants can experience the occurrence of accidents in situ and without harm, and according to specific operation content and the reaction of the operator, the operator is exercised, emergency problems are not avoided, and risks and disasters are bravely faced.

[0032] Additional aspects and advantages of the present application will be described in part below, will become apparent from the following description, or will be learned by practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0034] Figure 1 is a component block diagram of the training system of the present application.

[0035] Figure 2 is a structural schematic diagram of the training system of the present application.

[0036] Figure 3 is a structural schematic diagram of the whole frame in the present application.

[0037] Figure 4 is a structural schematic diagram of the metal skeleton in the present application.

[0038] Figure 5 is a structural schematic diagram of the lifting foot pedal in the present application.

[0039] Figure 6 is a schematic diagram of the lifting conveyor belt in the present application at the highest point.

[0040] Figure 7 is a schematic diagram of the lifting conveyor belt in the present application at the lowest point.

[0041] Figure 8 is a running schematic diagram of the auxiliary brick in the present application.

[0042] Figure 9 is a schematic diagram of the tool model hung on the augmented reality hanging rack in the present application.

[0043] Figure 10 is a schematic diagram of the touch sensor and the touch piece respectively arranged on the tool model and the auxiliary prop.

[0044] Figure 11 is the structure diagram of the extension frame in the present application.

[0045] Figure 12 is the diagram of the falling block in the present application.

[0046] Figure 13 is the diagram of the projector and the projection effect in the present application.

[0047] Figure 14 is the diagram of the operation synchronous projection in the present application.

[0048] Figure 15 is the effect diagram displayed by using the augmented reality glasses in the present application.

[0049] Figure 16 is the augmented reality operation effect diagram in the present application.

[0050] Figure 17 is the block diagram of the overall structure and operation relationship in the present application.

[0051] The reference signs in the description accompanying drawings include: augmented reality hanger 1, extension frame 2, camera 3, hanger support column 4, conveying belt 5, support column 6, baffle 7, high area 8, central control computer 9, computer desk 10, matching equipment slot 11, low area 12, abrasive wheel machine cutting model 13, abrasive wheel chassis 14, equipment expansion slot 15, lower slot 16, lifting rail 17, cover plate 18, metal framework 19, distribution box 20, fire extinguisher 21, lifting foot pedal 22, spring 23, pedal slot 24, jack 25, overall frame 26, auxiliary brick 27, extension hanging rope 28, touch tool (tool model) 29, telescopic top plate 30, lower extension plate 31, falling block 32, projection curtain 33, projection effect 34, projector 35, augmented reality glasses 36, augmented reality visual effect 37, augmented reality operation effect 38, touch sensor (electromagnet) 39, touch piece (ordinary magnet) 40. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described in detail below, and the examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary, only for explaining the present application, and cannot be understood as the limitation of the present application.

[0053] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "vertical", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0054] In the description of the present application, unless otherwise specified and limited, it should be noted that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication between two elements, it can be direct connection, or indirect connection through intermediate medium, and those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0055] The present application provides an engineering technology training system based on augmented reality, as shown in Figure 1 and Figure 2 In a preferred embodiment, the training system comprises a central control system, an overall framework 26 with an operating position, a simulation module and an auxiliary system, and the simulation module and the auxiliary system are connected with the central control system respectively.

[0056] The simulation module comprises simulation physical operation equipment, monitoring and interactive equipment.

[0057] As shown in Figure 2 and Figure 14 Among them, the interactive equipment comprises a camera 3 for obtaining the operation process of the operator and augmented reality glasses 36 for augmented reality performance, and the augmented reality glasses 36 can adopt the commonly used augmented reality AR glasses on the market or separately designed glasses.

[0058] As shown in Figure 1 and Figure 2As shown in the figure, the simulation physical operation device includes a conveyor belt 5 arranged on the overall frame 26, a grinder cutting model 13 arranged in the low area 12 of the overall frame 26, and a wired touch tool 29 for the operator to perform engineering technical operation, which is arranged on the overall frame 26. The conveyor belt 5 is a lifting conveyor belt 5, which serves as a workbench. The operator at the operation position wears an augmented reality glasses 36, picks up the touch tool 29 (physical tool), and performs operation on the virtual simulation engineering material (the simulation engineering material is displayed in the picture of the augmented reality glasses 36 through virtual technology, mainly including reinforcing steel bars, wood, bricks, pipes, welding materials, etc.) on the conveyor belt 5 to simulate engineering technical operation. The operator can simulate cutting operation at the grinder cutting model 13 through the virtual simulation engineering material. The surface of the conveyor belt 5 is provided with a sensor (not shown in the figure) for conducting the pressing effect of the touch tool 29. The conveyor belt 5 can be lifted to simulate the effect of continuously increasing the brick wall.

[0059] As shown in the figure, Figure 2 and Figure 12 The central control system includes a central control computer 9, a display, and a projection device. The central control computer 9 includes a host computer, a keyboard, and a mouse. The host computer is connected to the display and the projection device. The central control computer 9 is built-in with an overall operation system and a regional connection system. The central control computer 9 is an overall control device, which is connected to all devices and implements overall and real-time control on the operation process of the combination of software and hardware through the control system. The projection device includes a projector 35 and a projection screen 33. The projector 35 can be selected from common school classroom projector 35 models to ensure connection with the central control computer 9. The projection screen 33 is selected from common projection screen 33 materials with a width of 1.5 to 2 meters and a height of 1.5 to 2 meters. The display serves as a tool for normal operation display and information input in the central area. The central control computer 9, the display, and the projector 35 are placed on a computer desk 10. The camera 3 and the sensor transmit information to the central control computer 9 of the central control system. The picture of engineering technical operation is presented on the augmented reality glasses 36 and the projection screen 33 of the projection device. The operator adjusts the operation according to the picture on the projection screen 33.

[0060] As shown in the figure, Figure 1 and Figure 2 The auxiliary system includes a broadcast speaker connected to the central control system for emitting sound, which is used for playing prompt sound and conducting human-computer interaction sound. The auxiliary system further includes a distribution box 20 for power supply and a fire extinguisher 21 as a fire-fighting facility. The fire extinguisher 21 is a carbon dioxide type fire extinguisher 21. The broadcast speaker and the total switch of the training system are arranged on the distribution box 20.

[0061] When using the training system, the operator wears the augmented reality glasses 36, uses the physical touch tool 29, and makes the specified action. The touch tool 29 is connected with a wired sensor to collect the specific action parameters in real time, transmit them to the central control system, analyze and process the action effect through the virtual simulation system, form a real-time picture, and convert the operation effect into a visual display. At the same time, the operator sees the effect of his own operation through the augmented reality glasses 36, adjusts the operation in time, and makes the digital model and the touch tool 29 consistent in action and effect to achieve the teaching purpose. Before teaching, the information such as gestures, bending angle, and tool using method can be shot and input into the system as a standard for comparison during the training process. At the same time, the total control platform of the operation interaction system can be connected with the cloud computing system to send the running information of each device of the training system to the cloud application system, so as to grasp the effect as a whole and detect the entity connection, fault elimination, and virtual operation in real time.

[0062] As shown in Figures 2-4 , the overall frame 26 is composed of a metal skeleton 19 and a plurality of cover plates 18 covering the metal skeleton 19. The overall frame 26 has a high area 8 in the middle and a low area 12 located around the high area 8. The high area 8 is 0.25-0.5 meters high and 3-4 meters wide. The operation position, the conveyor belt 5, and the touch tool 29 are located in the high area 8, which is the main operation area for the operator. The low area 12 is about 0.1 meters higher than the indoor floor, which is the basic platform for the abrasive wheel cutting model 13 and the operation of the central control computer 9. The metal skeleton 19 is assembled by a plurality of shaped steel through welding or bolt connection, such as 20×20×1.5 hollow square steel, or other shaped steel such as I-beam and channel steel. Welding or steel bolt connection can be used to ensure the structural strength and stability, especially considering the vibration and impact of mechanical equipment and personnel walking. The metal skeleton 19 is covered with cover plates 18, which can be made of metal, hard plastic, or composite wood, with a thickness of 3-5 cm. Welding or bolt connection is used, and protective measures such as epoxy resin coating can be added on the upper part of the cover plate 18 to consider the stress and wear resistance. The left and right sides and the back of the overall frame 26 are provided with three side guards 7, which are 0.5-1 meters high and used to enclose the work in the overall frame 26 to ensure the operation environment. The side guards 7 can be made of plastic or metal.

[0063] As shown in Figure 2 , the computer desk 10 is installed in the low area 12 and is 0.5-1 meters high. The material is transparent acrylic plastic plate, which is used to place the central control computer 9, display, and projector 35. The shape of the desktop can be designed as a circle or a square. The lower part of the computer desk 10 is provided with a matching equipment slot 11 for matching and adding related servers, central control computer 9 host, and other equipment.

[0064] like Figure 2 As shown, the grinding wheel cutting model 13 is a machine model made of acrylic plastic sheet. It is set at a height of about 1 meter, which is suitable for the operator to operate from the operating position in the high area 8, and is used for augmented reality operation. The grinding wheel cutting model 13 is installed on the grinding wheel chassis 14 in the low area 12. It is a circular platform and it is strictly forbidden to place any objects on it. The equipment expansion slot 15 is located below the grinding wheel chassis 14. Its shape and position can serve as a carrier for augmented reality. That is, through augmented reality glasses 36, one can observe the position and operation of buttons and other controls below the grinding wheel cutting model 13.

[0065] like Figure 5 As shown, the training system also includes a lifting foot pedal 22 for simulating the feeling of working at height. A foot pedal groove 24 is provided at the operating position of the overall frame 26. For example, one foot pedal groove 24 is provided on each side of the conveyor belt 5 in the high zone 8. Several sets of lifting springs 23 are provided in the foot pedal groove 24. The lower end of the lifting foot pedal 22 is connected to and supported by the upper end of the lifting springs 23. The lifting springs 23 are responsible for raising and lowering the lifting foot pedal 22, with a maximum load capacity of 250 kg. By extending and retracting several lifting springs 23, the lifting foot pedal 22 can be raised, lowered, and swayed left and right. Initially, the lifting foot pedal 22 is located at the top of the foot pedal groove 24, sealing it. Preferably, two sets of lifting springs 23 are provided in the middle and on each of the left and right sides of the bottom of the lifting foot pedal 22, for a total of six sets of lifting springs 23. Each set of lifting springs 23 includes a columnar spring 23 and a jack 25 located within the spring 23. The jack 25 is connected to an electromagnetic switch to control its operation. The lower end of spring 23 is fixedly connected to the bottom of pedal groove 24, and the upper end of spring 23 is fixedly connected to the lower end of lifting foot pedal 22. The upper end of jack 25 is not connected to the lower end of lifting foot pedal 22; the two may or may not be in contact. By raising and lowering jack 25, spring 23 can extend and retract, causing lifting spring 23 to rise and lower, and lifting foot pedal 22 to rise and lower. By extending and retracting several lifting springs 23, lifting foot pedal 22 can rise and fall and swing left and right.

[0066] The operator's foot steps on the designated area of the lifting footboard 22, which is used to simulate the lifting footboard of the scaffold, the feeling of the operation of the high-altitude construction, and the range of the operator's foot stepping exceeds the designated area. All the lifting springs 23 are synchronously shortened, and the lifting footboard 22 will be lowered within 10 cm to play a warning role, and the safety management effect in the actual combat is highlighted. When the middle two groups of lifting springs 23 are lifted by the jacks 25 by 0.1 meters, the jacks 25 on the left and right sides are not in action, so that the springs 23 on the left and right sides are slightly higher than the jacks 25 inside, and the lifting footboard 22 swings left and right, which can create a subtle shaking feeling under the foot. After the jacks 25 on the left and right sides are lowered, the length of the spring 23 extending outside the jack 25 increases, and the swing is strengthened to simulate the safety problem in the operation process and give the operator a real experience. After the experience is over, the operator directly jumps in place, the jacks 25 are retracted, the lifting springs 23 are pressed back to the original normal height, and the lifting footboard 22 returns to the original normal height.

[0067] As shown in Figure 2 , Figure 6 and Figure 7 , in the present application, the conveyor belt 5 is installed in the support 6 frame fixed to the overall frame 26, and a lifting mechanism for driving the conveyor belt 5 to lift in the support 6 frame is installed on the support 6 frame. The lifting mechanism is a prior art, and its structure and principle are not described here. Among them, the support 6 frame includes four supports 6 arranged at the four corners outside the conveyor belt 5. The inner side of each support 6 frame is provided with a vertically extending lifting rail 17. The support 6 is selected as a slot steel to ensure its reliability, firmness and durability. The height is 2-3 meters, and the bottom is connected to the metal skeleton 19. The conveyor belt 5 has a sliding column clamped in the lifting rail 17 and sliding in the rail. The lifting rail 17 is provided with upper and lower barriers limiting the upward and downward movement range of the conveyor belt 5. A group of cameras 3 are installed at the top of each support 6 for observing the inside and outside of the overall frame 26 without dead angle, assisting operation, and the cameras 3 are designed to be 2.5-3-5 meters high, rotatable, and connected to the central control computer 9 through a network cable.

[0068] Preferably, the high area 8 of the overall frame 26 is provided with a downward recessed lower groove 16 below the lifting conveyor belt 5, and the lifting rail 17 extends downward into the lower groove 16. When the conveyor belt 5 moves downward, it can be embedded into the lower groove 16, and the upper surface of the conveyor belt 5 can be flush with the top of the lower groove 16.

[0069] As shown in Figure 6 and Figure 7As shown, the surface of the height-adjustable conveying belt 5 is provided with sensors for conducting the pressing effect of the touch tool 29. Initially, the conveying belt 5 is located at the lowest point, and the top of the conveying belt 5 is flush with the surroundings, at which time the laying of a ground wire, the laying of tiles at a low position, masonry, etc. can be simulated. The central control system adjusts the height of the conveying belt 5 in combination with the operation information collected by the camera 3 and the sensor information, and after the conveying belt 5 is lifted up, it can be used as an auxiliary prop for augmented reality simulation to realize masonry, reinforcement cutting, woodworking, etc. As the height of the conveying belt 5 is lifted up, the lifting footboard 22 is lifted up by the lifting spring 23 under the feet of the operator, which can enhance the simulation of high-altitude construction such as masonry, scaffold erection, formwork installation, etc., and the highest lifting height of the conveying belt 5 is set to 2.5 meters.

[0070] As shown, Figure 8 In another preferred embodiment, a plurality of auxiliary bricks 27 are also placed in the lower groove 16 below the conveying belt 5, the auxiliary bricks 27 can slide into the lower groove 16 as the conveying belt 5 runs, the auxiliary bricks 27 are also auxiliary props, and the surface thereof is also provided with a common magnet 40 capable of interacting with the electromagnet 39 at the end of the tool model 29. The size of the auxiliary brick 27 is the same as that of the actual use brick or standard brick, for example, 240mmx115mmx53mm, etc., and the material is plastic, and the weight is close to that of the real brick or brick. After the conveying belt 5 is lifted up, the operator takes out the auxiliary brick 27 from the lower groove 16, places the auxiliary brick 27 on the conveying belt 5 as a row of bricks or an auxiliary prop for simulating the masonry process according to the masonry requirements, simulates the masonry action by the wired touch tool 29, at which time the conveying belt 5 is paused, after the action is completed, a new auxiliary brick 27 is picked up from the lower groove 16 below the conveying belt 5, and the above-mentioned action is repeatedly performed, after a row of bricks is laid, the conveying belt 5 slowly runs to convey the auxiliary brick 27 to the left slide and drop into the lower groove 16. With the help of the information collection of the sensor and the camera 3, the masonry effect is presented on the augmented reality glasses 36, the display and the projection curtain 33.

[0071] In addition, the conveying belt 5 at the middle height position can also be used as an auxiliary prop for simulating the processing of pipe materials by a pipe worker, the welding of parts by a welder, the fine processing of wood by a woodworker, and the processing of reinforcement by a reinforcement worker, and in combination with the augmented reality glasses 36, the virtual operation is presented as a whole, and the specific augmented reality simulation mode is realized according to the design of the set action, the operation of the auxiliary prop and the scene requirements.

[0072] As shown, Figure 1 and Figure 9As shown, the touch tool 29 includes several tool models hung on the augmented reality hanger 1, which are real tools including but not limited to water basin, roller brush, shovel, trowel, chisel, line sinker, level and other tools used for actual construction work; the augmented reality hanger 1 is a ring-shaped hanger, and the tool model 29 is hung on the ring-shaped hanger through the telescopic hanging rope 28 and can rotate circumferentially on the ring-shaped hanger. Specifically, the augmented reality hanger 1 is designed as a circle, made of aluminum alloy or profile steel material, with a diameter of 2-3 meters, fixed on the top of the plurality of hanger support columns 4 fixedly connected with the high area 8 of the overall frame 26, and the hanger support columns 4 are made of profile steel and welded or threaded connected with the metal framework 19 of the high area 8. The augmented reality hanger 1 is provided with a ring-shaped hanging groove, and the upper end of the telescopic hanging rope 28 is fixed in and movable in the ring-shaped hanging groove, and the tool model 29 is fixed at the lower end of the telescopic hanging rope 28. The telescopic hanging rope 28 can adopt a plastic spring 23 (refer to the structure of a bicycle metal chain lock and a landline telephone), which can be stretched and lengthened, with a maximum length of 3-4 meters, covering all areas of the overall frame 26, to provide the operator with a real operation feeling.

[0073] The operator holds the tool model 29 with one hand or both hands, which is used to assist the operator to experience the real operation feeling, and can be set by a program to simulate the completion of masonry, steel processing, pipe processing, welding, decoration and other work.

[0074] As shown, Figure 10 The end of the tool model 29 is provided with a touch sensor 39, and the conveyor belt 5 and the cutting model 13 of the grinder are auxiliary props, which are provided with a touch piece 40 capable of interacting with the touch sensor 39 to generate a touch feeling. For example, the touch sensor 39 at the end of the tool model 29 is an electromagnet, and the telescopic hanging rope 28 is internally provided with a thin wire for supplying power to the electromagnet 39, and the touch piece 40 is a common magnet with opposite magnetic pole to the electromagnet 39. By setting the electromagnet 39 and the common magnet 40, the real touch feeling of the mortar pressing, mortar plastering, cutting or brush head and other tools is simulated by using the principle of magnetic repulsion between different levels; at the same time, a sensor is connected at the end of the telescopic hanging rope 28, and when the electromagnet 39 touches the common magnet 40 on the specified area of the conveyor belt 5, the cutting model 13 of the grinder, the telescopic roof 30 and the lower pull plate 31, the sensor returns information to the central control system.

[0075] As shown, Figure 11As shown, the middle of the augmented reality hanger 1 is provided with a telescopic frame 2, designed as a three-sided recess, respectively left and right and below. The left and right sides of the telescopic frame 2 are provided with a horizontally pullable telescopic top plate 30, made of plastic, pulled by a wire, completely unfolded, covering the top, as an auxiliary tool for simulating the top auxiliary simulation of the ceiling painting, and the surface is also provided with a common magnet 40 capable of interacting with the electromagnet 39 at the end of the tool model 29. The lower end of the telescopic frame 2 is provided with a vertically downward pullable lower pull plate 31, made of plastic, pulled by a wire, completely unfolded, which can fall to the ground, as an auxiliary tool for simulating the augmented reality construction of the wall surface, and the surface is also provided with a common magnet 40 capable of interacting with the electromagnet 39 at the end of the tool model 29.

[0076] As shown in Figure 12 As shown, the augmented reality hanger 1 is hung with a number of falling blocks 32 for simulating falling objects from high altitude, and the falling blocks 32 are falling blocks 32 suspended by a pull line and can fall and rise. The material of the falling block 32 is light, which is specially designed for simulating falling objects from high altitude. During the operation, the operator can receive the voice prompt of the broadcast speaker, and then the falling block 32 falls randomly around him, which may hit the operator's safety helmet. The falling block 32 falls randomly according to the operation, and when the tool model 29 in the operator's hand is dropped, the pull line of the upper falling block 32 is loosened and falls randomly, which is used to simulate falling objects from high altitude and also reminds the operator of the damage of falling tools or materials.

[0077] Figure 13 As shown in Figure 14 As shown, the projection effect 34 displayed on the projection screen 33 by the projector 35 is displayed as an aspect of the present application, and the projection effect 34 is synchronized with the effect in the augmented reality glasses 36, which is displayed by a special projection effect 34 to show the operation result of the operator, providing a reference for other onlookers.

[0078] Figure 15 As shown, the augmented reality visual effect 37 seen in the augmented reality glasses 36 is the real-time picture seen by the operator through the glasses, especially with the movement of the operator, to ensure that the scene seen in the augmented reality glasses 36 changes synchronously, but cannot change too violently.

[0079] Figure 16 As shown, the augmented reality operation effect 38 in the present application is the personnel operation action in the augmented reality presentation process, which is completed by various props (including the conveyor belt 5, the abrasive cutting model 13, the telescopic top plate 30, the lower pull plate 31, the auxiliary brick 27, the tool model 29 and the simulated engineering materials, etc.), combined with the augmented reality effect. The personnel can normally operate by holding the tool model 29 according to the scene seen in the augmented reality glasses 36 and the prompt sound emitted by the broadcast speaker.

[0080] Figure 17 The software and hardware of the entire training system are shown, which is mainly divided into four parts of overall framework 26, central control system, virtual simulation module and auxiliary system. Among them, the overall framework 26 is divided into a metal skeleton 19 and a cover plate 18, which constitutes the external contour of the training system, and the remaining three parts are a combination of software and hardware. The central control system is the core control function, which is mainly divided into three aspects, namely operation and running, blockchain storage and display function. The control system controls all the software and hardware; the blockchain storage sets the blockchain unit, divides different blocks according to the steps, and records the steps and processes accurately and effectively, ensures accuracy and does not allow tampering, especially the safety performance in each step, such as wearing a safety helmet, virtual high-altitude operation, and hanging a safety belt and other measures. The display function focuses on the effect of synchronous display, and requires the display, projector 35 and augmented reality glasses 36 to display different requirements of the same content. The virtual simulation module focuses on the combination of software and hardware operation and execution, and uses virtual auxiliary tools, especially wired touch tools 29, all equipped with electromagnets at the end to simulate tool touch, cooperate with augmented reality effect, and supervise the entire operation through the 360° rotation of the monitoring device camera 3. The auxiliary system mainly includes facilities for providing network, power supply, alarm and broadcasting for the entire system to ensure smooth and reliable operation.

[0081] The innovation of the central control system of the present application lies in the combination of augmented reality control, software and hardware cooperation and blockchain technology, which maximizes the restoration of the real scene and accurately records the operation process.

[0082] Among them, the first part of the central control system is operation and running, which is divided into three parts, 1. Sensing data processing module, which is designed to be divided into 1.1 part, the camera 3 shoots the whole operation lifting conveyor belt 5 support 6 top sets four groups of cameras 3, covering all areas, shooting in real time, and transmitting back the shooting information in real time; 1.2 physical model matching sensor information, which is used to realize the tool model 29 end rope connection sensor, and ensure that the tool model 29 operation a section of the connecting electromagnet, the electromagnet touches the designated area of the conveyor belt 5, the grinding machine cutting model 13, the telescopic top plate 30 and the lower pull plate 31, and the sensor returns the information; 2. Core control module is divided into 2.1 central control system execution process and 2.2 decision logic analysis, the specific design function is described below; 3. Three-dimensional registration and tracking requires real-time comparison of virtual objects and real models and real-time tracking of operation effect to ensure alignment.

[0083] The second part of the central control system is the blockchain storage, which is mainly designed as 4. Blockchain integration module, including 4.1 Blockchain network layer, which aims to realize the construction of blockchain network and ensure node communication and data synchronization, 4.2 Data security and privacy protection, ensure that learning performance is not tamperable, and the process of safety production awareness cultivation is not tamperable. The completion and storage process of the blockchain, need to ensure that each block represents each step and the safety operation requirements, and truly record the entire operation process and completion of the operator.

[0084] The third part of the central control system is the display function, including: 5. Augmented reality generation and rendering module and 6. User interface, wherein 5. Augmented reality generation and rendering module includes 5.1 Combining operation effect, generating virtual image and 5.2 Generating virtual information prompt to be put into augmented reality glasses 36 and projection curtain 33 display two parts, wherein the design requirements of 5.2 part realize superposition to reality through augmented reality glasses 36 display; The design of 6. User interface requires to contain interface element content, accept and send information position, ensure volume adjustment, especially the design of error and warning.

[0085] Specifically, the core control module of the first part of the central control system includes 2.1 central control system execution process and 2.2 decision logic analysis. Among them, 2.1 central control system execution process, the operator operates according to the scene and the prompt, the augmented reality glasses 36 display the augmented reality scene according to the setting, the hardware device action and the pressure perception, the camera 3, the sensor capture, the perception transmission data, the central control system real-time supervision and management, the data analysis and arrangement, the operation screen virtual image reality, the virtual image feedback, the operator's real-time display in the glasses, the operator adjusts and improves according to the display. 2.2 decision logic analysis contains five parts: 1) data collection and preprocessing, operation model touch feedback input signal, including, operator posture, strength, prop position, device state parameter feedback requires comparison with preset data, analysis effect, preprocessing needs to process three aspects, including: judging the problem of augmented reality glasses 36 and device, the data deviation is large in augmented reality glasses 36, directly prompting the warning, serious deviation or safety does not meet the requirements, directly issuing a stop command; 2) logic rules and algorithm application, which requires detailed analysis of the preprocessed data, including prompt data deviation, converting into action, operation and other prompts, on the other hand, algorithm application is required, which needs to design and use decision tree, network algorithm for overall operation decision; 3) decision generation and evaluation, including three aspects of design, respectively: how to judge the level of the operator, whether to operate according to the augmented reality glasses 36 prompt, whether to need prompt; through the prompt, further improve the improvement situation; summarize what problems exist, so as to further generate the transcript; 4) real-time feedback and adjustment, including issuing a notification, whether to adjust the operation, and generating the transcript of the operation result, generating the picture of the completed result; 5) user interaction and visualization, which requires the interactive interface to provide effective parameter support, and provides picture creation for augmented reality glasses 36, projection screen 33 and central control computer display.

[0086] In the description of the present specification, the description of the terms "preferred embodiment", "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0087] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. An augmented reality based engineering technology training system, characterized by, The simulation system comprises a central control system, an overall frame with an operation position, a simulation module and an auxiliary system, wherein the simulation module and the auxiliary system are connected with the central control system respectively. The simulation module comprises simulation physical operation equipment and monitoring and interactive equipment. The monitoring and interactive equipment comprises a camera for obtaining the operation process of the operator and an augmented reality (AR) glasses for AR performance. The simulation physical operation equipment comprises a conveying belt arranged on the overall frame, a grinding machine cutting model arranged on the overall frame and a touch tool for the operator to perform engineering technical operation, wherein the conveying belt is a lifting conveying belt, the conveying belt serves as a workbench, the operator at the operation position wears the AR glasses, picks up the touch tool and performs operation on the virtual simulation engineering material on the conveying belt to simulate the engineering technical operation, the operator can simulate cutting operation at the grinding machine cutting model through the virtual simulation engineering material, the surface of the conveying belt is provided with a sensor for conducting the pressing effect of the touch tool, and the conveying belt can be lifted to simulate the increasing effect of the brick wall. The central control system comprises a central control computer, a display and a projection device. The camera and the sensor transmit information to the central control computer of the central control system, and the engineering technical operation picture is presented on the AR glasses and the projection screen of the projection device, and the operator adjusts the operation according to the picture in the AR glasses. The auxiliary system comprises a broadcast speaker connected with the central control system for emitting sound.

2. The augmented reality based engineering technical training system as claimed in claim 1, wherein, The touch tool comprises a plurality of tool models hung on an AR hanger, the AR hanger is a ring-shaped hanger, the tool models are hung on the ring-shaped hanger through telescopic hanging ropes and can rotate circumferentially on the ring-shaped hanger. The end of the tool model is provided with a touch sensor, the conveying belt and the grinding machine cutting model are auxiliary props, and the auxiliary props are provided with touch pieces capable of interacting with the touch sensor to generate touch feeling.

3. The augmented reality based engineering technical training system as claimed in claim 2, wherein, A telescopic frame is arranged in the middle of the AR hanger, telescopic top plates capable of being pulled horizontally are arranged on the left and right sides of the telescopic frame, and a downward pulling plate capable of being pulled vertically downward is arranged at the lower end of the telescopic frame, the telescopic top plates and the downward pulling plate are auxiliary props provided with the touch pieces, and the telescopic top plates and the downward pulling plate are used to assist virtual plastering and wall painting.

4. The augmented reality based engineering technical training system as claimed in claim 2, wherein, A plurality of falling blocks for simulating high-altitude falling objects are hung on the AR hanger, and the falling blocks are pull line type falling blocks hung by pull lines and capable of falling and rising.

5. The augmented reality based engineering technical training system as claimed in claim 1, wherein, A lifting foot pedal for simulating the feeling of high-altitude construction is further included, a pedal groove is arranged at the operation position of the overall frame, a plurality of groups of lifting springs are arranged in the pedal groove, the lower end of the lifting foot pedal is connected with and supported by the upper end of the lifting springs, the lifting foot pedal can be lifted and swung left and right by telescoping the lifting springs.

6. The augmented reality based engineering technical training system of claim 5, wherein, Each group of the lifting springs comprises a spring and a jack in the spring, the upper end of the spring is fixedly connected with the lower end of the lifting foot pedal, the jack is in contact or not in contact with the lower end of the lifting foot pedal, and the spring can be telescoped by lifting the jack to make the lifting spring telescoped.

7. The augmented reality based engineering technical training system as claimed in claim 1, wherein, The conveying belt is installed in a support frame fixed to the integral frame, and a lifting mechanism is installed on the support frame to drive the conveying belt to lift in the support frame; A plurality of the cameras are installed on the top of the support frame.

8. The augmented reality based engineering technical training system as claimed in claim 7, wherein, The inner side of the support frame is provided with vertically extending lifting tracks, the conveying belt has slide columns clamped in the lifting tracks and sliding in the tracks, and the lifting tracks are provided with upper and lower barriers to limit the up and down movement range of the conveying belt.

9. The augmented reality based engineering technical training system as claimed in claim 1, wherein, The integral frame is provided with a downward recessed lower groove, the conveying belt can be embedded into the lower groove, and the upper surface of the conveying belt can be flush with the top of the lower groove. A plurality of auxiliary bricks are placed in the lower groove below the conveying belt, and the auxiliary bricks can slide into the lower groove along with the operation of the conveying belt.

10. The augmented reality based engineering technical training system as claimed in any one of claims 1 to 8, wherein, The integral frame is composed of a metal framework and a plurality of cover plates covering the metal framework, the integral frame has a high area in the middle and a low area around the high area, the operating position, the conveying belt and the touch tool are arranged in the high area, and the abrasive wheel cutting model, the central control computer, the display and the projector are arranged in the low area.

Images