Engineering technology training system based on augmented reality

Abstract

The invention provides an engineering technology training system based on augmented reality. The engineering technology training system comprises a central control system, an overall frame with an operation position, a simulation module and an auxiliary system. The simulation module comprises a simulation real object operation device and a monitoring and interaction device. The monitoring and interaction device comprises a camera and augmented reality glasses; the real object operation simulation equipment comprises a lifting type conveying belt, a grinding machine cutting model and a wired touch tool which are arranged on the overall frame. An operator at an operation position sees a virtual simulation engineering material through augmented reality glasses, performs operation through an entity tool, sees the effect of completing operation by himself / herself, and performs operation and adjustment in time; the projection effect of the projection equipment is synchronized with the effect in the augmented reality glasses, the operation result of an operator is displayed, and reference and basis are provided for other onlooker learners; and the broadcast sound equipment is used for playing prompt tones and carrying out man-machine interaction sound.

Description

Technical Field

[0001] The invention belongs to the technical field of augmented reality, and in particular relates to an engineering technology training system based on augmented reality. Background Art

[0002] In recent years, my country has attached great importance to the cultivation of practical operation capabilities of engineering technology. Various colleges and universities have also actively participated in the cultivation and training of skilled personnel, and actively organized students to participate in engineering technology related competitions, such as masonry and decoration competitions. They have long occupied an important position in national and world competitions, and are also highly valued by various colleges and universities.

[0003] From the perspective of teaching requirements, knowledge contents such as masonry and plastering, steel bar processing and binding, wood processing, and wall and ceiling decoration in engineering construction technology are all difficult and key contents. Therefore, as a knowledge that must be learned and learned well, the significance of combining theory with practice is very important.

[0004] From the analysis of relevant content requirements, the cultivation of engineering and technical professional capabilities, safety education, emergency problem handling, event training and pre-job training have high requirements for specific practical content. Therefore, real training requires a lot of resources, including but not limited to cement, sand and gravel, bricks, brick cutting grinders, scrapers, plumb lines, levels, trowels, rollers, paints, brushes and other tools; and professional personnel guidance, and high requirements for the working environment, need to consider many requirements such as weather, transportation, site size, and ease of cleaning.

[0005] From the perspective of educational resource conditions, masonry construction should especially take into account the fact that cement has a shelf life of only three months, standard blocks are easily damaged during loading and unloading, mixing water is used for transportation, tool wear and tear, and dust pollution has an impact on health. The finished masonry products cannot be reused and directly become construction waste, and the volume and quality are very large, and the difficulty of loading and unloading and transportation is similar to that of demolition. A school trained 6-8 masons to participate in a vocational skills competition, and produced more than 20 cubic meters of construction waste in one month. For example, decorators, for wall tile laying, paint painting and other operation training, must also consider serious problems such as material fragility, material odor, and formaldehyde pollution. Carpenters need to consider wood processing, the dangers of operating electric saws and planers, and steel workers need to process materials, but steel cutting machines, straightening machines, etc. are expensive, and the placement of steel bars requires a large space and requires waterproofing and rust prevention.

[0006] From the perspective of job requirements, although plastering robots have appeared, they are limited by the cost of use, especially the conditions of use. They are usually used for simple penetration of walls and require uncomplicated large-surface operations, such as shear walls. When encountering locations such as corners, door and window openings, work efficiency drops significantly. In addition, efficiency will also be affected by problems such as site flatness, working environment, electricity, and mechanical jams. Once the technology is upgraded and the level of artificial intelligence of the equipment is improved, its cost will rise, which will have a greater impact on the cost. At present, relatively few high-tech equipment for related engineering technologies have been put into use, and the problems that can be solved are relatively simple. Therefore, manual completion of masonry, plastering, steel and wood processing and decoration is still the main demand. Even if prefabricated buildings are added, it is still necessary to consider the problems of manual collaborative assistance and manual operation of complex nodes.

[0007] From the perspective of job management and accident prevention, operators can clearly realize that accidents occur in the actual work process, not in the specially prepared drill prevention process. As the saying goes, it is better to take precautions before they happen. Safety and risk control also need to start from beginners. Through augmented reality simulation of safety problems encountered in specific operations, such as bricks accidentally falling and hitting feet, personnel misoperation, causing tools to fall from the air and hit personnel, personnel suddenly stepping on empty space, etc., operators are reminded of the importance of safety risk control at all times.

[0008] In summary, the existing education and teaching, professional ability training, and safety emergency training for masonry, plastering, and wall construction have problems such as high material costs, dust pollution, large space occupied by consumables, and exercises are easily restricted by the weather environment. Therefore, it is necessary to provide an engineering technology training system based on augmented reality to solve the above problems.

[0009] After searching, no similar technical solution to the present invention has been disclosed. Summary of the invention

[0010] The present invention aims to solve the technical problems existing in the prior art, and the purpose of the present invention is to provide an engineering technology training system based on augmented reality.

[0011] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: an engineering technology training system based on augmented reality, comprising a central control system, an overall frame with an operating position, a simulation module and an auxiliary system, wherein the simulation module and the auxiliary system are respectively connected to the central control system; the simulation module comprises a simulated physical operation device, a monitoring and interactive device; the monitoring and interactive device comprises a camera for acquiring the operator's operation process and augmented reality glasses for augmented reality performance; the simulated physical operation device comprises a conveyor belt arranged on the overall frame, a grinding wheel machine cutting model arranged on the overall frame, and a wired tactile tool arranged on the overall frame for the operator to perform engineering technology operations; the conveyor belt is a lifting conveyor belt, the conveyor belt serves as a workbench, and the operator at the operating position Wearing augmented reality glasses, the operator can simulate engineering technical operations by picking up tactile tools and operating virtual simulated engineering materials on a conveyor belt. The operator can simulate cutting operations at a grinding machine cutting model with virtual simulated engineering materials. The surface of the conveyor belt is provided with sensors for transmitting the pressing effect of the tactile tools. The conveyor belt can be raised and lowered to simulate the effect of continuously improving the masonry brick wall. The central control system includes a central control computer, a display and a projection device. The camera and the sensor transmit information to the central control computer of the central control system, and the pictures of the engineering technical operations are presented on the projection screen of the augmented reality glasses and the projection device. The operator adjusts the operation according to the pictures in the augmented reality glasses. The auxiliary system includes a broadcast speaker connected to the central control system for emitting sound.

[0012] In the above technical solution, the overall framework constitutes the outer contour of the training system, which is used to install the central control system, simulation modules and auxiliary systems, and provide an operating space for operators; the tactile tools are physical tools used in actual construction work. The central control system adjusts the height of the conveyor belt in combination with the operating information collected by the camera and sensor information. After the conveyor belt is lifted, it can be used as an auxiliary prop for augmented reality simulation to achieve masonry, steel bar cutting, carpentry and other operations; the operator sees the virtual simulated engineering materials through the augmented reality glasses, and operates through the physical tools, sees the results of his own operations, and makes timely adjustments; the projection effect of the projection equipment is synchronized with the effect in the augmented reality glasses, showing the operator's operation results, and providing reference and basis for other onlookers; the broadcasting audio is used to play prompt sounds and human-computer interaction sounds.

[0013] In a preferred embodiment of the present invention, the tactile tool includes a plurality of tool models suspended on an augmented reality hanger, the augmented reality hanger is an annular hanger, the tool models are suspended on the annular hanger by a telescopic hanging rope and can rotate circumferentially on the annular hanger; a touch sensor is provided at the end of the tool model, and the conveyor belt and the grinding machine cutting model are auxiliary props, and the auxiliary props are provided with a touch member that can interact with the touch sensor to generate a touch sense.

[0014] In the above technical solution, several tool models are hung on the augmented reality hanger by telescopic hanging ropes, which is convenient for the operator to pick up various tool models, assist the augmented reality effect, and realize the simulation to the greatest extent; a touch sensor is set at the end of the tool model, and a touch part is set on the auxiliary prop to simulate the touch of the tool model. When the tool model touches the conveyor belt, grinding machine cutting model and other designated areas, the force feel is simulated and information is sent back.

[0015] In a preferred embodiment of the present invention, a telescopic frame is provided in the middle of the augmented reality hanger, and telescopic top plates that can be pulled laterally are installed on the left and right sides of the telescopic frame, and a lower stretching plate that can be pulled vertically downward is installed at the lower end of the telescopic frame. The telescopic top plate and the lower stretching plate are used to assist virtual plastering and wall painting.

[0016] In the above technical solution, the telescopic top plate and the lower stretching plate are retracted into the telescopic frame when not in use; when in use, the telescopic top plate and the lower stretching plate are pulled out from the telescopic frame to assist in the augmented reality effect of virtual plastering and wall painting.

[0017] In a preferred embodiment of the present invention, a plurality of falling blocks for simulating falling objects from high altitude are suspended on the augmented reality hanger, and the falling blocks are pull-wire type falling blocks which are suspended by pull wires and can fall and rise.

[0018] In the above technical solution, the falling blocks are used to simulate falling objects from high altitude. During the operation, the falling blocks will randomly fall according to the operation. When the tool model in the operator's hand slips out of his hand, the pull wire of the falling block above will loosen and the falling block will randomly fall down to simulate falling objects from high altitude and to remind the operator of the harmfulness of falling tools or materials.

[0019] In a preferred embodiment of the present invention, the training system also includes a lifting foot pedal for simulating the working feeling of high-altitude construction. A pedal groove is provided at the operating position of the overall frame, and several groups of lifting springs are provided in the pedal groove. The lower end of the lifting foot pedal is connected to and supported by the upper end of the lifting spring. By extending and retracting several lifting springs, the lifting foot pedal can be raised and lowered and swung left and right.

[0020] The above technical solution, by lifting the lifting foot pedal through the lifting spring, can enhance the reality and simulate high-altitude construction, such as masonry, scaffolding erection, formwork installation, etc.; the lifting foot pedal swings left and right to create a feeling of shaking under the feet, which is used to simulate the safety issues during the operation and give the operator a close to real experience.

[0021] In another preferred embodiment of the present invention, each group of lifting springs includes a spring and a jack located in the spring, the upper end of the spring is fixedly connected to the lower end of the lifting foot pedal, the jack is in contact with or not in contact with the lower end of the lifting foot pedal, and by raising and lowering the jack, the spring can be extended and retracted to make the lifting spring extend and retract.

[0022] In the above technical solution, the spring and the jack form a lifting spring, the upper end of the spring is fixedly connected to the lower end of the lifting pedal, and when the jack contacts the lower end of the lifting pedal, the lifting pedal can be lifted or lowered; when the jack does not contact the lower end of the lifting pedal, the spring is slightly higher than the top of the jack therein, and the lifting pedal can swing left and right, creating a shaking feeling under the feet.

[0023] In another preferred embodiment of the present invention, the conveyor belt is installed in a support frame fixed to the overall frame, and a lifting mechanism for driving the conveyor belt to rise and fall in the support frame is installed on the support frame; a plurality of cameras are installed on the top of the support frame.

[0024] In the above technical solution, the conveyor belt is supported by a support frame to assist augmented reality operation; multiple cameras are installed on the top of the support frame to monitor all operations in the entire venue without blind spots.

[0025] In another preferred embodiment of the present invention, a vertically extending lifting track is provided on the inner side of the support frame, the conveyor belt has a sliding column which is clamped in the lifting track and slides in the track, and the lifting track is provided with upper and lower baffles which limit the upper and lower movable range of the conveyor belt.

[0026] In the above technical solution, the lifting rail and the slide column guide the lifting movement of the conveyor belt, making the lifting of the conveyor belt smoother; upper and lower barriers are set to limit the upper and lower movement range of the conveyor belt, which can prevent the conveyor belt from detaching from the support frame.

[0027] In another preferred embodiment of the present invention, a downwardly recessed lower groove is provided on the overall frame, the conveyor belt can be embedded in the lower groove, and the upper surface of the conveyor belt can be flush with the top of the lower groove; a number of auxiliary bricks located below the conveyor belt are also placed in the lower groove, and the auxiliary bricks can slide into the lower groove as the conveyor belt runs.

[0028] The above technical solution enables the conveyor belt to be embedded in the lower groove and its top to be flush with the surrounding area. At this time, ground layout, low-position tile laying, masonry and other construction can be simulated; the lower groove serves as a space for storing auxiliary bricks. By setting auxiliary bricks, the auxiliary bricks are used as auxiliary props for brick arrangement or simulation of the masonry process and placed on the conveyor belt. The masonry action is simulated by tactile tools, which makes it more realistic.

[0029] In another preferred embodiment of the present invention, the overall frame is composed of a metal frame and a plurality of cover plates covering the metal frame, the overall frame has a high zone in the middle and a lower zone located around the high zone, the operating position, conveyor belt and tactile tools are arranged in the high zone, and the grinding machine cutting model, central control computer, display and projector are arranged in the low zone.

[0030] The above technical solution divides the overall framework into a high area and a low area. The high area is the main operating activity area for the operator, and other personnel can operate the central control computer in the low area without interfering with each other.

[0031] Compared with the prior art, the superior technical solution of the present invention has the following beneficial effects: the present invention is a human-computer interactive simulation teaching system that can enhance reality. Through augmented reality operations, especially the specifically set virtual scenes of safety warnings and safety accidents during operation, participants can experience the occurrence of accidents in an immersive and harmless way. According to the specific operation content and the operator's reaction, the operator is trained to not avoid emergency issues and to have the courage to face risks and disasters.

[0032] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0034] Figure 1 It is a block diagram of the training system of the present invention.

[0035] Figure 2 It is a structural schematic diagram of the training system of the present invention.

[0036] Figure 3 It is a structural schematic diagram of the overall framework of the present invention.

[0037] Figure 4 It is a schematic diagram of the structure of the metal skeleton in the present invention.

[0038] Figure 5 It is a structural schematic diagram of the lifting foot pedal in the present invention.

[0039] Figure 6 It is a schematic diagram of the lifting conveyor belt in the present invention being at the highest point.

[0040] Figure 7 It is a schematic diagram of the lifting conveyor belt in the present invention being at the lowest point.

[0041] Figure 8 It is a schematic diagram of the operation of the auxiliary bricks in the present invention.

[0042] Fig. 9 It is a schematic diagram of a tool mounting model on an augmented reality rack in the present invention.

[0043] Fig.10 It is a schematic diagram of setting a touch sensor and a touch piece on a tool model and an auxiliary prop respectively.

[0044] Fig.11 It is a structural schematic diagram of the telescopic frame of the present invention when it is unfolded.

[0045] Fig.12 It is a schematic diagram of the falling block in the present invention.

[0046] Fig.13 It is a schematic diagram of the projector and projection effect in the present invention.

[0047] Fig.14 It is a schematic diagram of synchronous projection of the operation of the present invention.

[0048] Fig.15 This is a diagram showing the effect of using the augmented reality glasses of the present invention.

[0049] Fig.16 It is an augmented reality operation effect diagram of the present invention.

[0050] Fig.17 It is a block diagram of the overall structure and operational relationship of the present invention.

[0051] The figure marks in the drawings of the specification include: augmented reality bracket 1, telescopic bracket 2, camera 3, bracket support column 4, conveyor belt 5, pillar 6, baffle 7, high area 8, central control computer 9, computer desk 10, supporting equipment slot 11, low area 12, grinding machine cutting model 13, grinding wheel chassis 14, equipment expansion slot 15, lower slot 16, lifting track 17, cover plate 18, metal frame 19, distribution box 20, fire extinguisher 21, lifting foot pedal 22, spring 23, pedal slot 24, jack 25, overall frame 26, auxiliary bricks 27, telescopic hanging rope 28, tactile tool (tool model) 29, telescopic top plate 30, lower stretching plate 31, drop block 32, projection screen 33, projection effect 34, projector 35, augmented reality glasses 36, augmented reality visual effect 37, augmented reality operation effect 38, touch sensor (electromagnet) 39, touch piece (ordinary magnet) 40. DETAILED DESCRIPTION

[0052] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0053] In the description of the present invention, it is to be understood that the terms “longitudinal”, “lateral”, “vertical”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside” and “outside” etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0054] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0055] The present invention provides an engineering technology training system based on augmented reality, such as Figure 1 and Figure 2 As shown, in a preferred embodiment, the training system includes a central control system, an overall frame 26 with operating positions, a simulation module and an auxiliary system, and the simulation module and the auxiliary system are respectively connected to the central control system.

[0056] The simulation module includes simulated physical operation equipment, monitoring and interactive equipment.

[0057] like Figure 2 and Fig.14 As shown, the interactive device includes a camera 3 for acquiring the operator's operation process and augmented reality glasses 36 for augmented reality performance. The augmented reality glasses 36 can be augmented reality AR glasses commonly used in the market, or separately designed glasses.

[0058] like Figure 1 and Figure 2As shown, the simulated physical operation equipment includes a conveyor belt 5 provided on the overall frame 26, a grinder cutting model 13 provided on the lower area 12 of the overall frame 26, and a wired tactile tool 29 provided on the overall frame 26 for the operator to perform engineering technical operations. The conveyor belt 5 is a lifting conveyor belt 5, and the conveyor belt 5 serves as a workbench. The operator at the operating position wears augmented reality glasses 36, and picks up the tactile tool 29 (physical tool) and operates the virtual simulated engineering materials (the simulated engineering materials are displayed in the screen of the augmented reality glasses 36 through virtual technology, mainly including steel bars, wood, bricks, pipes, welding materials, etc.) on the conveyor belt 5 to simulate the engineering technical operation. The operator can simulate the cutting operation at the grinder cutting model 13 through the virtual simulated engineering materials. The surface of the conveyor belt 5 is provided with a sensor (not shown in the figure) for transmitting the pressing effect of the tactile tool 29, and the conveyor belt 5 can be raised and lowered to simulate the effect of the continuous improvement of the brick wall.

[0059] like Figure 2 and Fig.12 As shown, the central control system includes a central control computer 9, a display and a projection device. The central control computer 9 includes a host, a keyboard and a mouse. The host is connected to the display and the projection device; the central control computer 9 has a built-in overall operation system and a regional connection system. The central control computer 9 is an overall control device, which connects all devices and implements overall and real-time control of the overall software and hardware combined operation process through the control system. The projection device includes a projector 35 and a projection screen 33. The projector 35 can be a common school classroom projector 35 style to ensure connection to the central control computer 9; the projection screen 33 uses a common projection screen 33 material, with a size of 1.5 meters to 2 meters in width and 1.5 meters to 2 meters in height. The display is used as a tool for normal operation display and information entry in the central area. The central control computer 9, the display and the projector 35 are placed on the computer desk 10 above. The camera 3 and the sensor transmit information to the central control computer 9 of the central control system, and the screen of the engineering technology operation is presented on the augmented reality glasses 36 and the projection screen 33 of the projection device. The operator adjusts the operation according to the screen on the projection screen 33.

[0060] like Figure 1 and Figure 2 As shown, the auxiliary system includes a broadcasting speaker connected to the central control system for emitting sound, which is used for playing prompt sounds and performing human-computer interaction sounds. The auxiliary system also includes a distribution box 20 for power supply, and a fire extinguisher 21 as a fire-fighting facility. The fire extinguisher 21 is a carbon dioxide fire extinguisher 21. The broadcasting speaker and the main switch of the training system are arranged on the distribution box 20.

[0061] When using the training system, the operator wears augmented reality glasses 36 and uses the physical tactile tool 29 to perform the predetermined actions. The tactile tool 29 is connected to a wired sensor to collect the specific action parameters in real time and transmit them to the central control system. Through the virtual simulation system, the action effect is analyzed and processed to form a real-time picture, and then the operation effect is converted into a visual display. At the same time, the operator can see the effect of the operation completed by himself through the augmented reality glasses 36, and make timely adjustments to make the digital model and the tactile tool 29 consistent in action and effect, so as to achieve the purpose of teaching; before teaching, standardized operation actions such as gestures, bending angles, methods of using tools, etc. can be photographed and entered into the input system, and used as a comparison standard during the training process; at the same time, the overall control platform of the interactive system can be connected to the Internet and the cloud computing system, and the operation information of each device of the training system can be sent to each application system in the cloud, so as to facilitate the overall grasp of the effect and real-time detection of physical connection, troubleshooting, virtual operation and other needs.

[0062] like Figure 2-Figure 4 As shown, the overall frame 26 is composed of a metal frame 19 and a number of cover plates 18 covering the metal frame 19. The overall frame 26 has a high area 8 located in the middle and a lower low area 12 located around the high area 8. The high area 8 is 0.25-0.5 meters high and 3-4 meters wide. The operating position, conveyor belt 5 and tactile tool 29 are located in the high area 8, which is the main operating activity area for the operator; the low area 12 is about 0.1 meters above the indoor ground and is the basic platform for the grinding machine cutting model 13 and the central control computer 9. The metal frame 19 is assembled by welding or bolting a number of steel sections, such as 20×20×1.5 hollow square steel, or other forms of steel sections such as I-beams and channel steels. Welding or steel bolt connection can be used to ensure structural strength and ensure that the structure is stable and reliable, especially considering the vibration and impact of mechanical equipment and personnel walking up and down. The metal frame 19 is covered with a cover plate 18, which can be made of metal, hard plastic or composite wood, with a thickness of 3cm-5cm, and is connected by welding or bolts. Considering the stress and wear resistance, a protective measure can be laid on the upper part of the cover plate 18, such as epoxy resin. Three baffles 7 are set on the left and right sides and the rear side of the overall frame 26. The baffles 7 are 0.5-1 meters high and are used to enclose the work inside the overall frame 26 to ensure the operating environment. The baffles 7 can be made of plastic plates or metal plates.

[0063] like Figure 2 As shown, the computer table 10 is installed in the low area 12, with a height of 0.5-1 meters, and is made of a transparent acrylic plastic plate, for placing the central control computer 9, the display and the projector 35, and the desktop shape can be designed to be round or square. A supporting equipment slot 11 is provided below the computer table 10 for matching and adding related servers, the central control computer 9 host and other equipment.

[0064] like Figure 2 As shown, the grinding machine cutting model 13 is a machine model, made of acrylic plastic plate, and is set at a height of about 1 meter. The height is appropriate, and the operator can operate it by standing at the operating position of the high area 8, which is used for augmented reality operation; the grinding machine cutting model 13 is installed on the grinding wheel chassis 14 of the low area 12, and the shape is a circular platform. It is strictly forbidden to place objects; an equipment expansion slot 15 is provided below the grinding wheel chassis 14, and its shape and position can be used as a carrier of augmented reality, that is, through the augmented reality glasses 36, the position and operation of the buttons below the grinding machine cutting model 13 can be observed.

[0065] like Figure 5 As shown, the training system also includes a lifting pedal 22 for simulating the working feeling of high-altitude construction, and a pedal slot 24 is provided at the operating position of the overall frame 26. For example, a pedal slot 24 is provided on each side of the conveyor belt 5 in the high area 8, and a plurality of lifting springs 23 are provided in the pedal slot 24. The lower end of the lifting pedal 22 is connected to and supported by the upper end of the lifting spring 23. The lifting spring 23 is responsible for the lifting and lowering of the lifting pedal 22. The maximum weighing is 250kg. By extending and retracting a plurality of lifting springs 23, the lifting pedal 22 can be lifted and lowered and swung left and right. Initially, the lifting pedal 22 is located at the top of the pedal slot 24 to close it. Preferably, two groups of lifting springs 23 are provided in the middle and left and right sides of the bottom of the lifting pedal 22, for a total of six groups of lifting springs 23. Each group of lifting springs 23 includes a columnar spring 23 and a jack 25 located in the spring 23. The jack 25 is connected to an electromagnetic switch to control the operation of the jack 25. The lower end of the spring 23 is fixedly connected to the bottom of the pedal groove 24, the upper end of the spring 23 is fixedly connected to the lower end of the lifting pedal 22, and the upper end of the jack 25 is not connected to the lower end of the lifting pedal 22, and the two can be in contact or not. By raising and lowering the jack 25, the spring 23 can be extended and retracted, so that the lifting spring 23 can be raised and lowered and the lifting pedal 22 can be raised and lowered, and by extending and retracting a plurality of lifting springs 23, the lifting pedal 22 can be raised and lowered and swung left and right.

[0066] The operator's foot is on the designated area of ​​the lifting pedal 22. During operation, it is used to simulate the feeling of lifting and stepping on the scaffolding and high-altitude construction. When the operator's footstepping range exceeds the designated area, all lifting springs 23 will be shortened synchronously, and the lifting pedal 22 will drop within 10cm to serve as a warning, focusing on the effect of safety management in actual combat. When the jacks 25 of the two sets of lifting springs 23 in the middle lift the lifting pedal 22 by 0.1 meters, the jacks 25 on the left and right sides do not move, so that the springs 23 on the left and right sides are slightly higher than the top of the jacks 25 inside, and the lifting pedal 22 swings left and right, which can create a slight shaking feeling under the feet; after the jacks 25 on the left and right sides drop, the length of the spring 23 extending outside the jacks 25 increases, and the swing will be strengthened, which is used to simulate the safety problems in the operation process and give the operator a close to real experience. After the experience is over, the operator jumps directly on the spot, the jacks 25 retract, presses the lifting springs 23 back to the original normal height, and the lifting pedal 22 returns to the original normal height.

[0067] like Figure 2 , Figure 6 and Figure 7 As shown, in the present invention, the conveyor belt 5 is installed in the support frame 6 fixed to the overall frame 26, and the support frame 6 is equipped with a lifting mechanism that drives the conveyor belt 5 to rise and fall in the support frame 6. The lifting mechanism is a prior art, and its structure and principle are not described in detail here. Among them, the support frame 6 includes four support frames 6 arranged at the four corners outside the conveyor belt 5, and the inner side of each support frame 6 is provided with a vertically extending lifting track 17. The support frame 6 uses channel steel as a support to ensure its sturdy and durable reliability. The height is 2-3 meters, non-retractable, and the bottom is connected to a metal skeleton 19; the conveyor belt 5 has a sliding column that is stuck in the lifting track 17 and slides in the track. The lifting track 17 is provided with upper and lower partitions that limit the upper and lower range of movement of the conveyor belt 5. A group of cameras 3 are installed on the top of each support frame 6 to observe the internal and external conditions of the overall frame 26 without blind spots and assist in coordinated operation. The camera 3 is designed to be 2.5-3-5 meters high, rotatable, and connected to the central control computer 9 through a network cable.

[0068] Preferably, the high area 8 of the overall frame 26 is provided with a downwardly recessed lower groove 16, which is located below the lifting conveyor belt 5, and the lifting track 17 extends downward into the lower groove 16. When the conveyor belt 5 moves downward, it can be embedded in the lower groove 16, and the upper surface of the conveyor belt 5 can be flush with the top of the lower groove 16.

[0069] like Figure 6 and Figure 7As shown, the surface of the conveyor belt 5, which can be adjusted in height up and down, is provided with sensors to transmit the effect of the tactile tool 29 pressing. Initially, the conveyor belt 5 is at the lowest point, and the top of the conveyor belt 5 is flush with the surroundings. At this time, the ground laying, low-position tile paving, masonry and other construction can be simulated. The central control system adjusts the height of the conveyor belt 5 in combination with the operation information collected by the camera 3 and the sensor information. After the conveyor belt 5 is lifted, it can be used as an auxiliary prop for augmented reality simulation to achieve masonry, steel bar cutting, carpentry and other operations. As the height of the conveyor belt 5 is raised, the lifting spring 23 under the operator's feet lifts the lifting foot pedal 22, which can enhance the reality simulation of high-altitude construction, such as masonry, scaffolding, template installation, etc. The maximum lifting height of the conveyor belt 5 is set to 2.5 meters.

[0070] like Figure 8 As shown, in another preferred embodiment, a plurality of auxiliary bricks 27 located below the conveyor belt 5 are also placed in the lower groove 16. The auxiliary bricks 27 can slide into the lower groove 16 as the conveyor belt 5 runs. The auxiliary bricks 27 are also auxiliary props, and a common magnet 40 is also provided on the surface thereof, which can interact with the electromagnet 39 at the end of the tool model 29. The size of the auxiliary bricks 27 is the same as the size of the actual building blocks or standard bricks, such as 240mm×115mm×53mm, etc., and is made of plastic material, and the weight is close to the weight of real bricks and building blocks. After the conveyor belt 5 is raised, the operator takes out the auxiliary bricks 27 from the lower groove 16, and places the auxiliary bricks 27 on the conveyor belt 5 as auxiliary props for brick arrangement or simulation of the masonry process according to the masonry requirements, and simulates the masonry action through the wired touch tool 29. At this time, the conveyor belt 5 stops moving. After completing this action, a new auxiliary brick 27 is picked up from the lower groove 16 under the conveyor belt 5, and the above actions are repeated. After a row of masonry is completed, the conveyor belt 5 runs slowly, and the auxiliary bricks 27 are conveyed to the left slide and fall into the lower groove 16. With the help of information collection by the sensor and camera 3, the masonry effect is presented on the augmented reality glasses 36, the display and the projection screen 33.

[0071] In addition, the conveyor belt 5 at the middle height position can also be used as an auxiliary prop to simulate plumbers processing pipes, welders welding parts, carpenters processing fine work, and rebar workers processing rebars. Combined with the augmented reality glasses 36, the overall virtual operation is presented. The specific augmented reality simulation method is implemented according to the set actions, operation auxiliary props, and scene requirements.

[0072] like Figure 1 and Fig. 9As shown, the tactile tool 29 includes several tool models hanging on the augmented reality hanger 1, which are real tools, including but not limited to basins, roller brushes, shovels, trowels, screwdrivers, plumb bobs, spirit levels, etc., which are used for actual construction work; the augmented reality hanger 1 is a ring hanger, and the tool model 29 is hung on the ring hanger by a telescopic hanging rope 28 and can rotate circumferentially on the ring hanger. Specifically, the augmented reality hanger 1 is designed to be circular, made of aluminum alloy or steel, with a diameter of 2-3 meters, and fixed to the top of multiple hanger support columns 4 fixed to the high area 8 of the overall frame 26. The hanger support columns 4 are made of steel and are welded or threaded to the metal frame 19 of the high area 8, and are firmly connected. The augmented reality hanger 1 is provided with an annular hanging groove, the upper end of the telescopic hanging rope 28 is fixed in the annular hanging groove and can move in the annular hanging groove, the tool model 29 is fixed to the lower end of the telescopic hanging rope 28, and the telescopic hanging rope 28 can adopt a plastic spring 23 (refer to the structure of a bicycle metal chain lock and a landline telephone line), which can be retracted and extended to a maximum length of 3-4 meters, covering all areas of the overall frame 26, providing the operator with a sense of realistic operation.

[0073] The operator holds the tool model 29 with one hand or two hands to assist the operator in experiencing the feel of real operation during operation. The operator can simulate the completion of masonry, steel bar processing, pipe processing, welding, decoration and other tasks by setting a program.

[0074] like Fig.10 As shown, a touch sensor 39 is provided at the end of the tool model 29, and the conveyor belt 5 and the grinding machine cutting model 13 are auxiliary props, and a touch piece 40 that can interact with the touch sensor 39 to generate a touch feeling is provided on the auxiliary prop. For example, the touch sensor 39 at the end of the tool model 29 is an electromagnet, and the telescopic hanging rope 28 is built with a thin wire for powering the electromagnet 39, and the touch piece 40 is an ordinary magnet with a magnetic pole opposite to that of the electromagnet 39. By setting the electromagnet 39 and the ordinary magnet 40, the principle of magnet repulsion is used to simulate the real touch of tools such as mortar pressing, mortar plastering, cutting, or brush heads; at the same time, a sensor is connected to the end of the telescopic hanging rope 28, and when the electromagnet 39 touches the ordinary magnet 40 on the designated areas such as the conveyor belt 5, the grinding machine cutting model 13, the telescopic top plate 30 and the lower stretching plate 31, the sensor transmits information back to the central control system.

[0075] like Fig.11As shown, a telescopic frame 2 is provided in the middle of the augmented reality hanger 1, which is designed with three grooves, one on the left and the other on the bottom. The left and right sides of the telescopic frame 2 are equipped with a telescopic top plate 30 that can be pulled horizontally. The material is plastic. It is pulled by a rope and fully unfolded to cover the top. It serves as an auxiliary prop for the top auxiliary simulation of ceiling painting and plastering. Its surface is also provided with a common magnet 40 that can interact with the electromagnet 39 at the end of the tool model 29. The lower end of the telescopic frame 2 is equipped with a lower stretching plate 31 that can be pulled vertically downward. The material is plastic. It is pulled by a rope and fully unfolded. It can fall to the ground. It serves as an auxiliary prop for the augmented reality simulation construction of the wall. Its surface is also provided with a common magnet 40 that can interact with the electromagnet 39 at the end of the tool model 29.

[0076] like Fig.12 As shown, a plurality of falling blocks 32 for simulating falling objects from a high altitude are suspended on the augmented reality hanger 1. The falling blocks 32 are hanging by a pull line and can fall and rise. The material of the falling blocks 32 is relatively light and is specially designed to simulate falling objects from a high altitude. During the operation, the operator can receive a sound reminder from the broadcasting speaker to be careful of falling objects, and then the falling blocks 32 fall randomly around him and may also hit the operator's helmet. The falling blocks 32 fall randomly according to the operation. When the tool model 29 in the operator's hand falls out of hand, the pull line of the upper falling block 32 is loosened and falls randomly, which is used to simulate falling objects from a high altitude and also remind the operator of the harmfulness of falling tools or materials.

[0077] Fig.13 and Fig.14 Shown is a projection effect 34 displayed on a projection screen 33 by a projector 35. This effect is displayed as one aspect of the present invention. The projection effect 34 is synchronized with the effect in the augmented reality glasses 36. Through the special projection effect 34 display, the operator's operation results are displayed to provide reference and basis for other onlookers and learners.

[0078] Fig.15 Shown is the augmented reality visual effect 37 seen in the augmented reality glasses 36, which is a real-time picture seen by the operator through the glasses, especially as the operator moves, ensuring that the scene seen in the augmented reality glasses 36 changes synchronously with the operator's movements, but the changes cannot be too drastic.

[0079] Fig.16 The augmented reality operation effect 38 of the present invention is shown, which is the operating action of the personnel in the augmented reality presentation process. The operation is completed through various props (including conveyor belt 5, grinding machine cutting model 13, telescopic top plate 30, lower stretching plate 31, auxiliary bricks 27, tool model 29 and simulated engineering materials, etc.) combined with augmented reality effects. The personnel hold the tool model 29 and operate normally according to the scene seen in the augmented reality glasses 36 and the prompt sound emitted by the broadcast speaker.

[0080] Fig.17 The figure shows the combination of software and hardware of the entire training system, which is mainly divided into four parts: the overall frame 26, the central control system, the virtual simulation module and the auxiliary system. Among them, the overall frame 26 is divided into a metal frame 19 and a cover plate 18, which constitute the external outline of the training system, and the other three parts are a combination of software and hardware. The central control system is the core control function, which is mainly divided into three aspects, namely operation and operation, blockchain storage and display functions, and controls all the software and hardware of the system during operation; blockchain storage is set up with a blockchain unit, which divides different blocks according to the steps, and records the steps and processes in a real and effective manner to ensure accuracy and not allow tampering, especially the safety performance in each step, the implementation of safety requirements, such as wearing a safety helmet, virtual high-altitude work with a safety belt and other measures; the display function focuses on the effect presentation of synchronous display, requiring the display, projector 35 and augmented reality glasses 36 to display different requirements of the same content at the same time. The virtual simulation module focuses on the combination of software and hardware operation and execution, with the help of virtual auxiliary tools, especially wired tactile tools 29, all of which are equipped with electromagnets at the end to simulate the touch of tools and cooperate with augmented reality effects, and through the monitoring device camera 3, 360° rotation supervision, all operations in the entire venue can be monitored without blind spots. The auxiliary system mainly includes facilities that provide network, power supply, alarm and broadcasting for the entire system to ensure smooth and reliable operation.

[0081] The innovation of the central control system of the present invention lies in the combination of augmented reality control, software and hardware collaboration and blockchain technology to restore the real scene to the greatest extent and record the operation process truthfully and reliably.

[0082] Among them, the first part of the central control system is operation and operation, which is divided into three parts: 1. Perception data processing module, the design is divided into 1.1 part, camera 3 shoots the overall operation of the lifting conveyor belt 5. Four groups of cameras 3 are set on the top of the pillar 6 to cover the entire area, shoot in real time, and transmit the shooting information in real time; 1.2 The physical model matches the sensor information, which is used to realize the connection sensor of the hanging rope at the end of the tool model 29, and ensure that the tool model 29 operates a section of the connected electromagnet, the electromagnet touches the conveyor belt 5, the grinding machine cutting model 13, the telescopic top plate 30 and the lower stretching plate 31 and other designated areas, and the sensor returns information; 2. The core control module is divided into 2.1 central control system execution process and 2.2 decision logic analysis. The specific design functions are detailed in the following text; 3. Three-dimensional registration and tracking require the realization of: real-time comparison between virtual objects and real models and real-time tracking of operation effects to ensure alignment.

[0083] The second part of the central control system is blockchain storage, which is mainly designed as 4. Blockchain integration module, including 4.1 blockchain network layer, which aims to build a blockchain network and ensure node communication and data synchronization, 4.2 data security and privacy protection, to ensure that learning performance cannot be tampered with, and the process of cultivating safety production awareness cannot be tampered with. The completion and storage process of the blockchain needs to ensure that each block represents each step and safety operation requirement, and truly records the operator's entire operation process and completion status.

[0084] The third part of the central control system is the display function, including: 5. Augmented reality generation and rendering module and 6. User interaction interface, wherein 5. Augmented reality generation and rendering module includes, 5.1 combining operation effects to generate virtual images and 5.2 generating virtual information prompts to be displayed on the augmented reality glasses 36 and the projection screen 33, wherein the design of part 5.2 requires superimposing it on reality and displaying it through the augmented reality glasses 36; 6. User interaction interface design requires including interface element content, the location of receiving and sending information, ensuring volume adjustment, and especially the design of error reporting and early warning.

[0085] Specifically, the core control modules of the first part of the central control system include 2.1 central control system execution process and 2.2 decision logic analysis. In 2.1 central control system execution process, the operator operates according to the scene and prompts, the augmented reality glasses 36 display the augmented reality scene according to the settings, the hardware equipment moves and senses pressure, the camera 3 and the sensor capture and sense the transmission data, the central control system supervises and manages in real time, the data is analyzed and sorted, the virtual image of the operation screen is realized, the virtual image feedback is displayed in real time in the operator's glasses, and the operator continuously adjusts and improves according to the display situation. 2.2 Decision logic analysis includes five parts: 1) Data collection and preprocessing, operation model touch and other feedback input signals, including operator posture, strength, prop position, equipment status parameter feedback requires comparison with preset data, analysis effect, and preprocessing. Three aspects need to be processed, including: judging whether there is a problem with the augmented reality glasses 36 and the equipment. If the data deviation is large, the augmented reality glasses 36 will directly prompt an early warning. If there is a serious deviation or safety does not meet the requirements, a stop command will be directly issued; 2) The application of logic rules and algorithms requires a detailed analysis of the preprocessed data, including prompts for data deviations, and conversion into prompts such as actions and operations. On the other hand, the algorithm application requires the setting of The design uses decision trees and network algorithms to make overall operational decisions; 3) Decision generation and evaluation, including three aspects of design, namely: how to judge the operator's level, whether to operate according to the prompts of the augmented reality glasses 36, and whether prompts are needed; through prompts, further improve the improvement situation; summarize what problems exist in order to further generate transcripts; 4) Real-time feedback and adjustment, the design includes issuing notifications, whether to adjust the operation, and generating transcripts of the operation results, and generating pictures of the printed completed results; 5) User interaction and visualization, requiring the interactive interface to provide effective parameter support, and provide screen creation for the augmented reality glasses 36, projection screen 33, and central control computer display.

[0086] In the description of this specification, the description with reference to the terms "preferred embodiment", "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0087] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An engineering technology training system based on augmented reality, characterized in that: It includes a central control system, an overall frame with operating positions, a simulation module and an auxiliary system, wherein the simulation module and the auxiliary system are respectively connected to the central control system; The simulation module includes simulation physical operation equipment, monitoring and interactive equipment; The monitoring and interaction equipment includes a camera for acquiring the operator's operation process and augmented reality glasses for augmented reality performance; The simulated physical operation equipment includes a conveyor belt arranged on the overall frame, a grinding wheel cutting model arranged on the overall frame, and a tactile tool arranged on the overall frame for an operator to perform engineering technical operations. The conveyor belt is a lifting conveyor belt, and the conveyor belt serves as a workbench. The operator at the operating position wears the augmented reality glasses, and simulates engineering technical operations by picking up the tactile tool and operating the virtual simulated engineering materials on the conveyor belt. The operator can simulate cutting operations at the grinding wheel cutting model through the virtual simulated engineering materials. The surface of the conveyor belt is provided with a sensor for transmitting the pressing effect of the tactile tool, and the conveyor belt can be raised and lowered to simulate the effect of continuous improvement of brick wall construction; The central control system includes a central control computer, a display and a projection device; The camera and sensor transmit information to the central control computer of the central control system, and the screen of the engineering technology operation is presented on the augmented reality glasses and the projection screen of the projection device, and the operator adjusts the operation according to the screen in the augmented reality glasses; The auxiliary system includes a broadcasting speaker connected to the central control system for emitting sound.

2. The engineering technology training system based on augmented reality according to claim 1, characterized in that: The tactile tool comprises a plurality of tool models hung on an augmented reality hanger, wherein the augmented reality hanger is an annular hanger, and the tool models are hung on the annular hanger by a retractable hanging rope and can rotate circumferentially on the annular hanger; A touch sensor is arranged at the end of the tool model, and the conveyor belt and the grinding machine cutting model are auxiliary props. A touch piece capable of interacting with the touch sensor to generate a touch sense is arranged on the auxiliary prop.

3. The engineering technology training system based on augmented reality according to claim 2, characterized in that: A telescopic frame is provided in the middle of the augmented reality hanger, and telescopic top plates that can be pulled laterally are installed on the left and right sides of the telescopic frame, and a lower stretching plate that can be pulled vertically downward is installed at the lower end of the telescopic frame. The telescopic top plate and the lower stretching plate are auxiliary props provided with the touch member, and the telescopic top plate and the lower stretching plate are used to assist virtual plastering and wall painting.

4. The engineering technology training system based on augmented reality according to claim 2, characterized in that: A plurality of falling blocks for simulating falling objects from high altitude are suspended on the augmented reality hanger, and the falling blocks are pull-wire type falling blocks which are suspended by pull wires and can fall and rise.

5. The engineering technology training system based on augmented reality according to claim 1, characterized in that: It also includes a lifting foot pedal for simulating the working feeling of high-altitude construction. A pedal groove is provided at the operating position of the overall frame. Several groups of lifting springs are provided in the pedal groove. The lower end of the lifting foot pedal is connected to and supported by the upper end of the lifting spring. By extending and retracting several lifting springs, the lifting foot pedal can be raised and lowered and swung left and right.

6. The engineering technology training system based on augmented reality according to claim 5, characterized in that: Each group of the lifting springs includes a spring and a jack located in the spring. The upper end of the spring is fixedly connected to the lower end of the lifting foot pedal. The jack is in contact with or not in contact with the lower end of the lifting foot pedal. By raising and lowering the jack, the spring can be extended and retracted to make the lifting spring extend and retract.

7. The engineering technology training system based on augmented reality according to claim 1, characterized in that: The conveyor belt is installed in a support frame fixed to the overall frame, and a lifting mechanism for driving the conveyor belt to rise and fall in the support frame is installed on the support frame; A plurality of cameras are installed on the top of the support frame.

8. The engineering technology training system based on augmented reality according to claim 7, characterized in that: A vertically extending lifting track is arranged inside the support frame, the conveyor belt has a sliding column which is clamped in the lifting track and slides in the track, and the lifting track is provided with upper and lower partitions which limit the upper and lower movable range of the conveyor belt.

9. The engineering technology training system based on augmented reality according to claim 1, characterized in that: The overall frame is provided with a downwardly concave lower groove, the conveyor belt can be embedded in the lower groove, and the upper surface of the conveyor belt can be flush with the top of the lower groove; A plurality of auxiliary bricks located below the conveyor belt are also placed in the lower groove, and the auxiliary bricks can slide into the lower groove as the conveyor belt runs.

10. An engineering technology training system based on augmented reality according to any one of claims 1-8, characterized in that: The overall frame is composed of a metal frame and a plurality of cover plates covering the metal frame. The overall frame has a high area in the middle and a lower area located around the high area. The operating position, conveyor belt and tactile tool are arranged in the high area, and the grinding machine cutting model, central control computer, display and projector are arranged in the low area.

Images