Tunnel construction hidden project intelligent detection equipment and method based on machine vision

Abstract

The invention provides tunnel construction hidden engineering intelligent detection and analysis equipment and method based on machine vision. The method comprises the following steps: a multimedia data acquisition module is used for acquiring multimedia data; the embedded edge AI application development module is used for preprocessing the multimedia data to obtain input data for an embedded pre-training model, and an output tunnel construction hidden engineering intelligent detection result is obtained; the data storage module is used for storing multimedia data and corresponding intelligent detection results; the network communication module is used for remotely transmitting multimedia data and / or corresponding intelligent detection results and receiving remotely transmitted external instructions; the man-machine interaction module is used for displaying an intelligent detection result and receiving an external instruction input by a user; and the control module is used for processing external instructions and managing the multimedia data acquisition module, the embedded edge AI application development module, the data storage module, the network communication module and the man-machine interaction module.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection and analysis of tunnel construction, and in particular to an intelligent detection device and method for hidden engineering of tunnel construction based on machine vision. Background Art

[0002] Tunnel construction work surface analysis is a key technical management task in the tunnel construction process. It covers the comprehensive evaluation and prediction of the geological conditions in front of the tunnel excavation face, the stability of the surrounding rock, potential risks (such as landslides, water gushing, gas outbursts, etc.) and the construction technology plan. The main purpose of the work surface analysis is to ensure the safety and efficiency of tunnel excavation. Through detailed geological forecasts, on-site monitoring and measurement, and data analysis, it provides a scientific basis for construction decisions. Its content includes but is not limited to the following two aspects: 1. Geological prediction: understand the rock properties, structure, and joint fissure development in front, and analyze and prevent possible adverse geological problems based on this. 2. Support detection and optimization: The role of tunnel support is to ensure the stability and safety of the tunnel structure, prevent the collapse and deformation of the surrounding rock, and analyze whether the support construction is carried out in accordance with the design standards. It is directly related to the safety of tunnel operations.

[0003] Regarding the inspection process of support construction on the tunnel construction working surface, the current situation is that the staff regularly go to the site to check the situation, manually measure the support spacing and record the data, and after the unified summary of the information is reported, a dedicated person will check the reported measurement data and design standards. This process is inefficient and cannot timely discover the possibility of safety accidents caused by construction that does not meet the design requirements. In addition, manual measurement and recording are prone to errors and omissions. Some supports on site are at a high position. Manual measurement is not only time-consuming and labor-intensive, but also prone to safety hazards.

[0004] Regarding the analysis of the surrounding rock of the tunnel construction working face, the current situation is that the staff regularly go to the site to collect photo data of the surrounding rock of the tunnel face. After summarizing the data, experienced technicians analyze the surrounding rock photos and issue results. This process is similar to the above-mentioned support data collection, and it also has the disadvantages of low efficiency, failure to find problems in time, and incomplete data collection.

[0005] In summary, the problems existing in the detection of support rod construction include: (1) the support rod is too high to be measured or the measurement is difficult; (2) it is impossible to judge whether the project meets the design requirements in time; (3) the working distance adjacent to the tunnel face is dangerous; (4) the manual statistics of on-site technicians are slow, the data are prone to errors or omissions, and the efficiency of paper data processing is low. The problems existing in the detection of tunnel face surrounding rock include: (1) it is impossible to judge the surrounding rock condition of the tunnel face in time; (2) the manual data collection of on-site technicians is prone to errors or omissions, and the efficiency of paper data processing is low.

[0006] Therefore, how to focus on the lightweight intelligent detection technology and equipment for the quality of hidden projects in railway construction, and propose a highly integrated and comprehensive intelligent detection and analysis equipment and method for hidden projects in tunnel construction based on machine vision to automatically realize the quality identification and detection of hidden railway projects, is a technical problem that needs to be solved urgently. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides an intelligent detection device and method for hidden engineering in tunnel construction based on machine vision to eliminate or improve one or more defects existing in the prior art.

[0008] One aspect of the present invention provides an intelligent detection device for hidden engineering of tunnel construction based on machine vision, the device comprising: a multimedia data acquisition module, which is used to collect multimedia data including structured light patterns, color images and sounds in the hidden engineering environment of tunnel construction, and send the collected multimedia data to an embedded edge AI application development module; an embedded edge AI application development module, which is used to preprocess the multimedia data to obtain input data for an embedded pre-trained model, and obtain an output intelligent detection result of hidden engineering of tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a tunnel face surrounding rock state analysis model, and the intelligent detection result includes a support rod spacing detection result and / or a tunnel face surrounding rock state analysis result; a data storage module, which is used to store multimedia data and corresponding intelligent detection results; a network communication module, which is used to remotely transmit multimedia data and / or corresponding intelligent detection results, and to receive external instructions transmitted remotely; a human-computer interaction module, which is used to display the intelligent detection results, and to receive external instructions input by a user; a control module, which is used to process external instructions, and to manage the multimedia data acquisition module, the embedded edge AI application development module, the data storage module, the network communication module and the human-computer interaction module.

[0009] In some embodiments of the present invention, the multimedia data acquisition module includes a structured light camera, an RGB camera and a microphone; wherein the structured light camera is used to collect structured light patterns in a hidden engineering environment of tunnel construction, the RGB camera is used to collect color images in the hidden engineering environment of tunnel construction, and the microphone is used to collect sounds in the hidden engineering environment of tunnel construction.

[0010] In some embodiments of the present invention, the training set of the support pole spacing detection model includes support pole depth images, color images and annotated bounding box labels extracted based on structured light patterns; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted through input data.

[0011] In some embodiments of the present invention, the training set of the tunnel face surrounding rock state analysis model includes color images of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of broken state, weathered state and water seepage state.

[0012] In some embodiments of the present invention, the data storage module includes an SSD solid state drive, which is used to store multimedia data and corresponding intelligent detection results; the network communication module includes a WiFi and Bluetooth module and a 4G wireless routing module, which are respectively used for data communication under different network communication conditions.

[0013] In some embodiments of the present invention, the intelligent detection device also includes an LED fill light board, a shutter switch and an external sleep switch. The LED fill light board is used to fill light when collecting the multimedia data, the shutter switch is used to adjust the exposure when collecting structured light patterns and / or color images, and the external sleep switch is used to control the operation and sleep of the multimedia data acquisition module; the control module includes a control board, and the control board includes a power supply unit, a light source driving unit, a switch detection unit and a communication unit. The power supply unit is used to meet the power requirements of each structure on the control board in terms of voltage and power. The light source driving unit is used to drive the fill light board, the switch detection unit is used to detect the status of the power supply unit and the shutter switch, and the communication unit is used to communicate bidirectionally with the embedded edge AI application development module via UART.

[0014] In some embodiments of the present invention, the intelligent detection device further comprises an external housing with a passive heat dissipation design, wherein the external housing comprises front and rear covers, a top cover, a plastic lined heat conductive block, a power button and a shutter button.

[0015] Corresponding to the above method, the present invention also provides an intelligent detection method for hidden engineering of tunnel construction based on machine vision, the method comprising: a multimedia data acquisition module acquires multimedia data including structured light patterns, color images and sounds in the hidden engineering environment of tunnel construction, and sends the acquired multimedia data to an embedded edge AI application development module; the embedded edge AI application development module preprocesses the multimedia data to obtain input data for an embedded pre-trained model, and obtains an output intelligent detection result of hidden engineering of tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a tunnel face surrounding rock state analysis model, and the intelligent detection result includes a support rod spacing detection result and / or a tunnel face surrounding rock state analysis result; the data storage module stores multimedia data and corresponding intelligent detection results; the network communication module remotely transmits multimedia data and / or corresponding intelligent detection results, and receives external instructions transmitted remotely; the human-computer interaction module displays the intelligent detection results, and receives external instructions input by the user; the control module processes external instructions, and manages the multimedia data acquisition module, the embedded edge AI application development module, the data storage module, the network communication module and the human-computer interaction module.

[0016] In some embodiments of the present invention, the method also includes a pre-training step for a support pole spacing detection model. During the training process, the training set of the support pole spacing detection model includes support pole depth images, color images and annotated bounding box labels extracted based on structured light patterns; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted by the input data.

[0017] In some embodiments of the present invention, the method also includes a pre-training step for a tunnel face surrounding rock state analysis model. During the training process, the training set of the tunnel face surrounding rock state analysis model includes a color image of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of broken state, weathered state and water seepage state.

[0018] The intelligent detection equipment for hidden engineering projects in tunnel construction based on machine vision proposed in the present invention can automatically collect multimedia data using a multimedia data acquisition module, and use a pre-trained model embedded in an embedded edge AI application development module to realize automated intelligent detection in hidden engineering environments of tunnel construction based on the collected multimedia data, thereby saving manpower and material resources to a great extent, reducing the working time of on-site technicians at the tunnel construction site, and thus reducing the danger.

[0019] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0020] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0022] Figure 1 The present invention is a block diagram of the structure of an intelligent detection device for hidden engineering in tunnel construction based on machine vision in one embodiment of the present invention.

[0023] Figure 2 This is a structural block diagram of an intelligent detection device for hidden engineering in tunnel construction based on machine vision in another embodiment of the present invention.

[0024] Figure 3 The present invention is an overall flow chart of an intelligent detection method for hidden engineering in tunnel construction based on machine vision in one embodiment of the present invention.

[0025] Figure 4 FIG. 4 is a hardware layout diagram of an intelligent detection device in one embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the hardware structure of the intelligent detection device in the specific embodiment 1 of the present invention.

[0027] Figure 6 This is a schematic diagram of the hardware structure of the intelligent detection device in the specific embodiment 2 of the present invention.

[0028] Figure 7 This is a schematic diagram of the hardware structure of the intelligent detection device in specific embodiment 3 of the present invention.

[0029] Figure 8 This is a schematic diagram of the hardware structure of the intelligent detection device in specific embodiment 4 of the present invention.

[0030] Fig. 9 This is a flow chart of applying intelligent detection equipment to support analysis in one embodiment of the present invention.

[0031] Fig.10 The present invention is a flowchart of an intelligent detection device applied to the surrounding rock analysis of a tunnel face in one embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0033] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0034] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0035] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0036] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0037] In order to overcome the problems existing in the prior art, the present invention proposes an intelligent detection and analysis device and method for hidden engineering in tunnel construction based on machine vision. The device and method are based on the automatic extraction technology of the structure and parameter information of railway hidden engineering components based on machine vision, and design a target object detection algorithm and model for application scenarios and a hidden engineering structure status and key parameter acquisition and identification system to continuously promote the "de-hiding" of hidden engineering and improve the quality management level of hidden engineering in railway engineering. The device can use artificial intelligence algorithm analysis methods to preliminarily process the raw data collected on site, generate reports, and provide the reports to project stakeholders for use.

[0038] Figure 1 This is a block diagram of the structure of an intelligent detection device for hidden engineering in tunnel construction based on machine vision in one embodiment of the present invention. The intelligent detection device includes:

[0039] (1) A multimedia data acquisition module 100, which is used to collect multimedia data including structured light patterns, color images and sounds in a hidden engineering environment of tunnel construction, and send the collected multimedia data to an embedded edge AI application development module.

[0040] (2) An embedded edge AI application development module 200, which is used to preprocess the multimedia data to obtain input data for an embedded pre-trained model, and obtain output intelligent detection results of hidden engineering projects in tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a tunnel face surrounding rock state analysis model, and the intelligent detection results include support rod spacing detection results and / or tunnel face surrounding rock state analysis results.

[0041] (3) Data storage module 300, used to store multimedia data and corresponding intelligent detection results.

[0042] (4) Network communication module 400, used for remotely transmitting multimedia data and / or corresponding intelligent detection results, and for receiving external instructions transmitted remotely.

[0043] (5) A control module 500, which is used to process external instructions and manage the multimedia data acquisition module, the embedded edge AI application development module, the data storage module, the network communication module and the human-computer interaction module.

[0044] (6) Human-computer interaction module 600, used to display intelligent detection results and to receive external instructions input by users.

[0045] In the specific implementation process, the human-computer interaction module may include a touch LCD screen and an external speaker. For example, the touch LCD screen may be an 8-inch touch LCD screen as a hardware interface for human-computer interaction.

[0046] The intelligent detection equipment for hidden engineering projects in tunnel construction based on machine vision proposed in the present invention can automatically collect multimedia data using a multimedia data acquisition module, and use a pre-trained model embedded in an embedded edge AI application development module to realize automated intelligent detection in hidden engineering environments of tunnel construction based on the collected multimedia data, thereby saving manpower and material resources to a great extent, reducing the working time of on-site technicians at the tunnel construction site, and thus reducing the danger.

[0047] In some embodiments of the present invention, the multimedia data acquisition module includes a structured light camera, an RGB camera and a microphone; wherein the structured light camera is used to collect structured light patterns in a hidden engineering environment of tunnel construction, the RGB camera is used to collect color images in the hidden engineering environment of tunnel construction, and the microphone is used to collect sounds in the hidden engineering environment of tunnel construction.

[0048] In the specific implementation process, a Gemini 2L structured light camera can be used to connect to the embedded edge AI application development module via a USB interface to capture structured light patterns for 3D modeling and measurement of tunnel support. An automatic fast focus 4K RGB USB camera (with integrated digital MIC) can be used to connect to the embedded edge AI application development module via a USB interface to provide high-resolution color images for AI analysis and recording of rock types and structures at the face. The digital MIC is used to pick up sound during video recording.

[0049] By adopting the embodiment of the invention, the structured light camera, RGB camera and microphone included in the multimedia data acquisition module can be used to collect multimedia data, thereby realizing the target function of the device designed by the present invention.

[0050] In some embodiments of the present invention, the training set of the support pole spacing detection model includes support pole depth images, color images and annotated bounding box labels extracted based on structured light patterns; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted by input data.

[0051] In the specific implementation process, the model of the embedded edge AI application development module can be NVIDIA JetsonOrin NX 16G / Jetson Orin Nano 8GB. The embedded edge AI application development module serves as a central processing unit, connecting all components and responsible for the operation of AI algorithms and image processing.

[0052] The beneficial effects of using the device proposed in the present invention to detect support rod construction by using the embodiment of the invention are as follows: (1) it can assist on-site technicians in measuring support spacing; (2) it can determine on-site whether the support spacing of the construction meets the design requirements; (3) it can reduce the working time of on-site technicians at the face work point and reduce the danger; (4) data can be archived through the application and uploaded through the network, thereby improving the efficiency of data collection and archiving.

[0053] In some embodiments of the present invention, the training set of the tunnel face surrounding rock state analysis model includes color images of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of broken state, weathered state and water seepage state.

[0054] The beneficial effects of using the device proposed in the present invention to detect the surrounding rock of the tunnel face are as follows: (1) it can assist on-site technicians in analyzing and judging the surrounding rock conditions and assist geological prediction accordingly; (2) data can be archived through the application and uploaded through the network, thereby improving the efficiency of data collection and archiving.

[0055] In some embodiments of the present invention, the data storage module includes an SSD solid state drive, which is used to store multimedia data and corresponding intelligent detection results; the network communication module includes a WiFi and Bluetooth module and a 4G wireless routing module, which are respectively used for data communication under different network communication conditions.

[0056] In the specific implementation process, the 4G wireless routing module is an LTE router, which is used to connect to the embedded edge AI application development module through the Ethernet interface, provide wireless network connection under Wi-Fi conditions, and realize remote data transmission.

[0057] In the specific implementation process, the WiFi and Bluetooth modules can be a Wi-Fi / BT two-in-one network card, which is connected to the embedded edge AI application development module through the PCI-E interface. Wi-Fi provides wireless network connection to realize remote data transmission, and BT is used for short-range wireless data transmission between mobile phones.

[0058] By adopting the embodiment of the invention, the data storage module can be used to realize data archiving and uploading, thereby improving the efficiency of data collection and archiving.

[0059] In some embodiments of the present invention, the intelligent detection equipment further includes an LED fill light board, a shutter switch and an external sleep switch, wherein the LED fill light board is used for fill light when collecting the multimedia data, the shutter switch is used for adjusting the exposure when collecting structured light patterns and / or color images, and the external sleep switch is used to control the operation and sleep of the multimedia data acquisition module.

[0060] In other embodiments of the present invention, the control module includes a control board, which includes a power supply unit, a light source driving unit, a switch detection unit and a communication unit. The power supply unit is used to meet the power requirements of various structures on the control board in terms of voltage and power. The light source driving unit is used to drive the fill light board. The switch detection unit is used to detect the status of the power supply unit and the shutter switch. The communication unit is used to perform bidirectional communication with the embedded edge AI application development module via UART.

[0061] In the specific implementation process, the control board is connected to the embedded edge AI application development module through UART, which is responsible for the power management of all components, key switch status detection and dimming control of the fill light. The onboard circuit on the control board of the device includes a power supply unit, a light source drive unit, a switch detection unit and a communication unit. The core is a low-power 32-bit microcontroller STM32L051C8T6 ARM MCU based on the ARM architecture.

[0062] In the specific implementation process, the functional description of each part of the onboard circuit includes: (1) Power supply unit: The device is powered by a 12V / 20Ah ternary lithium battery and is charged using a waterproof TYPE-C interface external charger. The power supply unit uses a DC / DC method to generate multiple power supplies from the lithium battery to power each component, and the power supply voltage and power meet the power requirements of the above-mentioned units. When not in use, each component can be controlled to be in a dormant state to save energy. The battery power can be monitored in real time and turned on and off through software. (2) Light source driving unit: Drives the fill light board in a constant current manner and dims in a PWM manner. (3) Switch detection unit: Detects the status of the power button and shutter button in real time. (4) Communication unit: Communicates bidirectionally with the embedded edge AI application development module through UART, uploads the detected button switch status in real time, and receives instructions issued by the embedded edge AI application development module to perform corresponding control operations. (5) Microcontroller, used to realize the control of the power supply unit, light source driving unit, switch detection unit and communication unit.

[0063] In the specific implementation process, the LED fill light board can be controlled through the PWM interface on the control board to provide an LED light source with adjustable brightness to enhance the image quality under low light conditions. You can also use a high CRI LED white light source with an oblong layout and a total light source power of 16W, which is driven and dimmed by the control board.

[0064] By adopting the embodiment of the invention, a control panel and a variety of control switches can be used to implement a variety of control operations and realize intelligent detection in different application scenarios.

[0065] In some embodiments of the present invention, the intelligent detection device further comprises an external housing with a passive heat dissipation design, wherein the external housing comprises front and rear covers, a top cover, a plastic lined heat conductive block, a power button and a shutter button.

[0066] In the specific implementation process, the external housing can be formed by CNC processing of 6061 aluminum alloy and also serve as a passive heat sink for the embedded edge AI application development module. The original fan-type active heat sink of the embedded edge AI application development module is not suitable for the application scenario of the terminal, and passive heat dissipation must be adopted. The housing design takes into account dustproof, waterproof and impact resistance to adapt to the use environment. The external housing consists of front and rear covers, top cover, plastic lined heat conductive block, power button and shutter button.

[0067] By adopting the embodiment of the invention, the external casing with a passive heat dissipation design can achieve dustproof, waterproof and impact-resistant effects, and can be better suitable for intelligent detection in tunnel environments.

[0068] Figure 2This is a block diagram of the structure of the intelligent detection equipment for hidden engineering in tunnel construction based on machine vision in another embodiment of the present invention. The intelligent detection equipment for hidden engineering in tunnel construction is a portable complete device, which is implemented based on an embedded software and hardware platform. It includes the following main components: the core processing unit is an embedded edge AI application development module 200 (NVIDIA Jetson Orin NX 16G / Jetson Orin Nano 8GB), an 8-inch touch LCD screen 610, an external speaker 620, an Occipital Gemini 2L structured light camera 110, an automatic fast focus 4K RGB USB camera 120 (integrated digital MIC), an SSD solid state drive 310 based on NVME technology, a Wi-Fi / BT two-in-one network card 410 based on PCIe technology, an LTE 4G wireless routing module 420, a control board 510, an LED fill light board, a power switch and a shutter switch. These components together constitute a complete tunnel hidden engineering detection system, which can achieve efficient and accurate tunnel detection.

[0069] in, Figure 2 The 4G wireless routing module 420 shown is connected to the main antenna and the auxiliary antenna, the WiFi and Bluetooth module 410 is also connected to the main antenna and the auxiliary antenna, and the control board 510 is connected to the LED fill light board, the shutter switch and the peripheral sleep switch. The touch LCD screen 610 transmits the touch screen input signal to the AI ​​application development module 200 via USB, and receives the display signal from the AI ​​application development module 200 via DP.

[0070] Correspondingly, the assembly and operation of the intelligent detection equipment proposed in the present invention include: the touch LCD screen is fixed on the surface of the plastic lining, the plastic lining is embedded in the back cover, the charging TYPE-C seat is fixed on the bottom of the back cover, the embedded edge AI application development module (NVIDIAJetson Orin NX 16G / Jetson Orin Nano 8GB), Gemini 2L structured light camera, 4KRGB USB camera, LTE routing board and control board are fixed on the top surface of the plastic lining as shown in the layout diagram, and each component is connected according to the wiring diagram. The heat conduction block is fixed on the PCB of the embedded edge AI application development module, the fill light board is fixed to the front cover lamp cavity with screws, the front cover is covered, and the front cover is fixed to the heat conduction block and the front cover with screws, and the heat conduction block is coated with thermal grease to effectively conduct heat to the front cover with a larger heat dissipation area. The power and shutter keys are respectively fixed to the inclined positions on the left and right sides of the top cover, and the key caps are embedded in the key holes before fixing. The Wi-Fi and LTE antennas are respectively fixed to the left and right sides of the top surface of the top cover, and the top cover is fixed to the front and rear covers with screws.

[0071] Figure 3The overall flow chart of the intelligent detection method for hidden engineering in tunnel construction based on machine vision in one embodiment of the present invention includes the following steps: (1) First, the device is turned on and the application is entered; (2) The camera function is entered to collect data; (3) The target detection algorithm is used to analyze the supporting image and output the analysis result; The deep learning algorithm is used to analyze the surrounding rock image data and output the analysis result; (4) The output analysis result is manually checked to see if there is any error. If there is, the deviation is manually marked to correct it. If there is no error, it is saved in the report and archived; (5) Exit the application and the process ends.

[0072] Figure 4 FIG. 4 is a hardware layout diagram of an intelligent detection device in one embodiment of the present invention. Figure 4 The embodiment shown includes a charging port and a battery buckle, the network communication module also includes a SIM card slot, the onboard interfaces include a UART port, an LCD power supply, a camera power supply, an AI motherboard power box network port, the structured light camera is adjacent to the LED fill light board in the structure, and the 4K RGB camera is also adjacent to the microphone. In addition, the onboard interfaces also include a TYPE-C socket, an RJ45 socket, a USB port, a DP socket, a DC socket, and so on.

[0073] Figure 5 This is a schematic diagram of the hardware structure of the intelligent detection device in the specific embodiment 1 of the present invention. Figure 5 The position of the front cover and the sound hole are shown in the figure, both of which are located on the front cover of the external shell. The 4K RGB camera is adjacent to the sound hole. The LED fill light board and the structured light camera are located on the same side of the external shell. In addition, the device also includes an antenna, which is located on the same side of the device and extends outward.

[0074] Figure 6 This is a schematic diagram of the hardware structure of the intelligent detection device in the specific embodiment 2 of the present invention. Figure 6 The image shows that the battery is located at the bottom of the shell, and inside the shell are closely arranged the control board, 4G wireless routing module, shutter switch, main and auxiliary antennas, structured light camera, 4K RGB camera and microphone, peripheral sleep switch, heat conduction plate and embedded edge AI application development module.

[0075] Figure 7 This is a schematic diagram of the hardware structure of the intelligent detection device in specific embodiment 3 of the present invention. Figure 7 The middle is the back cover corresponding to the front cover, on which a touch LCD screen and sound holes are arranged.

[0076] Figure 8 This is a schematic diagram of the hardware structure of the intelligent detection device in specific embodiment 4 of the present invention. Figure 8 The middle shows the layout inside the back cover. There is an external speaker above the touch LCD screen and a battery below it.

[0077] The following two specific implementation methods are used to illustrate the intelligent detection of hidden projects in tunnel construction based on machine vision.

[0078] Implementation method 1: Support rod spacing detection.

[0079] First, the training preparation needs to be pre-loaded into the intelligent detection equipment. It includes the following steps: (1) Data preparation: Collect data, use a structured light camera to collect enough RGB images and corresponding depth images. Label the data, annotate the bounding box for the target object in each RGB image, and record the position information of the support rod in the depth image, so as to calculate the distance between objects later. (2) Model training: Preprocessing, convert the RGB image and depth image into a format suitable for YOLO model input. Scale the image to a uniform size and normalize the depth image. Fusion RGB and depth information, design a network structure, add additional branches to process depth information on the basis of YOLO, or fuse the RGB image and depth image into a four-channel input. Training model, use the labeled data set to train the YOLO model to ensure that the model can accurately detect the support rod and output its position information. (3) Distance calculation: Extract bounding box, extract the bounding box coordinates of each detected object from the output of the YOLO model. Obtain depth value, according to the corresponding position of the bounding box in the depth image, obtain the average depth or center point depth value of each object. Calculate the distance: Use the depth values ​​of the two objects and their relative positions in the image to calculate the actual distance between the objects through the triangulation principle or the camera's intrinsic parameter matrix. (4) Result optimization: Post-processing: Perform post-processing operations such as filtering on the detection results to improve detection accuracy and stability. Error analysis: Analyze the detection errors in the application scenario and continuously adjust the model parameters to optimize performance. (5) Practical application: Prune and quantize the model and embed it into the device to achieve real-time analysis.

[0080] Fig. 9 This is a flow chart of applying intelligent detection equipment to support analysis in one embodiment of the present invention.

[0081] Steps for detecting the spacing between support rods: 1. Turn on the device and log in to the application interface; 2. The user clicks the "Image Capture" button in the application; 3. The user enters the necessary basic information and selects the image type as "support"; 4. The user selects either circumferential or longitudinal according to the direction of the support rod; 5. Next, select the source of the image; 6. Here you can choose local album pictures or pictures taken on site; 7. If the user chooses to take a new picture, the device lens will be opened, and the user will confirm that the taken picture is clear and usable after the shooting is completed. Then deliver the algorithm for recognition and analysis; 8. If the user wants to use photos that have been collected, select one from the local album and confirm to deliver it to the algorithm for recognition and analysis; 9. The algorithm replies with the processing results to generate a previewable support report; 10. The user views the report and confirms whether it is available; 11. If the user confirms that it is available, the process continues to step 15. 12. If the user confirms that the report is unavailable, the report data can be manually corrected; 13. The user needs to make dot marks on the original image, and then deliver the photos and annotation content to the algorithm for analysis and calculation again; 14. The algorithm returns the processing results again to generate a previewable support report. This process can be repeated until the user confirms that the generated analysis report is available; 15. The user chooses to export the analysis report, which is saved locally in PDF format; 16. The application exit process ends.

[0082] The beneficial effects of using the equipment proposed in the present invention to inspect support rod construction are as follows: (1) It can assist on-site technicians in measuring support spacing; (2) It can determine on-site whether the support spacing during construction meets the design requirements; (3) It can reduce the working time of on-site technicians at the face work point and reduce the risk; (4) Data can be archived through applications and uploaded through the network, thereby improving the efficiency of data collection and archiving.

[0083] Implementation method 2: analysis of surrounding rock at the tunnel face.

[0084] First, the training preparation needs to be pre-loaded into the intelligent detection equipment. The following steps are included: 1. Environment preparation: Install and set up the development environment: opencv-python, numpy, matplotlib, scikit-image, tensorflow, Pillow, shapely. 2. Data collection and preprocessing: Data collection, use a high-resolution camera to take images of surrounding rock samples under different conditions (such as different lighting and angles) to ensure that the data set is representative. Data annotation, annotate the collected images, mark the different properties, structures and joints and cracks of the rocks. Use the tools LabelMe and VGG Image Annotator (VIA) for annotation. Data enhancement, increase the amount of data by rotation, flipping, scaling and other methods to improve the generalization ability of the model. Read the image, use OpenCV or Pillow to read the image. Image normalization, normalize the image pixel values ​​to the range of [0,1]. Resize, adjust the image to a uniform size for input into the deep learning model. 3. Deep learning model design: Select a model, select a suitable deep learning model according to the task requirements, such as U-Net, Mask R-CNN, etc. Model construction, use TensorFlow or Keras to build the model. Compile the model, set the loss function, optimizer and evaluation indicators. 4. Model training: Dataset division, divide the dataset into training set, validation set and test set. Train the model, use the training set to train the model, and regularly evaluate the performance on the validation set. Save the model, save the model weight after training. 5. Model evaluation and optimization: Evaluate the model, evaluate the model performance on the test set, and calculate indicators such as accuracy, recall, and F1 score. Model optimization, adjust the model parameters according to the evaluation results, and optimize the hyperparameters. Model pruning, reduce the model size without significantly reducing its performance. 6. Result visualization: Predict the results, use the trained model to predict new images. Result visualization, use Matplotlib or OpenCV to visualize the prediction results and display the analysis results of rock properties, structures, and joints and fissures. 7. Result interpretation and application: Analyze the output of the model, including the degree of rock fragmentation (complete, relatively complete, relatively fragmented, and fragmented), weathering state (unweathered, weakly weathered, and strongly weathered), and water seepage state (dry, with water seepage, and with water gushing). 8. Model deployment: The model is embedded in this device to achieve real-time analysis.

[0085] Fig.10 The present invention is a flowchart of an intelligent detection device applied to the surrounding rock analysis of a tunnel face in one embodiment of the present invention.

[0086] Operation steps for face surrounding rock analysis: 1. Turn on the device and log in to the application interface; 2. The user clicks the "Image Capture" button in the application; 3. Fill in basic information and select the type of photo to be collected as the face; 4. Next, select the source of the picture; 5. You can choose local album pictures or on-site photos here; 6. If the user chooses to take a new picture, the device lens will be opened. After the user completes the shooting and confirms that the taken photo is clear and usable, it will be delivered to the algorithm for recognition and analysis; 7. If the user wants to use photos that have already been collected, select a photo from the local album and confirm that it will be delivered to the algorithm for recognition and analysis; 8. After the algorithm analysis is completed, a previewable face surrounding rock analysis report is returned; 9. Based on the recognition results, a previewable face surrounding rock analysis report is generated; 10. The user chooses to export the analysis report, which is saved locally in PDF format; 11. The application process ends.

[0087] The beneficial effects of using the equipment proposed in the present invention to detect the surrounding rock of the tunnel face are as follows: (1) It can assist on-site technicians in analyzing and judging the surrounding rock conditions and assist geological prediction accordingly; (2) Data can be archived through the application and uploaded through the network, which improves the efficiency of data collection and archiving.

[0088] Corresponding to the above method, the present invention also provides a method for intelligent detection of hidden engineering in tunnel construction based on machine vision, the method comprising: (1) a multimedia data acquisition module acquires multimedia data including structured light patterns, color images and sounds in the hidden engineering environment of tunnel construction, and sends the acquired multimedia data to an embedded edge AI application development module; (2) the embedded edge AI application development module pre-processes the multimedia data to obtain input data for an embedded pre-trained model, and obtains output intelligent detection results of hidden engineering in tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a face surrounding rock state analysis model, wherein the intelligent detection results include support rod spacing detection results and / or face surrounding rock state analysis results; (3) a data storage module stores multimedia data and corresponding intelligent detection results; (4) a network communication module remotely transmits multimedia data and / or corresponding intelligent detection results, and receives external instructions transmitted remotely; (5) a human-computer interaction module displays the intelligent detection results and receives external instructions input by a user; (6) a control module processes external instructions and manages the multimedia data acquisition module, the embedded edge AI application development module, the data storage module, the network communication module, and the human-computer interaction module.

[0089] In some embodiments of the present invention, the method also includes a pre-training step for a support pole spacing detection model. During the training process, the training set of the support pole spacing detection model includes support pole depth images, color images and annotated bounding box labels extracted based on structured light patterns; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted by the input data.

[0090] In some embodiments of the present invention, the method also includes a pre-training step for a tunnel face surrounding rock state analysis model. During the training process, the training set of the tunnel face surrounding rock state analysis model includes a color image of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of broken state, weathered state and water seepage state.

[0091] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0092] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0093] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent detection device for hidden engineering in tunnel construction based on machine vision, characterized in that: include: A multimedia data acquisition module, used to collect multimedia data including structured light patterns, color images and sounds in a hidden engineering environment of tunnel construction, and send the collected multimedia data to an embedded edge AI application development module; An embedded edge AI application development module is used to pre-process the multimedia data to obtain input data for an embedded pre-trained model, and obtain output intelligent detection results of hidden engineering projects in tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a tunnel face surrounding rock state analysis model, and the intelligent detection results include support rod spacing detection results and / or tunnel face surrounding rock state analysis results; A data storage module, used for storing multimedia data and corresponding intelligent detection results; A network communication module, used for remotely transmitting multimedia data and / or corresponding intelligent detection results, and for receiving external instructions transmitted remotely; Human-computer interaction module, used to display intelligent detection results and receive external instructions input by users; The control module is used to process external instructions, and to manage the multimedia data acquisition module, embedded edge AI application development module, data storage module, network communication module and human-computer interaction module.

2. The intelligent detection device according to claim 1, characterized in that: The multimedia data acquisition module includes a structured light camera, an RGB camera and a microphone; wherein the structured light camera is used to collect structured light patterns in a hidden engineering environment of tunnel construction, the RGB camera is used to collect color images in the hidden engineering environment of tunnel construction, and the microphone is used to collect sounds in the hidden engineering environment of tunnel construction.

3. The intelligent detection device according to claim 1, characterized in that: The training set of the support pole spacing detection model includes support pole depth images, color images and annotated bounding box labels extracted based on structured light patterns; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted by input data.

4. The intelligent detection device according to claim 1, characterized in that: The training set of the tunnel face surrounding rock state analysis model includes color images of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of broken state, weathered state and water seepage state.

5. The intelligent detection device according to claim 1, characterized in that: The data storage module includes an SSD solid state hard disk, and the SSD solid state hard disk is used to store multimedia data and corresponding intelligent detection results; The network communication module includes WiFi and Bluetooth modules and a 4G wireless routing module, which are respectively used for data communication under different network communication conditions.

6. The intelligent detection device according to claim 1, characterized in that: The intelligent detection device further comprises an LED fill light board, a shutter switch and an external sleep switch, wherein the LED fill light board is used for fill light when collecting the multimedia data, the shutter switch is used for adjusting the exposure when collecting structured light patterns and / or color images, and the external sleep switch is used for controlling the operation and sleep of the multimedia data collection module; The control module includes a control board, which includes a power supply unit, a light source driving unit, a switch detection unit and a communication unit. The power supply unit is used to meet the power requirements of various structures on the control board in terms of voltage and power. The light source driving unit is used to drive the fill light board. The switch detection unit is used to detect the status of the power supply unit and the shutter switch. The communication unit is used to perform bidirectional communication with the embedded edge AI application development module via UART.

7. The intelligent detection device according to claim 1, characterized in that: The intelligent detection device also includes an external casing adopting a passive heat dissipation design, and the external casing includes front and rear covers, a top cover, a plastic-lined heat-conducting block, a power button and a shutter button.

8. An intelligent detection method for hidden engineering in tunnel construction based on machine vision, characterized in that: The method comprises: The multimedia data acquisition module acquires multimedia data including structured light patterns, color images and sounds in the hidden engineering environment of tunnel construction, and sends the acquired multimedia data to the embedded edge AI application development module; The embedded edge AI application development module preprocesses the multimedia data to obtain input data for the embedded pre-trained model, and obtains output intelligent detection results of hidden engineering in tunnel construction; wherein the pre-trained model includes a support rod spacing detection model and a tunnel face surrounding rock state analysis model, and the intelligent detection results include support rod spacing detection results and / or tunnel face surrounding rock state analysis results; The data storage module stores multimedia data and corresponding intelligent detection results; The network communication module remotely transmits multimedia data and / or corresponding intelligent detection results, and receives external instructions transmitted remotely; The human-computer interaction module displays intelligent detection results and receives external commands input by users; The control module processes external commands and manages the multimedia data acquisition module, embedded edge AI application development module, data storage module, network communication module and human-computer interaction module.

9. The intelligent detection method according to claim 8, characterized in that: The method also includes a pre-training step for a support pole spacing detection model. During the training process, the training set of the support pole spacing detection model includes a support pole depth image, a color image and annotated bounding box labels extracted based on a structured light pattern; wherein the trained support pole spacing detection model calculates the actual support pole spacing using the bounding box extracted by the input data.

10. The intelligent detection method according to claim 1, characterized in that: The method also includes a pre-training step for a tunnel face surrounding rock state analysis model. During the training process, the training set of the tunnel face surrounding rock state analysis model includes a color image of the tunnel face surrounding rock and annotated surrounding rock state labels; wherein the surrounding rock state includes multiple states of a broken state, a weathered state, and a water seepage state.

Images