Surrounding rock detection system and method of tunneling equipment and computing equipment

By designing a surrounding rock detection system in the tunnel boring equipment, using an annular track and data scanning device to collect data and perform image stitching and feature processing, the problems of low surrounding rock detection accuracy and insufficient information utilization in the existing technology are solved, and construction is improved Safety and efficiency have improved the level of equipment intelligence.

CN118941550APending Publication Date: 2024-11-12CHINA RAILWAY ENGINEERING EQUIPMENT GROUP CO LTD

Patent Information

Application Number
CN202411164542.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing tunnel boring equipment has low accuracy and insufficient information utilization in the surrounding rock detection, resulting in reduced excavation efficiency, equipment damage and safety hazards, and the detection results are subjective and hysteresis.

Method used

Design a surrounding rock detection system for tunnel boring equipment, including an annular track, a data scanning device and a computing device, drive the data scanning device to move on the annular track through a transmission mechanism, collect surrounding rock images and point cloud data, and perform image stitching and characteristics. Process, generate joint feature maps, identify joint areas and predict surrounding rock states.

Benefits of technology

It improves the accuracy and reliability of surrounding rock detection, enhances construction safety and efficiency, improves the intelligence level of tunnel boring equipment, and reduces manual inspection costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941550A_ABST
    Figure CN118941550A_ABST
Patent Text Reader

Abstract

The invention provides a surrounding rock detection system and method of tunneling equipment, computing equipment, a storage medium and a program product. The system comprises an annular track, a data scanning device, a transmission mechanism and the computing equipment, the data scanning device is driven by the transmission mechanism to move on the annular track, the data scanning device scans the surrounding rock in the tunnel, and obtained scanning data comprises a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; and the computing device generates a surrounding rock spliced image and a joint feature map based on the scanning data, and obtains disaster information, joint areas, position information, surrounding rock prediction information and the like according to the joint feature map. According to the method, various surrounding rock geological information such as images and point clouds can be fully utilized, the precision and reliability of surrounding rock detection are improved, the safety and efficiency of construction are improved, and the intelligent level of tunneling equipment is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of surrounding rock detection, and in particular to a surrounding rock detection system, method, computing device, storage medium and program product for tunnel boring equipment. Background Art

[0002] Tunnel boring equipment is widely used in the construction of underground tunnel engineering projects. Tunnel boring equipment includes full-section tunnel boring machines and other equipment. The applicability, construction efficiency and cost of full-section tunnel boring machines and other equipment in tunnel construction are significantly affected by the surrounding rock geological conditions. In tunnel engineering, the rock mass is affected by various complex geological conditions, and the surrounding rock geological information is difficult to obtain. In addition, the acquisition of surrounding rock status information is seriously subjective and lagging. At present, there are problems such as insufficient utilization of surrounding rock geological information and low accuracy of surrounding rock detection results in the technology of surrounding rock detection. If the accuracy of the surrounding rock detection results is low, when the formation conditions change, equipment such as full-section tunnel boring machines are prone to problems such as reduced excavation efficiency, tool wear, and equipment damage, which can easily cause major accidents such as jamming, damage, scrapping, and even casualties. Summary of the invention

[0003] The present disclosure provides a surrounding rock detection system, method, computing device, storage medium and program product for tunnel boring equipment.

[0004] According to a first aspect of the present disclosure, there is provided a surrounding rock detection system for tunnel boring equipment, comprising: a circular track, a data scanning device, a transmission mechanism and a computing device; the circular track is installed in a tunnel, and the transmission mechanism is used to drive the data scanning device to move on the circular track; the data scanning device, upon receiving a data acquisition instruction, scans and processes the surrounding rock in the tunnel to obtain scanning data, wherein the scanning data includes a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; the computing device is used to perform correction processing on the data scanning device; perform splicing processing on a plurality of surrounding rock images to generate a surrounding rock splicing image; generate a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data; perform joint recognition processing on the joint feature map to obtain disaster information; obtain joint area and position information, as well as surrounding rock prediction information based on the joint feature map.

[0005] Optionally, the computing device includes: an image stitching module, used to take the first image among the multiple surrounding rock images as the initial image and add it to the image stitching area, and perform multi-scale image fusion optimization processing on the first image; add the second image data among the multiple surrounding rock images as a new image to the image stitching area, align the center of the first image with the center of the second image, and perform edge alignment processing on the second image and the first image; determine the translation distance of the second image based on the entropy weights of the first image, the second image and the pixel points; and stitch the first image and the second image.

[0006] Optionally, the image stitching module is used to perform feature extraction using a multi-scale SURF algorithm during the stitching process, and to determine a transformation matrix using a RANSAC algorithm based on the matching distance of feature points, a matching distance threshold, and a confidence weight; and to perform correction processing on an image stitched together from two or more of the surrounding rock images using an adaptive filtering correction algorithm during the stitching process.

[0007] Optionally, the image stitching module is used to optimize the accuracy of the surrounding rock stitching image according to the offset of each surrounding rock image during the stitching process; determine the timestamp of the surrounding rock stitching image based on the timestamps and timestamp weights of the multiple surrounding rock images; determine the mileage information of the surrounding rock stitching image based on the mileage information and mileage weights of the multiple surrounding rock images; and label the surrounding rock stitching image based on the timestamp and mileage information of the surrounding rock stitching image.

[0008] Optionally, the computing device includes: a joint feature generation module, which is used to use a visible light extraction network to process the surrounding rock splicing image to obtain an RGB feature map corresponding to the surrounding rock splicing image; use a point cloud feature extraction network to process the point cloud data corresponding to the multiple surrounding rock images to obtain a point cloud feature map; upsample the point cloud feature map; and superimpose the RGB feature map and the point cloud feature map through a feature superposition network to generate the joint feature map.

[0009] Optionally, the computing device includes: a surrounding rock joint identification module, which is used to process the joint feature map using a joint identification network to determine the surrounding rock grade of the pixels in the joint feature map; divide the joint feature map into multiple joint feature sub-maps; and when the surrounding rock grade corresponding to the joint feature sub-map is greater than a grade threshold, use a disaster identification model to perform joint identification processing on the joint feature sub-map to obtain the disaster information.

[0010] Optionally, the surrounding rock joint identification module is used to use a joint area identification model to identify the joint feature subgraph to obtain the joint area and the position information of the joint feature subgraph; and use a surrounding rock situation prediction model to predict the joint feature graph to obtain the surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

[0011] Optionally, the data scanning device includes an image acquisition device and a point cloud acquisition device; the computing device includes: a correction module, used to obtain a standard plate image acquired by the image acquisition device for the standard plate, and determine first position information and second position information of the center and positioning hole of the standard plate in the standard plate image; determine third position information and fourth position information of the center and positioning hole of the standard plate in the standard plate point cloud data based on the standard plate point cloud data acquired by the point cloud acquisition device for the standard plate; and perform correction processing on the point cloud acquisition device based on the first position information, the second position information, the third position information and the fourth position information.

[0012] Optionally, a drag chain is installed on the annular track; the transmission mechanism includes a base, a servo drive, a servo motor, a reducer, and a transmission gear; the transmission gear is installed at the bottom of the base, the transmission gear is engaged with the drag chain, and the data acquisition device is installed on the base; the servo motor is connected to the reducer through a coupling, and the reducer is used to drive the transmission gear to rotate; when the driver receives the data acquisition instruction, it drives the servo motor to operate, and drives the transmission gear to rotate through the reducer, so that the base moves on the drag chain.

[0013] According to a second aspect of the present disclosure, there is provided a surrounding rock detection method, which is applied to a surrounding rock detection system of a tunnel boring device, wherein the surrounding rock detection system comprises: a circular track, a data scanning device, a transmission mechanism and a computing device; the circular track is installed in a tunnel, the data scanning device is installed on the circular track, and the transmission mechanism is used to drive the data scanning device to move on the circular track; the data scanning device, upon receiving a data acquisition instruction, scans and processes the surrounding rock in the tunnel to obtain scanning data, wherein the scanning data comprises a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; the surrounding rock detection method is executed in the computing device, comprising: correcting the data scanning device; splicing a plurality of surrounding rock images to generate a surrounding rock splicing image; generating a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data; performing joint recognition processing on the joint feature map to obtain disaster information; and obtaining joint area and position information, as well as surrounding rock prediction information based on the joint feature map.

[0014] Optionally, the stitching processing of multiple surrounding rock images to generate a surrounding rock stitching image includes: taking the first image among the multiple surrounding rock images as the initial image and adding it to the image stitching area, and performing multi-scale image fusion optimization processing on the first image; adding the second image data among the multiple surrounding rock images as a new image to the image stitching area, aligning the center of the first image with the center of the second image, and performing edge alignment processing on the second image and the first image; determining the translation distance of the second image based on the entropy weights of the first image, the second image and the pixel points; and stitching the first image and the second image.

[0015] Optionally, during the stitching process, a multi-scale SURF algorithm is used to extract features, and a RANSAC algorithm is used to determine the transformation matrix based on the matching distance of feature points, matching distance threshold and confidence weight; during the stitching process, an adaptive filtering correction algorithm is used to correct the image stitched together from two or more of the surrounding rock images.

[0016] Optionally, the accuracy of the surrounding rock splicing image is optimized according to the offset of each surrounding rock image during the splicing process; the timestamp of the surrounding rock splicing image is determined based on the timestamps and timestamp weights of the multiple surrounding rock images; the mileage information of the surrounding rock splicing image is determined based on the mileage information and mileage weights of the multiple surrounding rock images; and the surrounding rock splicing image is labeled based on the timestamp and mileage information of the surrounding rock splicing image.

[0017] Optionally, generating a joint feature map based on the surrounding rock stitching image and the corresponding point cloud data includes: using a visible light extraction network to process the surrounding rock stitching image to obtain an RGB feature map corresponding to the surrounding rock stitching image; using a point cloud feature extraction network to process the point cloud data corresponding to the multiple surrounding rock images to obtain a point cloud feature map, and upsampling the point cloud feature map; and superimposing the RGB feature map and the point cloud feature map through a feature superposition network to generate the joint feature map.

[0018] Optionally, performing joint identification processing on the joint feature map to obtain disaster information includes: processing the joint feature map using a joint identification network to determine the surrounding rock grade of pixels in the joint feature map; dividing the joint feature map into multiple joint feature sub-maps; and when the surrounding rock grade corresponding to the joint feature sub-map is greater than a grade threshold, performing joint identification processing on the joint feature sub-map using a disaster identification model to obtain the disaster information.

[0019] Optionally, obtaining the joint area and location information, and surrounding rock prediction information based on the joint feature map includes: using a joint area recognition model to perform identification processing on the joint feature sub-map to obtain the joint area and the location information of the joint feature sub-map; using a surrounding rock situation prediction model to perform prediction processing on the joint feature map to obtain the surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

[0020] Optionally, the data scanning device includes an image acquisition device and a point cloud acquisition device; the correction processing of the data scanning device includes: acquiring a standard plate image acquired by the image acquisition device for the standard plate, and determining first position information and second position information of the center and positioning hole of the standard plate in the standard plate image; determining third position information and fourth position information of the center and positioning hole of the standard plate in the standard plate point cloud data based on acquiring standard plate point cloud data acquired by the point cloud acquisition device for the standard plate; and performing correction processing on the point cloud acquisition device based on the first position information, the second position information, the third position information and the fourth position information.

[0021] According to a third aspect of the present disclosure, there is provided a computing device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the method as described above based on instructions stored in the memory.

[0022] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the instructions are executed by a processor as described above.

[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, wherein the computer program product stores computer instructions, and the instructions are executed by a processor according to the method described above.

[0024] The surrounding rock detection system, method, computing device, storage medium and program product of tunnel boring equipment disclosed in the present invention drive a data scanning device to move on a circular track through a transmission mechanism, and the data scanning device scans and processes the surrounding rock in the tunnel, and the obtained scanning data includes a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; the computing device generates a surrounding rock splicing image and a joint feature map based on the scanning data, and obtains disaster information, joint area, location information and surrounding rock prediction information according to the joint feature map; it can make full use of various surrounding rock geological information such as images and point clouds, improve the accuracy and reliability of surrounding rock detection, improve the safety and efficiency of construction, and improve the intelligence level of tunnel boring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present disclosure will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other purposes and advantages of the present disclosure will be further described below in conjunction with specific embodiments and with reference to the accompanying drawings. In the accompanying drawings, the same or corresponding technical features or components will be represented by the same or corresponding figure marks.

[0026] Figure 1 Schematic diagrams of some embodiments of the tunnel geological detection system according to the present disclosure;

[0027] Figure 2 is a schematic diagram of a transmission system in some embodiments of the tunnel geological detection system according to the present disclosure;

[0028] Figure 3 is a schematic diagram of transmission of scanning data in some embodiments of the tunnel geological detection system according to the present disclosure;

[0029] Figure 4 It is a module schematic diagram of a computing device in some embodiments of the tunnel geological detection system according to the present disclosure;

[0030] Figure 5 is a schematic diagram of a standard plate in some embodiments of the tunnel geological detection system according to the present disclosure;

[0031] Figure 6 A schematic diagram of a flow chart of an embodiment of a surrounding rock detection method according to the present disclosure;

[0032] Figure 7 It is a schematic diagram of a process of generating a surrounding rock splicing image in one embodiment of the surrounding rock detection method disclosed in the present invention;

[0033] Figure 8 It is a schematic diagram of a flow chart of labeling processing in one embodiment of the surrounding rock detection method disclosed in the present invention;

[0034] Fig. 9 A schematic diagram of a process for generating a joint feature map in an embodiment of a surrounding rock detection method according to the present disclosure;

[0035] Fig.10 A schematic diagram of a process for obtaining disaster information in an embodiment of a surrounding rock detection method according to the present disclosure;

[0036] Fig.11 It is a schematic diagram of a flow chart of a correction process for a data scanning device in one embodiment of a surrounding rock detection method according to the present disclosure;

[0037] Fig.12 The module diagram of one embodiment of a computing device according to the present disclosure is shown in FIG. DETAILED DESCRIPTION

[0038] Exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the embodiments are described in the specification. However, it should be understood that many implementation-specific settings must be made in the process of implementing the embodiments in order to achieve the specific goals of the developer, for example, to meet those restrictions related to the device and the service, and these restrictions may vary depending on the implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is only a routine task for those skilled in the art who benefit from the content of this disclosure.

[0039] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0040] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0041] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0042] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0043] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0044] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0045] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0046] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0047] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0048] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0049] In addition, in order to avoid obscuring the present disclosure due to unnecessary details, only the processing steps and / or device structures closely related to at least the scheme according to the present disclosure are shown in the drawings, and other details that are not closely related to the present disclosure are omitted. It should also be noted that similar reference numerals and letters in the drawings indicate similar items, and therefore once an item is defined in one drawing, it does not need to be discussed again for subsequent drawings.

[0050] In some embodiments, the present disclosure provides a surrounding rock detection system for tunnel boring equipment, including a circular track, a data scanning device, a transmission mechanism, a computing device, etc. Tunnel boring equipment is a device with high process complexity, and tunnel excavation equipment can be a variety of equipment, such as a full-section tunnel boring machine, etc. A full-section tunnel boring machine is a large-scale mechanized, automated, integrated tunnel excavation and lining complete set of equipment.

[0051] like Figure 1 As shown, the circular track 1 is installed in the tunnel, and the circular track 1 is installed on a fixed bracket. The size of the circular track 1 is smaller than the design diameter of the tunnel. A drag chain 2 is installed on the circular track 1, and a transmission mechanism 3 can be engaged with the drag chain 2 through gears. The transmission mechanism 3 is used to drive the data scanning device 5 to move on the circular track 1, and the transmission mechanism 3 is powered by a battery 4.

[0052] like Figure 2 As shown, the transmission mechanism 3 includes a base 7, a servo driver 8, a servo motor 9, a reducer 10, a transmission gear 11, etc.; the transmission gear 11 is installed at the bottom of the base 7, the transmission gear 11 is engaged with the drag chain 2, and the data acquisition device 5 is installed on the base 7; the servo motor 9 is connected to the reducer 10 through a coupling, and the reducer 10 is used to drive the transmission gear 10 to rotate; the transmission gear 11 is engaged with the drag chain 2; when the servo driver 8 receives the data acquisition instruction, it drives the servo motor 9 to run, and drives the transmission gear 11 to rotate through the reducer 10, so that the base 7 and the data scanning device 5 move on the drag chain 2.

[0053] The drag chain 2 can be a variety of drag chains, which are arranged on each fixed slot of the annular guide rail 1 to prevent the transmission mechanism from being separated from the drag chain during operation, which affects the accuracy of the action execution. The transmission mechanism 3 also includes a guide wheel 6 and corresponding accessories, etc. The guide wheel 6 is a guide device for stabilizing the transmission mechanism and the data scanning device 5. The servo driver 8 and the servo motor 9 provide power. The servo motor 9 is connected to the reducer 10 through a coupling. The reducer 10 is connected to the transmission gear 11. The transmission gear 11 is meshed with the drag chain 2, and the guide wheel 6 provides a stable support function for it.

[0054] When receiving the data acquisition instruction, the data scanning device 5 moves along the circular track 1 to scan and process the surrounding rock in the tunnel to obtain scanning data, wherein the scanning data includes surrounding rock images and point cloud data corresponding to a section of surrounding rock in the tunnel; the surrounding rock image is a segmented image photo of the surrounding rock corresponding to a certain length of surrounding rock, and the point cloud data is point cloud data of the same scale as the surrounding rock image, and the surrounding rock image and the point cloud data correspond to the same section of surrounding rock.

[0055] The data scanning device 5 includes an image acquisition device and a point cloud acquisition device, etc. The image acquisition device is used to acquire a surrounding rock image corresponding to a section of surrounding rock in the tunnel, and the point cloud acquisition device is used to acquire point cloud data corresponding to a section of surrounding rock in the tunnel. The image acquisition device includes a visible light sensor, etc., the point cloud acquisition device includes a point cloud sensor, etc., and the data scanning device 5 also includes a laser, a heat dissipation system, a wireless transmission module, and a battery, etc.

[0056] like Figure 3 As shown, each time the data scanning device 5 receives a PLC interaction instruction (data acquisition instruction), the data scanning device 5 moves once on the circular track 1 and completes a scan to obtain scan data, and the scan data includes a surrounding rock image and point cloud data corresponding to a section of surrounding rock where the circular track 1 is currently located. The data scanning device 5 transmits the scan data to the computing device 17 (edge ​​computing device) through a wireless method such as Bluetooth, and the computing device 17 processes the scan data through a built-in algorithm and transmits the processing result to the storage system 18.

[0057] The computing device 17 is equipped with a variety of built-in algorithms, including image correction algorithm, preprocessing algorithm, data splicing algorithm and surrounding rock joint recognition algorithm. The computing device 17 performs correction processing on the data scanning device 5; the computing device 17 performs splicing processing on multiple surrounding rock images to generate surrounding rock splicing images; the computing device 17 generates a joint feature map based on the surrounding rock splicing images and the corresponding point cloud data; the computing device 17 performs joint recognition processing on the joint feature map to obtain disaster information; the computing device 17 obtains joint area and location information, as well as surrounding rock prediction information based on the joint feature map.

[0058] like Figure 4As shown, the computing device 17 may include a correction module 170, an image stitching module 171, a joint feature generation module 172, a surrounding rock joint identification module 173, etc. The correction module 170 obtains the standard plate image acquired by the image acquisition device for the standard plate, and determines the first position information and the second position information of the center and the positioning hole of the standard plate in the standard plate image.

[0059] The correction module 170 obtains the point cloud data of the standard plate collected by the point cloud acquisition device for the standard plate, and determines the third position information and the fourth position information of the center and the positioning hole of the standard plate in the point cloud data of the standard plate; the correction module 170 performs correction processing on the point cloud acquisition device based on the first position information, the second position information, the third position information and the fourth position information. There are many methods for performing correction processing on the point cloud acquisition device. For example, the parameters of the point cloud acquisition device are adjusted so that the first position information is the same as or matches the third position information, and the second position information is the same as or matches the fourth position information.

[0060] In some embodiments, the correction module 170 is provided with an image correction algorithm for correcting the parallax between the point cloud acquisition device (including a point cloud sensor) and the image acquisition device (including a visible light sensor). After the corrected image of the standard plate captured by the image acquisition device (the image of the standard plate captured by the image acquisition device for the standard plate), the position information of the center and the four corners of the standard plate in the image (the first position information and the second position information) is calculated according to the center positions of the positioning holes and the hollow pattern in the image. For example, the standard plate image can be processed by a neural network model to obtain the first position information and the second position information.

[0061] The standard image plate is scanned by the point cloud acquisition device. Since the ToF distances of the hollow area and the non-hollow area are different, the positions of the center and the four corners of the standard image plate at the center of the matrix (the third position information and the fourth position information) can be calculated according to the point cloud distance matrix. The point cloud data of the standard plate can also be processed by a neural network model to obtain the third position information and the fourth position information.

[0062] The correction module 170 performs correction processing on the point cloud acquisition device based on the first position information, the second position information, the third position information and the fourth position information. For example, the data matrix of the point cloud acquisition device may be cropped, scaled, etc., so that the first position information is the same as or matches the third position information, and the second position information is the same as or matches the fourth position information.

[0063] like Figure 5As shown, the standard plate includes an image plate 12, a connector, and a positioning hole 14. The image plate 12 is printed with a correction image, and the four corners of the image plate 12 are punched with positioning holes 14. The image plate 12 can be connected to the data scanning device through four connectors, and the four connectors are of equal length. The four connectors are of equal length, which ensures that the standard plate can be fixedly installed on the top of the window of the data scanning device to avoid movement or tilting during the calibration process, thereby affecting the calibration accuracy.

[0064] The standard plate can be installed on the top of the window of the data scanning device through a latch to shield the field of view of the data scanning device, ensuring that it is not affected by the external environment during the calibration process. The positioning hole 14 includes a hollow pattern of a specific shape. When the surrounding rock detection system is working, a standard plate is required for calibration. After the calibration is completed, the standard plate is removed to obtain the widest sampling area. A hollow correction frame with a certain boundary is provided on the image board 12, which can realize real-time calibration to obtain the most accurate image.

[0065] Through the positioning holes 14 and the hollow pattern in the image plate 12, the standard plate can provide a standard calibration reference image for the image acquisition device and the point cloud acquisition device. By selecting the visible light image as the benchmark, the coordinates of the center and surrounding positioning points of the standard plate in the visible light image and the point cloud data are made the same, and the data size and dimension are the same, so as to achieve a matching effect. The parallax correction of the point cloud sensor and the visible light sensor can be realized, ensuring the consistency and accuracy of the data, and ensuring the high precision and stability of the system in practical applications.

[0066] In some embodiments, the image stitching module 171 uses the first image among the multiple surrounding rock images as the initial image and adds it to the image stitching area, and performs multi-scale image fusion optimization processing on the first image. The image stitching module 171 adds the second image data among the multiple surrounding rock images as a new image to the image stitching area, aligns the center of the first image with the center of the second image, and performs edge alignment processing on the second image and the first image. A variety of methods can be used to perform center alignment processing and edge alignment processing.

[0067] The image stitching module 171 determines the translation distance of the second image according to the first image, the second image and the entropy weights of the pixels. The image stitching module 171 performs stitching processing on the first image and the second image. The image stitching module 171 can use a variety of existing methods to perform stitching processing on the first image and the second image. For example, the SURF algorithm is used to extract features from the first image and the second image, the RANSAC algorithm is used to screen features and determine the transformation matrix, and the first image and the second image are subjected to image fusion processing.

[0068] The SURF algorithm is a fast and robust feature extraction algorithm that is highly robust to scale, rotation, and illumination changes. The RANSAC algorithm is an iterative method. In image stitching, the RANSAC algorithm is used to align images and estimate the transformation relationship between two images. Combining the SURF algorithm and the RANSAC algorithm for image stitching can give full play to the advantages of both.

[0069] During the stitching process, the image stitching module 171 uses the multi-scale SURF algorithm to extract features, and uses the RANSAC algorithm to determine the transformation matrix based on the matching distance of the feature points, the matching distance threshold, and the confidence weight. The multi-scale SURF algorithm provides accurate feature point matching, and the RANSAC algorithm can filter out incorrect matching point pairs, thereby improving the accuracy and efficiency of stitching.

[0070] In some embodiments, the first picture (first image) is selected from multiple surrounding rock pictures (images) as the initial image, and the first picture is imported into the stitched image area (image stitching area) as the starting point of the stitching process to ensure that the stitching process has a stable starting point.

[0071] The multi-scale image fusion algorithm is used to perform multi-scale image fusion optimization processing on the initial image, which can optimize the detail retention and contrast of the initial image and ensure the clarity and accuracy of the initial image. The multi-scale image fusion algorithm for multi-scale image fusion optimization processing is as follows:

[0072]

[0073] Among them, I init (x, y) is the pixel value of the initial stitched image (first image) at position (x, y), S is the number of different scales of the image, α s is the fusion coefficient of the s-th scale, is the pixel value of the initial image at position (x, y) at the sth scale; the scale can be multiple scales, and x and y are the coordinates of the pixel point.

[0074] The next image (second image) is introduced into the stitched image area in turn, and the center of the image is aligned along the X axis according to the mileage of the tunnel boring equipment. The image is edge-aligned using the adaptive edge detection and dynamic weighted matching method to ensure that the newly introduced image can be accurately aligned with the previous image at the edge, thereby avoiding the phenomenon of dislocation or incoherence in the stitched image. The algorithm for edge alignment using adaptive edge detection and dynamic weighted matching technology is as follows:

[0075] I align (x,y)=I init(x,y)⊕I2(x+Δx,y+Δy,ω(x,y)) (1-2);

[0076] Among them, I align (x, y) is the pixel value of the aligned image (second image) at position (x, y), ⊕ is the image fusion operator, and a variety of image fusion methods can be used; I init (x, y) is the pixel value of the initial stitched image at position (x, y), I2(x+Δx, y+Δy, ω(x, y)) is the pixel value of the second image (second image) at position (x, y) under offset (Δx, Δy) and weight ω(x, y), Δx, Δy is the image translation distance, ω(x, y) is the edge weight function, which can be dynamically adjusted according to the edge detection results. Adaptive edge detection technology can detect changes in image edges in real time, and dynamic weighted matching technology can achieve more accurate image alignment by adjusting weights.

[0077] An optimization alignment algorithm based on entropy weights is used to align the new image (second image) and the stitched image (first image) pixel by pixel to maximize the overlap: The optimization alignment algorithm based on entropy weights (determining the translation distance of the second image) is as follows:

[0078] Δd=argmin Δx,Δy ∑ x,y (ω1(x,y)·|I init (x,y)-I2(x+Δx,y+Δy)|)(1-3);

[0080] Among them, Δd is the optimal translation distance, ω1(x, y) is the entropy weight of the pixel point, and I2(x+Δx, y+Δy) is the pixel value of the second image at position (x, y) under the offset (Δx, Δy).

[0081] The formula for feature extraction using the multi-scale SURF algorithm is:

[0082]

[0083] Among them, F surf is the set of SURF feature points, S is the number of feature points at different scales, is the i-th feature point of the s-th scale, including the position (x i ,y i ),scale and direction N s is the number of feature points at the sth scale.

[0084] The matching formula of the improved RANSAC algorithm is:

[0085]

[0086] Among them, H ransac is the transformation matrix, N is the number of feature points, δ is the indicator function, which is 1 when the condition is met and 0 otherwise. i,1 ,f i,2 ) is the feature point matching distance, ∈ is the matching distance threshold, w i is the confidence weight of the feature point.

[0087] Repeat the above steps of introducing new images, aligning images, and fusing images, and introduce each surrounding rock image in turn until all surrounding rock images are spliced ​​together to generate a surrounding rock splicing image.

[0088] The image stitching module 171 performs precision optimization processing on the surrounding rock stitching image according to the offset of each surrounding rock image during the stitching process. For example, after all surrounding rock images are stitched together, a surrounding rock stitching image is generated, and a multi-stage optimization strategy can be used to gradually optimize the stitching precision:

[0089]

[0090] Among them, I merged (x, y) is the pixel value of the image spliced ​​after multiple alignments (the image spliced ​​after multiple surrounding rock images are aligned) at the position (x, y), M is the total number of images (the total number of multiple surrounding rock images), is the multi-image fusion operator, I k (x+Δx k ,y+Δy k ,ω k (x,y)) is the kth image at the offset (Δx k ,Δy k ) and weight ω k The pixel value at position (x,y) below (x,y).

[0091] The image stitching module 171 determines the timestamp of the surrounding rock stitching image based on the timestamps and timestamp weights of the multiple surrounding rock images; the image stitching module 171 determines the mileage information of the surrounding rock stitching image based on the mileage information and mileage weights of the multiple surrounding rock images; the image stitching module 171 performs labeling processing on the surrounding rock stitching image based on the timestamp and mileage information of the surrounding rock stitching image. The mileage information is information such as the mileage of the tunnel boring equipment, and this mileage information corresponds to the surrounding rock image or the surrounding rock stitching image.

[0092] In some embodiments, an adaptive timestamp and mileage calibration algorithm is introduced. After all images (surrounding rock images) are stitched together, the stitched image uses the end time as the stitched timestamp, and the mileage of the image center as the image center mileage, and performs annotation processing:

[0093]

[0094] Among them, timestam pend is the end timestamp of the stitched image (the timestamp of the surrounding rock stitching image), timestamp(I k ) is the timestamp of the kth image (surrounding rock image), ω t (k) is the weight of the timestamp, mileage center is the center mileage of the stitched image (mileage information of the surrounding rock stitched image), mileage(I k ) is the mileage of the kth image, ω m (k) is the weight of the mileage; M is the number of surrounding rock images.

[0095] The image stitching module 171 uses an adaptive filtering correction algorithm to correct the image stitched from two or more surrounding rock images during the stitching process. During the image stitching process, the image correction algorithm based on adaptive filtering can be used to adjust the image parameters such as brightness, contrast and color:

[0096]

[0097] Among them, I corrected (x, y) is the pixel value of the corrected image (the image formed by stitching two or more surrounding rock images) at position (x, y), I merged (x, y) is the pixel value of the stitched image (the image stitched together from two or more surrounding rock images) at position (x, y), μ merged (x, y) is the local mean of the stitched image at position (x, y), σ target (x, y) is the standard deviation of the target image at position (x, y), σ merged (x, y) is the standard deviation of the stitched image at position (x, y), μ target (x, y) is the mean value of the target image at the position (x, y); the target image is the target image corresponding to the image spliced ​​from two or more surrounding rock images.

[0098] The above-mentioned image stitching method can be used to stitch multiple surrounding rock images to generate a surrounding rock stitching image. The above-mentioned image stitching method can also be used to stitch two or more surrounding rock images to generate a segment of surrounding rock stitching image, and the above-mentioned image stitching method can be used to stitch multiple segments of surrounding rock stitching images to generate a surrounding rock stitching image.

[0099] For example, the first segment of the picture (the first segment of the surrounding rock stitching image) is selected as the initial image and imported into the stitched image area as the starting point of the stitching process. The multi-scale image fusion technology is used to process the initial image to improve the detail retention and contrast of the initial image; the next segment of the picture (another segment of the surrounding rock stitching image) is gradually introduced into the stitching area, the image center is aligned along the X-axis according to the mileage, and the image is dynamically adjusted to maximize its overlap with the previous segment of the image; multi-scale SURF feature extraction and a robust matching algorithm based on improved RANSAC are used for feature matching to perform segment image stitching; if the movement gap is large, an intelligent prompt system is used to guide the user to perform manual stitching.

[0100] Repeat the above steps of introducing new images, aligning images, and fusing images until all images (multi-segment surrounding rock stitching images) are stitched together. In the process of image stitching, the alignment accuracy of the images can be optimized; multiple segments of images can be stitched together into a complete image step by step, and each step ensures high accuracy and consistency.

[0101] After all the images are stitched together, the stitched image is timestamped with the end time and marked with the mileage of the image center. All the stitched image segments are combined into a complete surrounding rock stitching image, which can provide continuous visual and geological information of the entire tunnel.

[0102] In the image stitching process disclosed in the present invention, by selecting the first picture as the initial image, using multi-scale image fusion technology to optimize the detail retention and contrast of the image, gradually introducing other pictures, and aligning the image centers along the X-axis by mileage, using adaptive edge detection and dynamic weighted matching technology to perform edge alignment on the images, the optimization of image details and contrast can be ensured by multi-scale image fusion technology, and the position and weight of the image can be dynamically adjusted by adaptive edge detection and dynamic weighted matching technology to achieve more accurate image alignment and improve the accuracy and consistency of stitching.

[0103] Through multi-scale SURF feature extraction and a robust matching algorithm based on improved RANSAC, the accuracy and robustness of feature point matching are improved; compared with the existing SURF algorithm, the concept of multi-scale is introduced, so that feature extraction can be performed at different scales, enhancing the adaptability and accuracy of the algorithm; at the same time, by improving the RANSAC algorithm, it is made more robust in the feature point matching process, reducing the probability of false matching and improving the overall matching accuracy.

[0104] In some embodiments, the joint feature generation module 172 uses a visible light extraction network to process the surrounding rock splicing image to obtain an RGB feature map corresponding to the surrounding rock splicing image; the joint feature generation module 172 uses a point cloud feature extraction network to process the point cloud data corresponding to multiple surrounding rock images to obtain a point cloud feature map; the point cloud feature map is upsampled; the joint feature generation module 172 superimposes the RGB feature map and the point cloud feature map through a feature superposition network to generate a joint feature map. The surrounding rock joint recognition module 173 uses a joint recognition network to process the joint feature map to determine the surrounding rock grade of the pixels in the joint feature map. The joint recognition network can be a variety of neural networks, such as a fully connected network.

[0105] The joint feature generation module 172 uses the visible light extraction network and the point cloud image feature extraction network to superimpose the RGB feature map and the point cloud feature map to generate a joint feature map, which can provide richer detection information and improve the accuracy of surrounding rock detection. The surrounding rock joint recognition module 173 uses a joint recognition network such as a fully connected network to process the joint feature map, and determines the surrounding rock grade of the pixel according to the superposition result of the feature map. The surrounding rock grade is I, II, III, IV, etc., making the classification method more detailed and accurate, and improving the accuracy and reliability of surrounding rock recognition.

[0106] For example, the visible light extraction network can be a variety of neural networks. The visible light extraction network is used to process the RGB visible light image part in the spliced ​​surrounding rock splicing image to extract the RGB feature map:

[0107]

[0108] Among them, F RGB (x, y) is the RGB feature map at position (x, y), K is the number of convolution kernels, W RGB,k is the weight of the kth convolution kernel, * is the convolution operator, I RGB (x+Δx k ,y+Δy k ) is the original RGB image at position (x, y), and σ is the activation function used for nonlinear transformation.

[0109] The point cloud feature extraction network can be a variety of neural networks. Use the point cloud feature extraction network to process the point cloud image part in the same image (multiple point cloud data corresponding to multiple surrounding rock images that constitute the surrounding rock splicing image) to obtain the point cloud feature map:

[0110]

[0111] Among them, F Depth (x, y) is the point cloud feature map at position (x, y), K is the number of convolution kernels, W Depth,kis the weight of the kth convolution kernel, I Depth (x+Δx k ,y+Δy k ) is the original point cloud image at position (x, y), Non-linear activation function for point cloud feature extraction.

[0112] The feature superposition network can be a variety of neural networks. The RGB feature map and the point cloud feature map can be superimposed through a fully connected network (feature superposition network) to generate a joint feature map:

[0113]

[0114] Among them, F Joint (x,y) is the joint feature map at position (x,y), N RGB is the number of RGB feature maps, N Depth is the number of point cloud feature maps, α ij , β ij is the feature fusion weight coefficient, F RGB,i (x,y),F Depth,j (x, y) is the value of the i-th RGB feature map and the j-th point cloud feature map at position (x, y).

[0115] The joint recognition network can be a variety of neural networks. The joint recognition network is used to process the joint feature map to determine the surrounding rock grade of the pixels in the joint feature map (the surrounding rock grade can be I, II, III, IV, etc.):

[0116]

[0117] Where L(x,y) is the surrounding rock grade at position (x,y), c is the surrounding rock grade category, and N Joint is the number of joint feature maps, γ c,i is the weight coefficient of the cth category, F Joint,i (x,y) is the value of the i-th joint feature map at position (x,y).

[0118] There are many ways to upsample the point cloud features for further combination with the RGB feature map. The formula for upsampling the point cloud feature map is as follows:

[0119]

[0120] in, is the value of the upsampled point cloud feature map at position (x, y), K is the number of Gaussian kernels, and λ k is the upsampling weight coefficient, s k is the scale factor of the kth Gaussian kernel, The scale is σk Gaussian kernel function.

[0121] By combining the joint feature map with the RGB feature map, visible light features, point cloud features, etc., and upsampling the point cloud features, a surrounding rock area grade map can be obtained.

[0122] In some embodiments, the visible light feature extraction network is a visible light Encoder feature extraction network. When the size of the input image (surrounding rock splicing image) is 518x518x3, the specific execution method is as follows:

[0123] 1. conv1: Perform two [3,3] 64-channel convolutions to obtain a preliminary effective feature layer of [518,518,64], and then perform 2X2 maximum pooling to obtain a feature layer of [256,256,64].

[0124] 2. conv2: Perform two [3,3] 188-channel convolutions to obtain a preliminary effective feature layer of [256,256,188], and then perform 2X2 maximum pooling to obtain a feature layer of [188,188,188].

[0125] 3. conv3: Perform three [3,3] 256-channel convolutions to obtain a preliminary effective feature layer of [188,188,256], and then perform 2X2 maximum pooling to obtain a feature layer of [64,64,256].

[0126] 4. conv4: Perform three [3,3] 518-channel convolutions to obtain a preliminary effective feature layer of [64,64,518], and then perform 2X2 maximum pooling to obtain a [32,32,518] feature layer.

[0127] 5. conv5: Perform three [3,3] 518-channel convolutions to obtain a preliminary effective feature layer of [32,32,518].

[0128] The point cloud feature extraction network can be the above Encoder feature extraction network, and a 518x518x3 point cloud image is obtained by multi-frame synthesis.

[0129] After the visible light feature layer is superimposed on the visible light feature layer, when the point cloud feature map is upsampled, it is directly upsampled twice and then feature fusion is performed. The final feature layer has the same height and width as the input image and the surrounding rock area grade map. Then a 1×1 convolution is used to adjust the channels to adjust the number of channels of the final feature layer to a recognizable classification.

[0130] The user or the system automatically selects specific image segments in the stitched surrounding rock mosaic image:

[0131]

[0132] Among them, D selected is the selected image segment, D is the set of all image segments, W(x,y) is the weight function of position (x,y), II(D i (x,y)) is the indicator function, indicating the image segment D i The presence or absence of the position (x,y).

[0133] Input the selected image segment to the surrounding rock region recognition algorithm to identify and classify the various regions in the image:

[0134]

[0135] Where R(x,y) is the surrounding rock region at position (x,y), r is the surrounding rock region number, N selected is the number of features in the selected image segment, δ r,i is the weight coefficient of the rth surrounding rock region, D selected,i (x,y) is the value of the i-th feature of the selected image segment at position (x,y).

[0136] The RGB visible light image contained in the surrounding rock splicing image can be processed through a visible light extraction network to obtain an RGB feature map, and the point cloud data corresponding to the cropped surrounding rock splicing image can be processed through a point cloud feature extraction network to obtain a point cloud feature map. The fully connected network obtains a joint feature map by superimposing the point cloud feature map and the RGB feature map.

[0137] The features of each layer of the joint feature map are sent to the feature pyramid network, and the feature maps of different scales are fused. The fused features are detected and segmented in parallel. The detection branch outputs the category, border, and K confidence information for each target object. For the segmentation branch, k mask images are output. The linear combination network multiplies the k confidences with the k mask prototype images and adds all the results to obtain the joint image instance area. Different joint areas are segmented, and the joint area information and position are recorded. The segmented joint image is sent to the joint feature recognition model. The joint feature recognition model can be an RGBD-Yolo model, etc., which is used to identify the number and position of joints.

[0138] In some embodiments, the surrounding rock joint identification module 173 divides the joint feature graph into a plurality of joint feature subgraphs; when the surrounding rock grade corresponding to the joint feature subgraph is greater than the grade threshold, the surrounding rock joint identification module 173 uses the disaster identification model to perform joint identification processing on the joint feature subgraph to obtain disaster information. The disaster information includes the type and location of the disaster, and the grade threshold may be level 2, level 3, etc.

[0139] For example, the disaster identification model can be a variety of neural network models. The joint feature map is divided into multiple joint feature sub-maps of equal size, and detailed joint identification is performed on each joint feature sub-map after segmentation. The surrounding rock grade is further refined according to the joint situation. When the joints or other features cannot be automatically identified, the image segment is sent to the manual recognition system for further analysis.

[0140] For the joint feature sub-graphs identified as level III, level IV, and level V extreme fragmentation, the RGBD-Yolo-ResNet50 model (disaster identification model) is used to identify the type of disaster. The RGBD-Yolo-ResNet50 model analyzes the joint feature graph, which can identify small-scale disasters such as water gushing, cracks, and sand holes, and determine the specific locations of small-scale disasters. The joint feature graph is upsampled and traversed in a fine-grained manner, and the ResNet50 classification network is used to identify large-scale geological disasters such as rock bursts and ruptures.

[0141] The joint feature subgraph can be input into the RGBD-Yolo-ResNet50 model for recognition. For example, the structure of the RGBD-Yolo-ResNet50 model is:

[0142] The first stage is INPUT input

[0143] Phase 2 is:

[0144] (STAGE0(3,244,244,CONV=(7*7,64, / 2,BN,RELU),MAXPOOL=3*3, / 2,(64*56*56)))

[0145] Phase 3 is:

[0146] (STAGE1(Btnk1=64*56*64*1),(256,56,56),(BTNK2=256,56),(256,56,56),(BTNK2=256,256),(256,56,56))

[0147] Phase 4 is:

[0148] (STAGE2(BTNK1=256,56,128,2),(512,28,28),(BTNK2=512,28),(512,28,28),(BTNK2=512,28),(512,28,28),(BTNK2=512,28),(512,28,28))

[0149] Phase 5 is:

[0150] (STAGE3(BTNK1=512,28,256,2),(1024,14,14),(BTNK2=1024,14),(1024,14,14),(BTNK2=1024,14),(1024,1 4,14),(BTNK2=1024,14),(1024,14,14),(BTNK2=1024,14),(1024,14,14),(BTNK2=1024,14),(1024,14,14))

[0151] Phase 6 is:

[0152] (STAGE6(BTNK1=1024,14,512,2),(2048,7,7),(BTNK2=2048,7),(2048,7,7),(BTNK2=2048,7),(2048,7,7))

[0153] Get the final classification.

[0154] In some embodiments, the surrounding rock joint identification module 173 uses the joint area identification model to identify the joint feature sub-graph to obtain the joint area and position information of the joint feature sub-graph; the surrounding rock joint identification module 173 uses the surrounding rock situation prediction model to predict the joint feature graph to obtain surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

[0155] For example, the joint feature map is sent to the joint region recognition model to identify and record the joint region and its features in the joint feature map. The joint region recognition model can be a variety of neural network models for identifying regions containing joint features in the image.

[0156] The surrounding rock situation prediction model can be a variety of neural network models. The surrounding rock situation prediction model is based on the situation prediction algorithm and is used to predict the surrounding rock situation that has not been excavated and assist engineering decision-making. The situation prediction algorithm matches the case data D1 in similar situations according to the latest results currently identified from the previous experience database, selects the data D2 of several recent examples of this project from the database, sets the current mileage to 0, the previous mileage distance is a negative value, and the unexcavated mileage is a positive value, and the surrounding rock area situation information of D1 and D2 constitutes a data set D, which is divided into a training set and a validation set at a ratio of 8:2. A CNN model (surrounding rock situation prediction model) is established for training and a parameter regression model is established. The mileage is used as an independent variable and the surrounding rock situation is used as a dependent variable. The model with the lowest MSE is selected as the prediction model. After obtaining the model, the prediction value of 0-500m is traversed with a granularity of 1m to predict the future surrounding rock situation, so that construction personnel can analyze historical experience and estimate future trends.

[0157] The surrounding rock situation prediction model can be a CNN model. A CNN model is established for training, with mileage as the independent variable and surrounding rock situation as the dependent variable. The model with the lowest MSE is selected as the prediction model. For example, the surrounding rock situation prediction model contains 14 layers, and the model structure is:

[0158] The first layer is (Conv1D(16,3,input_shape=(246,1),activation='relu'))

[0159] The second layer is (Conv1D(16,3,activation='relu'))

[0160] The third layer is (MaxPooling1D(3))

[0161] The 4th layer is (Conv1D(64,3,activation='relu'))

[0162] The 5th layer is (Conv1D(64,3,activation='relu'))

[0163] The sixth layer is (MaxPooling1D(3))

[0164] The 7th layer is (Conv1D(128,3,activation='relu'))

[0165] The 8th layer is (Conv1D(128,3,activation='relu'))

[0166] The 9th layer is (MaxPooling1D(3))

[0167] The 10th layer is (Conv1D(64,3,activation='relu'))

[0168] The 11th layer is (Conv1D(64,3,activation='relu'))

[0169] The 12th layer is (MaxPooling1D(3))

[0170] The 13th layer is (Flatten())

[0171] The 14th layer is (Dense(1, activation = 'linear'))

[0172] model.compile(optimizer='adam', loss='mean_squared_error', metrics=[coeff_determination]).

[0173] After the image segments are processed, the surrounding rock area situation (surrounding rock prediction information) is recorded according to the recognition results, including the location and type of the surrounding rock area, the number of joints in the surrounding rock, the location of disasters in the surrounding rock, etc., and output as a report in a standard format for use in engineering construction.

[0174] Figure 6 FIG. 1 is a flow chart of an embodiment of a surrounding rock detection method according to the present disclosure, as shown in FIG. Figure 6 As shown:

[0175] Step S601, calibrating the data scanning device.

[0176] Step S602: stitching multiple surrounding rock images to generate a surrounding rock stitching image.

[0177] Step S603: generating a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data.

[0178] Step S604, performing joint identification processing on the joint feature map to obtain disaster information.

[0179] Step S605, obtaining joint area and location information, and surrounding rock prediction information based on the joint feature map.

[0180] Figure 7 FIG. 1 is a flow chart of generating a surrounding rock splicing image in one embodiment of the surrounding rock detection method disclosed in the present invention. Figure 7 As shown:

[0181] Step S701: taking a first image among a plurality of surrounding rock images as an initial image and adding an image stitching region, and performing multi-scale image fusion optimization processing on the first image.

[0182] Step S702: adding the second image data in the plurality of surrounding rock images as a new image to the image stitching area, aligning the center of the first image with the center of the second image, and performing edge alignment on the second image and the first image.

[0183] Step S703: determining the translation distance of the second image according to the first image, the second image and the entropy weights of the pixels.

[0184] Step S704: stitching the first image and the second image.

[0185] During the stitching process, the multi-scale SURF algorithm can be used to extract features, and the RANSAC algorithm is used to determine the transformation matrix based on the matching distance of the feature points, the matching distance threshold, and the confidence weight.

[0186] Figure 8 FIG. 1 is a flow chart of labeling processing in an embodiment of the surrounding rock detection method according to the present disclosure, as shown in FIG. Figure 8 As shown:

[0187] Step S801, performing precision optimization processing on the surrounding rock splicing image according to the offset of each surrounding rock image during the splicing process.

[0188] Step S802: determining the timestamp of the surrounding rock splicing image based on the timestamps and timestamp weights of the plurality of surrounding rock images.

[0189] Step S803, determining the mileage information of the surrounding rock splicing image based on the mileage information and mileage weights of the plurality of surrounding rock images.

[0190] Step S804: labeling the surrounding rock splicing image based on the timestamp and mileage information of the surrounding rock splicing image.

[0191] Fig. 9 FIG. 1 is a schematic diagram of a process for generating a joint feature map in an embodiment of a surrounding rock detection method according to the present disclosure, as shown in FIG. Fig. 9 As shown:

[0192] Step S901: Use a visible light extraction network to process the surrounding rock mosaic image to obtain an RGB feature map corresponding to the surrounding rock mosaic image.

[0193] Step S902: Use a point cloud feature extraction network to process point cloud data corresponding to multiple surrounding rock images to obtain a point cloud feature map; and perform upsampling processing on the point cloud feature map.

[0194] Step S903: superimpose the RGB feature map and the point cloud feature map through a feature superposition network to generate a joint feature map.

[0195] Fig.10 FIG. 1 is a flow chart of obtaining disaster information in an embodiment of the surrounding rock detection method according to the present disclosure, as shown in FIG. Fig.10 As shown:

[0196] Step S1001, using the joint recognition network to process the joint feature map, and determine the surrounding rock grade of the pixels in the joint feature map.

[0197] Step S1002, dividing the joint feature graph into multiple joint feature subgraphs.

[0198] Step S1003, when the surrounding rock grade corresponding to the joint feature subgraph is greater than the grade threshold, the joint feature subgraph is processed for joint identification using the disaster identification model to obtain disaster information.

[0199] The joint feature sub-graph can be identified and processed using the joint region identification model to obtain the joint region and position information of the joint feature sub-graph; the surrounding rock situation prediction model can be used to predict the joint feature graph to obtain surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

[0200] Fig.11 FIG. 1 is a flow chart of a correction process for a data scanning device in an embodiment of a surrounding rock detection method according to the present disclosure, as shown in FIG. Fig.11 As shown:

[0201] Step S1101 , obtaining a standard plate image acquired by an image acquisition device for the standard plate, and determining first position information and second position information of the center and the positioning hole of the standard plate in the standard plate image.

[0202] Step S1102, determining the third position information and the fourth position information of the center and the positioning hole of the standard plate in the point cloud data of the standard plate according to the point cloud data of the standard plate collected by the point cloud collection device.

[0203] Step S1103, based on the first position information, the second position information, the third position information and the fourth position information, the point cloud acquisition device is calibrated.

[0204] In some embodiments, Fig.12 As shown, the computing device may include a memory 121, a processor 122, a communication interface 123 and a bus 124. The memory 121 is used to store instructions, the processor 122 is coupled to the memory 121, and the processor 122 is configured to execute the above-mentioned surrounding rock detection method based on the instructions stored in the memory 121.

[0205] The memory 121 may be a high-speed RAM memory, a non-volatile memory, etc. The memory 121 may also be a memory array. The memory 81 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules. The processor 122 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the surrounding rock detection method disclosed in the present invention.

[0206] In some embodiments, the present disclosure provides a computer-readable storage medium storing computer instructions, which implement the methods in any of the above embodiments when the instructions are executed by a processor.

[0207] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive enumeration) of readable storage media can include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0208] The embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0209] The surrounding rock detection system, method, computing device, storage medium and program product of the tunnel boring equipment in the above-mentioned embodiments drive the data scanning device to move on the circular track through the transmission mechanism, and the data scanning device scans and processes the surrounding rock in the tunnel, and the obtained scanning data includes a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; the computing device generates a surrounding rock splicing image and a joint feature map based on the scanning data, and obtains disaster information, joint area, location information and surrounding rock prediction information according to the joint feature map; it can make full use of various surrounding rock geological information such as images and point clouds, improve the accuracy and reliability of surrounding rock detection, and improve the safety and efficiency of construction; it can replace manual surrounding rock detection, improve the intelligence level of tunnel boring equipment, reduce labor costs, improve the working efficiency of tunnel boring equipment, and improve the user experience.

[0210] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0211] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0212] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0213] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0214] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0215] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, it will be appreciated by those skilled in the art that the above embodiments are merely illustrative and do not limit the scope of the present disclosure. It will be appreciated by those skilled in the art that the above embodiments may be combined, modified or replaced without departing from the scope and essence of the present disclosure.

Claims

1. A surrounding rock detection system for tunnel boring equipment, comprising: Circular track, data scanning device, transmission mechanism and computing equipment; The circular track is installed in the tunnel, and the transmission mechanism is used to drive the data scanning device to move on the circular track; when receiving the data acquisition instruction, the data scanning device scans the surrounding rock in the tunnel to obtain scanning data, wherein the scanning data includes a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; The computing device is used to perform correction processing on the data scanning device; perform splicing processing on multiple surrounding rock images to generate a surrounding rock splicing image; generate a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data; perform joint identification processing on the joint feature map to obtain disaster information; and obtain joint area and location information, as well as surrounding rock prediction information based on the joint feature map.

2. The method of claim 1, wherein: The computing device comprises: An image stitching module is used to take the first image among the multiple surrounding rock images as the initial image and add it to the image stitching area, and perform multi-scale image fusion optimization processing on the first image; add the second image data among the multiple surrounding rock images as a new image to the image stitching area, align the center of the first image with the center of the second image, and perform edge alignment processing on the second image and the first image; determine the translation distance of the second image according to the entropy weights of the first image, the second image and the pixel points; and stitch the first image and the second image.

3. The method of claim 2, wherein: The image stitching module is used to extract features using a multi-scale SURF algorithm during the stitching process, and to determine a transformation matrix using a RANSAC algorithm based on the matching distance of feature points, a matching distance threshold, and a confidence weight; and to correct an image stitched from two or more surrounding rock images using an adaptive filtering correction algorithm during the stitching process.

4. The method of claim 2, wherein: The image stitching module is used to perform precision optimization processing on the surrounding rock stitching image according to the offset of each surrounding rock image during the stitching process; Determining the timestamp of the surrounding rock splicing image based on the timestamps and timestamp weights of the plurality of surrounding rock images; The mileage information of the surrounding rock splicing image is determined based on the mileage information and mileage weights of the plurality of surrounding rock images; and the surrounding rock splicing image is labeled based on the timestamp and mileage information of the surrounding rock splicing image.

5. The system of claim 2, wherein: The computing device comprises: A joint feature generation module is used to process the surrounding rock splicing image using a visible light extraction network to obtain an RGB feature map corresponding to the surrounding rock splicing image; use a point cloud feature extraction network to process the point cloud data corresponding to the multiple surrounding rock images to obtain a point cloud feature map; upsample the point cloud feature map; and superimpose the RGB feature map and the point cloud feature map through a feature superposition network to generate the joint feature map.

6. The system of claim 2, wherein: The computing device comprises: The surrounding rock joint identification module is used to process the joint feature map using a joint identification network to determine the surrounding rock grade of the pixels in the joint feature map; divide the joint feature map into multiple joint feature sub-maps; and when the surrounding rock grade corresponding to the joint feature sub-map is greater than a grade threshold, use the disaster identification model to perform joint identification processing on the joint feature sub-map to obtain the disaster information.

7. The system of claim 6, wherein: The surrounding rock joint identification module is used to use the joint area identification model to identify the joint feature sub-graph to obtain the joint area and the position information of the joint feature sub-graph; use the surrounding rock situation prediction model to predict the joint feature graph to obtain the surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

8. The system of claim 1, wherein: The data scanning device includes an image acquisition device and a point cloud acquisition device; The computing device comprises: A correction module is used to obtain the standard plate image acquired by the image acquisition device for the standard plate, and determine the first position information and the second position information of the center and the positioning hole of the standard plate in the standard plate image; determine the third position information and the fourth position information of the center and the positioning hole of the standard plate in the standard plate point cloud data based on the standard plate point cloud data acquired by the point cloud acquisition device for the standard plate; and perform correction processing on the point cloud acquisition device based on the first position information, the second position information, the third position information and the fourth position information.

9. The system of claim 1, wherein: A drag chain is installed on the annular track; the transmission mechanism includes a base, a servo driver, a servo motor, a reducer, and a transmission gear; the transmission gear is installed at the bottom of the base, the transmission gear is meshed with the drag chain, and the data acquisition device is installed on the base; the servo motor is connected to the reducer through a coupling, and the reducer is used to drive the transmission gear to rotate; When the driver receives the data acquisition instruction, the servo motor is driven to operate, and the transmission gear is driven to rotate through the reducer, so that the base moves on the drag chain.

10. A surrounding rock detection method, applied to a surrounding rock detection system of a tunnel boring device, wherein: The surrounding rock detection system comprises: a circular track, a data scanning device, a transmission mechanism and a computing device; the circular track is installed in a tunnel, the data scanning device is installed on the circular track, and the transmission mechanism is used to drive the data scanning device to move on the circular track; the data scanning device scans and processes the surrounding rock in the tunnel when receiving a data acquisition instruction to obtain scanning data, wherein the scanning data comprises a surrounding rock image and point cloud data corresponding to a section of surrounding rock in the tunnel; the surrounding rock detection method is executed in the computing device, and comprises: Performing calibration processing on the data scanning device; Performing mosaic processing on multiple surrounding rock images to generate a surrounding rock mosaic image; Generate a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data; Performing joint identification processing on the joint characteristic map to obtain disaster information; Based on the joint feature map, joint area and location information, as well as surrounding rock prediction information are obtained.

11. The method of claim 10, wherein: The stitching of the plurality of surrounding rock images to generate a stitched surrounding rock image comprises: Taking a first image among the plurality of surrounding rock images as an initial image and adding it into an image stitching region, and performing multi-scale image fusion optimization processing on the first image; Adding the second image data among the plurality of surrounding rock images as a new image to the image stitching area, aligning the center of the first image with the center of the second image, and performing edge alignment processing on the second image and the first image; Determining a translation distance of the second image according to the first image, the second image and entropy weights of pixels; The first image and the second image are spliced.

12. The method of claim 11, further comprising: In the process of stitching, the multi-scale SURF algorithm is used for feature extraction, and the RANSAC algorithm is used to determine the transformation matrix based on the matching distance of feature points, matching distance threshold and confidence weight; During the splicing process, an adaptive filtering correction algorithm is used to correct the image formed by splicing two or more surrounding rock images.

13. The method of claim 11, further comprising: According to the offset of each of the surrounding rock images during the splicing process, the precision of the surrounding rock splicing image is optimized; Determining the timestamp of the surrounding rock splicing image based on the timestamps and timestamp weights of the plurality of surrounding rock images; Determining the mileage information of the surrounding rock splicing image based on the mileage information and mileage weights of the plurality of surrounding rock images; Based on the timestamp and mileage information of the surrounding rock splicing image, the surrounding rock splicing image is labeled.

14. The method of claim 11, wherein: The generating of a joint feature map based on the surrounding rock splicing image and the corresponding point cloud data comprises: Processing the surrounding rock splicing image using a visible light extraction network to obtain an RGB feature map corresponding to the surrounding rock splicing image; Using a point cloud feature extraction network to process the point cloud data corresponding to the plurality of surrounding rock images, obtaining a point cloud feature map, and performing upsampling processing on the point cloud feature map; The RGB feature map and the point cloud feature map are superimposed through a feature superposition network to generate the joint feature map.

15. The method of claim 11, wherein: The performing joint identification processing on the joint feature map to obtain disaster information comprises: Processing the joint feature map using a joint recognition network to determine the surrounding rock grade of pixels in the joint feature map; Splitting the joint feature graph into a plurality of joint feature subgraphs; When the surrounding rock grade corresponding to the joint characteristic subgraph is greater than the grade threshold, the joint characteristic subgraph is processed for joint identification using the disaster identification model to obtain the disaster information.

16. The method of claim 15, wherein: The obtaining of joint area and location information and surrounding rock prediction information based on the joint feature map includes: Using a joint region recognition model to perform recognition processing on the joint feature subgraph to obtain the joint region and position information of the joint feature subgraph; The surrounding rock situation prediction model is used to perform prediction processing on the joint characteristic map to obtain the surrounding rock prediction information corresponding to the state of the surrounding rock in the tunnel.

17. The method of claim 10, wherein: The data scanning device includes an image acquisition device and a point cloud acquisition device; the correction processing of the data scanning device includes: Acquire a standard plate image acquired by the image acquisition device for the standard plate, and determine first position information and second position information of the center and the positioning hole of the standard plate in the standard plate image; Determine third position information and fourth position information of the center and the positioning hole of the standard plate in the point cloud data of the standard plate according to the point cloud data of the standard plate collected by the point cloud collection device; Based on the first position information, the second position information, the third position information and the fourth position information, the point cloud acquisition device is calibrated.

18. A computing device comprising: Memory; and a processor coupled to the memory, the processor being configured to execute the method according to any one of claims 10 to 17 based on instructions stored in the memory.

19. A computer-readable storage medium storing computer instructions, wherein the instructions are executed by a processor to perform the method according to any one of claims 10 to 17.

20. A computer program product, wherein the computer program product stores computer instructions, wherein the instructions are executed by a processor to perform the method according to any one of claims 10 to 17.

Citation Information

Patent Citations

  • Rock structural surface occurrence measuring method integrated with laser-point cloud and digital imaging

    CN105180890A

  • BM tunnel surrounding rock morphology acquisition method and device

    CN110686616A

  • Tunnel disease detection method fusing laser point cloud and panoramic image

    CN113822891A

  • Rail-crossing tunnel apparent disease high-precision identification and live-action model construction device and method

    CN115937408A

  • Monitoring method and system for small-clear-distance tunnel surrounding rock stability control

    CN117975233A

Cited By

  • Tunnel surrounding rock stability intelligent evaluation system based on multi-source image fusion

    CN121121647A

  • Intelligent evaluation system for tunnel surrounding rock stability based on multi-source image fusion

    CN121121647B