A visual monitoring method and device for a cutterhead assembly of a tunnel boring machine
By installing multi-axis gimbals, high-precision cameras and laser scanning equipment in the soil chamber of the tunnel boring machine, the three-dimensional model of the cutter assembly is solved, and the construction efficiency and safety are improved sex.
Patent Information
- Application Number
- CN202411367242.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-29
AI Technical Summary
It is difficult to visually monitor the tunnel boring machine during operation, and it is impossible to update the three-dimensional model of the cutting board structure according to the actual construction conditions, resulting in low construction efficiency and safety risks.
Multiple visual monitoring equipment are symmetrically installed in the soil cabin, and visual data from different angles is collected through multi-axis gimbal, high-precision industrial cameras, laser scanning equipment, and three-dimensional models of the cutting board components are constructed and updated, combining iterative nearest point algorithm, Finite element analysis and convolutional neural networks generate shutdown maintenance strategies in real time and monitor morphological changes.
It realizes visual data acquisition and three-dimensional model update of high-precision cutting board components without being affected by the soil cabin environment, improves construction efficiency and safety, and meets the actual needs of operators.
Smart Images

Figure CN118862377B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of electrical digital data processing, and in particular to a visual monitoring method and device for a cutterhead assembly of a tunnel boring machine. Background Art
[0002] Tunnel boring machines are large-scale intelligent engineering machinery commonly used in underground projects. They have functions such as rock breaking and excavation, rock ballast transportation and segment assembly. Since the force on the cutterhead of a tunnel boring machine is very complex during construction, it is very easy to cause wear of the cutter and reduce the excavation efficiency. Currently, workers need to open the cabin to enter the soil cabin of the tunnel boring machine to observe the situation and perform maintenance work on the cutterhead components such as the roller, which is accompanied by great safety hazards.
[0003] The existing visualization monitoring system for tunnel boring machines can only provide two-dimensional image data with a fixed perspective. It is not only difficult to collect complete image data of the cutterhead assembly, but also cannot be used to accurately warn of unexpected situations such as foreign objects falling into the soil compartment, which is difficult to meet the actual needs of operators. In addition, due to the complex structure of the cutterhead and the large amount of rock debris blocking the construction, the existing three-dimensional modeling and detection system must be shut down and the soil compartment must be emptied before the three-dimensional model used to evaluate the status of the cutterhead assembly can be updated, making it difficult to obtain the status of the cutterhead assembly in a timely and effective manner, which may affect the actual construction efficiency. Summary of the invention
[0004] The embodiments of the present application provide a visual monitoring method and device for the cutter head assembly of a tunnel boring machine, which is used to solve the technical problems that the current tunnel boring machine is difficult to perform visual monitoring during operation, and the three-dimensional model of the cutter head structure cannot be updated according to the actual construction situation, resulting in the inability to meet the actual needs of the operator and affecting the actual construction efficiency of the tunnel boring machine.
[0005] On the one hand, an embodiment of the present application provides a visual monitoring method for a cutterhead assembly of a tunnel boring machine, the method comprising:
[0006] Acquire first visual data from a plurality of visual monitoring devices; wherein each of the visual monitoring devices is used to collect visual data of a cutterhead assembly at different angles; the visual monitoring device at least comprises an image acquisition unit provided with a multi-axis pan-tilt head; the first visual data comprises visual data of the cutterhead assembly of the tunnel boring machine in a first operating state;
[0007] Based on the first visual data, construct a three-dimensional model of the cutter head assembly and display it on a user terminal;
[0008] Acquire second visual data in a second operating state in real time;
[0009] generating a shutdown maintenance strategy for the tunnel boring machine based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly;
[0010] When the tunnel boring machine meets the conditions corresponding to the shutdown maintenance strategy, a first control instruction for entering the first operating state is generated, so as to update the first visual data acquired according to the execution state of the first control instruction, so as to update the three-dimensional model of the cutterhead assembly;
[0011] Based on the updated three-dimensional model of the cutter disc assembly, the morphological change amount corresponding to each cutter disc assembly is determined and synchronously displayed on the three-dimensional model of the cutter disc assembly, so as to update the analysis parameters corresponding to the first analysis processing and / or generate corresponding cutter disc assembly monitoring and alarm information according to each morphological change amount, until a second control instruction is generated to enter the second operating state after maintenance is completed.
[0012] In one implementation of the present application, the multiple visualization monitoring devices are symmetrically installed in the soil compartment of the tunnel boring machine, and the visual field corresponding to each of the visualization monitoring devices at least includes the area of all cutterhead components of the tunnel boring machine; the visualization monitoring equipment includes at least a high-precision industrial camera and a laser scanning device for collecting visual data; the visual data includes at least a two-dimensional image and three-dimensional scanning data obtained based on structured light scanning technology.
[0013] In one implementation of the present application, constructing a three-dimensional model of a cutter head assembly based on the first visual data specifically includes:
[0014] Preprocessing the three-dimensional scanning data in the first visual data to generate an initial three-dimensional point cloud model; the preprocessing includes: point cloud denoising, point cloud filtering, and point cloud registration;
[0015] The point cloud data in the initial three-dimensional point cloud model is converted into a surface model by using a preset B-spline surface fitting algorithm;
[0016] Based on the Marching Cubes algorithm and the surface model, a triangular mesh model is generated;
[0017] The two-dimensional image in the first visual data is used as texture information and mapped onto the surface of the triangular mesh model to construct a three-dimensional model of the cutter head assembly.
[0018] In one implementation of the present application, based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly, a shutdown maintenance strategy for the tunnel boring machine is generated, specifically including:
[0019] Performing point cloud alignment of the second visual data with the three-dimensional model of the cutter head assembly by an iterative closest point algorithm;
[0020] After the point cloud alignment process, the deviation of the point cloud data corresponding to the second visual data and the three-dimensional model of the cutter head assembly is calculated, so as to determine the deformation data of each cutter head assembly according to the point cloud data deviation and the finite element analysis method; the point cloud data deviation is obtained based on the Hausdorff distance;
[0021] Extracting visual features of the cutter head assembly through a convolutional neural network based on the second visual data and a pre-trained wear prediction model, and processing output results of the visual features of the cutter head assembly through a long short-term memory artificial neural network to determine a current predicted wear amount curve of each cutter head assembly;
[0022] Based on the deformation data, the current predicted wear curves and the analysis parameter set corresponding to the three-dimensional model of the cutter disc assembly, the shutdown maintenance strategy of each cutter disc assembly is determined; wherein the analysis parameter set includes the analysis parameters for characterizing the remaining service life of the cutter disc assembly; the analysis parameters include at least a deformation threshold and a wear threshold; the shutdown maintenance strategy includes at least one or more of maintenance time, replacement of worn components and adjustment of operating parameters.
[0023] In one implementation of the present application, the method further includes:
[0024] Obtain excavation condition data from pre-deployed monitoring sensors;
[0025] Inputting the excavation condition data into a pre-trained excavation condition analysis model to generate an excavation condition feature vector;
[0026] Calculating the cosine similarity between the tunneling condition feature vector and the current control parameter feature vector;
[0027] The working condition maintenance strategy is determined based on the cosine similarity and its changing trend and the preset working condition maintenance list, and is added to the shutdown maintenance strategy.
[0028] In one implementation of the present application, updating the cutter head assembly three-dimensional model specifically includes:
[0029] Preprocessing the point cloud data in the updated first visual data to obtain updated visual data;
[0030] Determine a point cloud transformation matrix of the updated visual data and the three-dimensional model of the cutter head assembly by an iterative closest point algorithm, so as to perform point cloud registration of the updated visual data and the three-dimensional model of the cutter head assembly according to the point cloud transformation matrix;
[0031] The three-dimensional model after point cloud registration is processed by an adaptive filtering algorithm, and the three-dimensional model after filtering and the corresponding two-dimensional image are fused to obtain an updated three-dimensional model of the cutter head assembly.
[0032] In one implementation of the present application, based on the updated three-dimensional model of the cutter head assembly, determining the morphological change amount corresponding to each cutter head assembly specifically includes:
[0033] Determine a rectangular area frame of a corresponding shape according to the component type corresponding to each cutter head component in the updated three-dimensional model of the cutter head component;
[0034] Determine, according to the rectangular area frame, a first form of the corresponding cutter head assembly in the three-dimensional model of the cutter head assembly before updating, and a second form of the corresponding cutter head assembly in the three-dimensional model of the cutter head assembly after updating;
[0035] The first morphology and the second morphology are input into a pre-trained wear prediction model to determine the morphology change amount according to the output result; the morphology change amount includes at least wear depth, wear area and surface morphology change.
[0036] In one implementation of the present application, after acquiring the second visual data in the second operating state in real time, the method further includes:
[0037] Determine the occluder and the type of occluder in the second visual data by using a preset image recognition model;
[0038] Generate abnormal warning information of the soil tank environment according to the type of the obstruction and the preset abnormal warning conditions;
[0039] The blocked cutter disc components corresponding to the blocking object are removed from the second visual data, so as to perform the first analysis process on the unblocked cutter disc components in the current second visual data.
[0040] In one implementation of the present application, the method further includes:
[0041] When the maintenance is completed, receiving maintenance feedback information from the user terminal;
[0042] According to the maintenance feedback information, it is determined whether to update and obtain the first visual data to update the three-dimensional model of the cutter head assembly.
[0043] On the other hand, an embodiment of the present application further provides a visual monitoring device for a cutterhead assembly of a tunnel boring machine, the device comprising:
[0044] At least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform a visual monitoring method for a tunnel boring machine cutter head assembly as described above.
[0045] Compared with the prior art, the present invention has the following significant effects:
[0046] Through the above scheme, the present application can collect visual data of the cutterhead assembly normally through visual monitoring equipment without being affected by the environment in the soil compartment, and then establish a highly accurate three-dimensional model of the cutterhead assembly. At the same time, each cutterhead assembly is analyzed and processed, and combined with the three-dimensional model of the cutterhead assembly and the visual monitoring equipment, the life of the cutterhead assembly is continuously and accurately evaluated during real-time operation, so as to formulate a reasonable shutdown maintenance strategy. Visual monitoring of the tunnel boring machine during operation and updating of the three-dimensional model of the cutterhead structure according to the actual construction situation are realized, which can meet the actual needs of the operator and improve the actual construction efficiency of the tunnel boring machine.
[0047] In addition, the use of visual monitoring equipment can realize the capture and analysis functions of fallen objects in the soil cabin, timely warn of abnormal conditions in the soil cabin, and improve construction safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0049] Figure 1 A schematic flow chart of a visual monitoring method for a cutterhead assembly of a tunnel boring machine in an embodiment of the present application;
[0050] Figure 2 A structural schematic diagram of a visualization monitoring device in a visualization monitoring method of a cutter head assembly of a tunnel boring machine in an embodiment of the present application;
[0051] Figure 3 Another structural schematic diagram of a visualization monitoring device in a visualization monitoring method of a cutter head assembly of a tunnel boring machine in an embodiment of the present application;
[0052] Figure 4 It is a structural schematic diagram of a visual monitoring device for a cutter head assembly of a tunnel boring machine in an embodiment of the present application.
[0053] List of parts and reference numerals:
[0054] 1-1, sealed box; 1-2, protective panel; 1-3, window protection glass; 2-1, camera; 2-2, fill light; 2-3, multi-axis gimbal; 3-1, bearing seat; 3-2, coupler; 3-3, ball screw; 3-4, scraper; 3-5, synchronous belt; 3-6, motor; 4-1, spray hole; 4-2, spray chamber; 4-3, automatic control ball valve; 4-4, three-way pipeline. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0056] The embodiments of the present application provide a visual monitoring method and device for the cutter head assembly of a tunnel boring machine, which is used to solve the technical problems that the current tunnel boring machine is difficult to perform visual monitoring during operation, and the three-dimensional model of the cutter head structure cannot be updated according to the actual construction situation, resulting in the inability to meet the actual needs of the operator and affecting the actual construction efficiency of the tunnel boring machine.
[0057] The following describes in detail various embodiments of the present application in conjunction with the accompanying drawings.
[0058] The present application embodiment provides a visual monitoring method for a tunnel boring machine cutterhead assembly, such as Figure 1 As shown, the method may include steps S101-S106:
[0059] S101, a server obtains first visual data from multiple visualization monitoring devices.
[0060] Each visualization monitoring device is used to collect visual data of the cutterhead assembly at different angles. The visualization monitoring device at least includes an image acquisition unit provided with a multi-axis pan-tilt head. The first visual data includes visual data of the cutterhead assembly of the tunnel boring machine in a first operating state. The first operating state is the state when the cutterhead assembly of the tunnel boring machine is idling.
[0061] The first visual data may be data that has been cleaned, denoised, standardized and normalized. The first visual data may include data during a rotation of the cutter head.
[0062] It should be noted that the server as the execution subject of the visual monitoring method of the cutter head assembly of the tunnel boring machine is only an example, and the execution subject is not limited to the server, and this application does not make specific restrictions on this. The execution subject as the host computer of the visual monitoring device can be a single-chip microcomputer, a personal computer (PC) and other devices in addition to the server.
[0063] Multiple visual monitoring devices are symmetrically installed in the soil compartment of the tunnel boring machine, and the visual field formed by each visual monitoring device at least includes the area of all cutterhead components of the tunnel boring machine. The visual monitoring device at least includes a high-precision industrial camera and a laser scanning device for collecting visual data. The visual data at least includes a two-dimensional image and a three-dimensional scanning data obtained based on structured light scanning technology.
[0064] The number of the above-mentioned multiple visual monitoring devices can be two, which are arranged inside the soil chamber of the tunnel boring machine, such as installed at symmetrical positions on the top of the soil chamber of the tunnel boring machine. The visual field formed by the two visual monitoring devices can collect images of all cutterhead components.
[0065] like Figure 2 , Figure 3 As shown, the visual monitoring equipment includes: a protective component consisting of a sealed box 1-1, a protective panel 1-2 and a window protective glass 1-3; an image acquisition unit consisting of a camera 2-1, a fill light 2-2 and a multi-axis gimbal 2-3, and fixedly installed in the sealed box 1-1; a spray cleaning component consisting of a bearing seat 3-1, a coupler 3-2, a ball screw 3-3, a scraper 3-4, a synchronous belt 3-5, a motor 3-6, a spray hole 4-1, a spray cavity 4-2, an automatic control ball valve 4-3 and a three-way pipe 4-4, and fixedly installed on the protective panel 1-2. The camera 2-1 includes at least a high-precision industrial camera and a laser scanning device.
[0066] The sealed box 1-1 and the protective panel 1-2 are connected by bolts and sealed with a sealing gasket. A rectangular window is set in the center of the protective panel 1-2 and a window protective glass 1-3 is installed to prevent soil cabin impurities from contaminating the image acquisition unit and other equipment. The volume of the sealed box 1-1 matches the volume of the image acquisition unit on the basis of leaving enough space for water, electricity and gas lines.
[0067] When the visual monitoring equipment is running, the fill light 2-2 is turned on to improve the problem of insufficient lighting in the soil cabin of the tunnel boring machine. The multi-axis gimbal 2-3 drives the camera 2-1 to rotate to expand the visual range of the camera 2-1 and avoid monitoring blind spots. In the embodiment of the present application, the multi-axis gimbal 2-3 can be a two-axis gimbal. The multi-axis gimbal 2-3 includes a first swivel seat, which is connected to the camera and the rotation axis of the first swivel seat is perpendicular to the optical axis of the camera. The second swivel seat is fixed to the side wall of the sealed box and is rotatably connected to the first swivel seat. The rotation axis of the second swivel seat is respectively perpendicular to the rotation axis of the first swivel seat and the optical axis of the camera. The fill light 2-2 is bolted and installed on the inner wall of the protective panel 1-2. The lamp beads are close to the window protective glass 1-3. The irradiation range is larger than the visual range of the camera 2-1, and can provide a high lumen light source to improve the clarity of the image data.
[0068] The two ends of the ball screw 3-3 are provided with journals for mounting the inner ring of the bearing, and the end close to the spray chamber 4-2 is connected to the synchronous belt 3-5. The scraper 3-4 is connected to the ball screw 3-3 through the coupler 3-2, and a replaceable rubber plate is fixed at the bottom, and the rubber plate is pressed against the surface of the window protection glass 1-3 at an appropriate angle. The ball screw 3-3 is connected to the output shaft of the motor 3-6 through a coupling, and the motor 3-6 drives the ball screw 3-3 to rotate, thereby driving the scraper 3-4 to move up and down to clean the window protection glass 1-3. Specifically, two ball screws 3-3 are installed in parallel on the narrow sides of the window protective glass 1-3. The motor 3-6 drives the ball screw 3-3 to rotate and drives the scraper 3-4 to move up and down through the coordinated action of the synchronous belt 3-5 and the coupling, so as to scrape away impurities on the window protective glass 1-3; in addition, the spray chamber 4-2 is installed on the upper part of the protective panel 1-2 and a spray hole 4-1 is arranged in the direction of the window protective glass 1-3. A three-way pipe 4-4 is arranged at the bottom of the spray chamber 4-2 to connect the water supply and air supply equipment at the rear. The spray cleaning component can switch the water and air through the three-way valve to realize the cleaning of the protective glass.
[0069] That is to say, through the above-mentioned visual monitoring equipment, on the one hand, clear visual data can be obtained during the real-time operation of the tunnel boring machine, which is not affected by the environment in the soil compartment and does not hinder the normal operation of the tunnel boring machine cutter head; on the other hand, the complete visual data of the cutter head assembly can be obtained at the maximum angle.
[0070] S102, the server constructs a three-dimensional model of the cutter head assembly based on the first visual data, and displays it on the user terminal.
[0071] In the embodiment of the present application, based on the first visual data, a three-dimensional model of the cutter head assembly is constructed, which specifically includes:
[0072] Preprocess the 3D scan data in the first visual data to generate an initial 3D point cloud model. Preprocessing includes: point cloud denoising, point cloud filtering, and point cloud registration. Convert the point cloud data in the initial 3D point cloud model to a surface model using a preset B-spline surface fitting algorithm. Generate a triangular mesh model based on the Marching Cubes algorithm and the surface model. Use the 2D image in the first visual data as texture information and map it to the surface of the triangular mesh model to construct a 3D model of the cutter head assembly.
[0073] In other words, the present application can perform point cloud denoising, point cloud filtering, and point cloud registration on the first visual data from the visualization monitoring device, and splice the first visual data from different visualization monitoring devices at the same time to obtain the three-dimensional scanning data of the complete tool disc assembly and generate an initial three-dimensional point cloud model. Subsequently, the server uses the B-spline curve fitting algorithm to select a set of control points in the initial three-dimensional point cloud model based on the preset control point selection rules. The control points are used to determine the shape and accuracy of the B-spline surface. Subsequently, the B-spline basis function is calculated based on the preset node vectors and control points, and then the B-spline basis function and control points are used to calculate the points on the B-spline surface by approximation, thereby fitting the B-spline surface and obtaining a surface model.
[0074] Furthermore, the server discretizes the B-spline surface in the surface model into a series of isosurface data, and traverses the preset voxels (preset small cubes) in the three-dimensional space through the Marching Cubes algorithm to check the intersection of each voxel and the isosurface. Then, using the predefined triangle template and linear interpolation method, triangular facets are generated in each voxel, and the triangular facets in adjacent voxels are connected to form a complete triangular network model. Subsequently, the two-dimensional image is fused with the triangular mesh model as texture information to construct a three-dimensional model of the cutter head assembly fused from multiple source data. In order to restore the three-dimensional form of the cutter head assembly with high precision, provide detailed structural features, and provide an accurate model basis for subsequent analysis.
[0075] S103, the server obtains the second visual data in the second operating state in real time.
[0076] After the construction of the above-mentioned three-dimensional model of the cutterhead assembly is completed, the second operating state can be entered, and the second operating state can be understood as the operating state when the tunnel boring machine is working normally. During the operation process, the visual monitoring device can work in real time to collect visual data of the cutterhead assembly in the second operating state.
[0077] In addition, after acquiring the second visual data in the second operating state in real time, the method further includes:
[0078] By using a preset image recognition model, the obstruction and the type of the obstruction in the second visual data are determined. According to the type of the obstruction and the preset abnormal warning condition, the abnormal warning information of the soil tank environment is generated. And the obstructed cutterhead component corresponding to the obstruction is removed from the second visual data, so as to perform the first analysis and processing on the unobstructed cutterhead component in the current second visual data.
[0079] In other words, the present application may pre-set an image recognition model, which may be a machine learning model that can identify obstructions and their types in the second visual data. Obstructions may be understood as fallen objects that obstruct the cutter disc assembly at the time of second visual data collection. The server may determine whether it is necessary to generate abnormal soil compartment environment alarm information based on the identified obstruction type and the pre-set abnormal warning conditions. Different types of obstructions correspond to different abnormal warning conditions, and the correspondence between abnormal warning conditions and types of obstructions is set by the user according to the actual scenario. The abnormal soil compartment environment alarm information may be in the form of sound or text. At the same time, the server will also crop the second visual data corresponding to the current moment, that is, retain the visual data of the cutter disc assembly that is not obstructed, and perform subsequent first analysis processing.
[0080] Through the above scheme, the environmental conditions in the soil compartment of the tunnel boring machine can be evaluated and abnormal conditions can be warned in time.
[0081] S104, the server generates a shutdown maintenance strategy for the tunnel boring machine based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly.
[0082] In the embodiment of the present application, based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly, a shutdown maintenance strategy for the tunnel boring machine is generated, specifically including:
[0083] The second visual data is aligned with the three-dimensional model of the cutter disc assembly by the iterative closest point algorithm (ICP). After the point cloud alignment process is calculated, the point cloud data deviation corresponding to the second visual data and the three-dimensional model of the cutter disc assembly is used to determine the deformation data of each cutter disc assembly according to the point cloud data deviation and the finite element analysis method. The point cloud data deviation is obtained based on the Hausdorff distance. According to the second visual data and the pre-trained wear prediction model, the visual features of the cutter disc assembly are extracted by the convolutional neural network, and the current predicted wear curve of each cutter disc assembly is determined according to the output result of the long short-term memory artificial neural network processing the visual features of the cutter disc assembly. Based on the analysis parameter set corresponding to each deformation data, each current predicted wear curve and the three-dimensional model of the cutter disc assembly, the shutdown maintenance strategy of each cutter disc assembly is determined. Among them, the analysis parameter set includes analysis parameters for characterizing the remaining service life of the cutter disc assembly. The analysis parameters include at least a deformation threshold and a wear threshold. The shutdown maintenance strategy includes at least one or more of the maintenance time, replacement of the wear component, and adjustment of the operating parameters.
[0084] That is to say, the present application can use the iterative closest point algorithm to perform point cloud alignment, and calculate the point cloud data deviations before and after the point cloud alignment, that is, the point cloud data deviation between the point cloud data of the tool disc in the initial tool disc assembly three-dimensional model and the point cloud data corresponding to the second visual data, which is obtained by calculating the Hausdorff distance between the first point cloud data set corresponding to the second visual data and the second point cloud data set corresponding to the tool disc assembly three-dimensional model. The tool disc assembly may include different cutting tools provided on the tool disc, such as a hob, a scraper, etc. The present application will also use the finite element analysis method to obtain the deformation data corresponding to each tool disc assembly according to the point cloud data deviation, and the deformation data can be understood as the degree of deformation of the tool disc assembly.
[0085] Furthermore, the present application will also be provided with a pre-trained wear prediction model, which can be obtained by training a number of historical cutter disc component wear data samples. The wear prediction model can be composed of a convolutional neural network and a long short-term memory artificial neural network. Among them, the convolutional neural network is used to extract the visual features of the current cutter disc component, and the long short-term memory artificial neural network can process the visual features of the cutter disc component to analyze and obtain the current predicted wear amount curve of each cutter disc component in the second operating state, including the wear amount in the past, present and future time periods.
[0086] Furthermore, the server compares the deformation data with the deformation threshold of the corresponding cutter disc assembly in the analysis parameter set, and compares the current predicted wear curve with the corresponding wear threshold to determine the remaining service life (or next maintenance time) of the cutter disc assembly. According to the remaining service life obtained and the identification of the cutter disc assembly, the shutdown maintenance strategy in the pre-set shutdown maintenance strategy list is matched. The initial analysis parameter set of this application can be pre-set by the user, and this application does not make specific restrictions on this.
[0087] In one embodiment of the present application, the method further includes:
[0088] Acquire the excavation condition data from the pre-deployed monitoring sensors. Input the excavation condition data into the pre-trained excavation condition analysis model to generate the excavation condition feature vector. Calculate the cosine similarity between the excavation condition feature vector and the current control parameter feature vector. According to the cosine similarity and its change trend, the preset condition maintenance list, the determined condition maintenance strategy is added to the shutdown maintenance strategy.
[0089] Among them, the pre-deployed monitoring sensors may include force sensors, displacement sensors, speed sensors, vibration sensors, etc., which are not specifically limited in this application. The monitoring sensors can be used to collect data related to the tunnel boring machine body, such as tunneling speed, propulsion force, cutter head speed, rock hardness, geological conditions, etc.
[0090] This application can pre-train a tunneling condition analysis model. First, a large amount of operation data (such as tunneling speed, propulsion force, cutter head speed, etc.) and geological data (such as rock hardness, etc.) during the tunneling process are collected, and the data is labeled to distinguish normal and abnormal conditions; then feature engineering is performed to extract key features, and the feature set is optimized through feature selection and dimensionality reduction (such as PCA); then an artificial intelligence model combining a long short-term memory network and a convolutional neural network is designed, and the training set is used for training, and the validation set is used for hyperparameter tuning. Finally, the performance of the model is evaluated using the test set, and the tunneling condition analysis model is obtained. Through the tunneling condition analysis model, the tunneling condition feature vector corresponding to the tunneling condition data can be output, and then the server calculates the cosine similarity between the tunneling condition feature vector and the corresponding current control parameter feature vector at this time. The cosine similarity evaluates the matching degree between the existing control parameters and the current tunneling condition. The higher the similarity, the more suitable the control parameters are for the current working conditions. When the similarity is continuously lower than the threshold or the trend of change is showing a downward trend, it indicates that the matching degree between the excavation working condition and the control parameters is poor, and there may be a risk of equipment wear or failure. At this time, according to the preset working condition maintenance list, a preventive maintenance plan should be formulated, including regular inspections, replacement of worn parts, optimization of equipment performance, etc., to reduce the probability of failure. If the similarity is extremely low or drops sharply, it indicates that the matching degree between the excavation working condition and the control parameters is seriously mismatched, and the equipment may have a serious failure or is about to fail. At this time, a shutdown maintenance decision should be made immediately, and the equipment should be shut down for maintenance to avoid further damage or safety accidents.
[0091] The changing trend and value of the cosine similarity correspond to different working condition maintenance strategies in the preset working condition maintenance list. For example, if the cosine similarity is continuously less than the preset working condition evaluation threshold, it indicates that maintenance is required.
[0092] In addition, the server can pre-set preset weights for the working condition maintenance and the above-mentioned morphological change maintenance, respectively, for calculating the final maintenance time. The above-mentioned preset working condition evaluation threshold and preset working condition maintenance list can be set by the user according to actual needs, and this application does not make specific restrictions on this.
[0093] Through the above scheme, a shutdown maintenance strategy can be formulated in combination with the working conditions and the actual deformation and wear of the cutter disc assembly, so as to perform maintenance work on the cutter disc assembly more reasonably.
[0094] S105, when the tunnel boring machine meets the corresponding conditions of the shutdown maintenance strategy, the server generates a first control instruction for entering a first operating state, and updates the acquired first visual data according to the execution state of the first control instruction, so as to update the three-dimensional model of the cutter head assembly.
[0095] If the tunnel boring machine reaches the maintenance time (i.e., the overhaul time), the corresponding condition of the shutdown maintenance strategy is met, and the server generates a first control instruction and sends it to the management terminal. After agreeing to execute the first control instruction, the management terminal determines that the execution state is executable, so that the management terminal controls the tunnel boring machine to enter the first operating state, and re-acquires the first visual data after the cutterhead assembly is cleaned. The three-dimensional model of the cutterhead assembly is updated through the re-acquired first visual data.
[0096] In the embodiment of the present application, updating the three-dimensional model of the cutter head assembly specifically includes:
[0097] The point cloud data in the updated first visual data is preprocessed to obtain updated visual data. The point cloud transformation matrix of the updated visual data and the three-dimensional model of the cutter head assembly is determined by an iterative closest point algorithm, so as to perform point cloud registration of the updated visual data and the three-dimensional model of the cutter head assembly according to the point cloud transformation matrix. The three-dimensional model after point cloud registration is processed by an adaptive filtering algorithm, and the three-dimensional model after filtering and the corresponding two-dimensional image are fused to obtain an updated three-dimensional model of the cutter head assembly.
[0098] That is to say, the present application can pre-process the point cloud data in the updated first visual data, which may include operations such as denoising and downsampling. Then, the iterative nearest point algorithm is used to iteratively find the optimal correspondence between the two groups of point clouds, and the point cloud transformation matrix (including rotation and translation) is calculated so that one group of point clouds can match the other group of point clouds to the greatest extent after transformation. The point cloud transformation matrix is used to perform point cloud registration of the three-dimensional model to ensure the spatial consistency of the new and old data.
[0099] Furthermore, the server will further process the point cloud data using an adaptive filtering algorithm to enhance the model's detailed information, including edge sharpening and texture reconstruction, so as to obtain an updated three-dimensional model of the cutterhead assembly after one run.
[0100] Through the above scheme, a relatively static three-dimensional model of the cutter disc assembly can be established by combining the image of the cutter disc assembly without obstruction in the idling state, and a real-time and flexible shutdown maintenance strategy can be formulated by combining the image of the cutter disc assembly with obstruction and complex working conditions in the working state. This can not only realize visual monitoring, but also more flexibly update the three-dimensional model to improve utilization efficiency.
[0101] S106, the server determines the morphological change amount corresponding to each cutter disc component based on the updated three-dimensional model of the cutter disc assembly, and synchronously displays it on the three-dimensional model of the cutter disc assembly, so as to update the analysis parameters corresponding to the first analysis processing and / or generate corresponding cutter disc assembly monitoring and alarm information according to each morphological change amount, until a second control instruction to enter a second operating state is generated after the maintenance is completed.
[0102] During the tunnel boring machine's mission, steps S101-S106 will be repeated continuously to achieve visual monitoring of the tool assembly.
[0103] In the embodiment of the present application, based on the updated three-dimensional model of the cutter head assembly, the morphological change amount corresponding to each cutter head assembly is determined, specifically including:
[0104] According to the component type corresponding to each cutter disc component in the updated cutter disc component three-dimensional model, a rectangular area frame of a corresponding shape is determined. According to the rectangular area frame, a first form of the corresponding cutter disc component in the cutter disc component three-dimensional model before the update and a second form in the cutter disc component three-dimensional model after the update are determined. The first form and the second form are input into a pre-trained wear prediction model to determine the amount of morphological change according to the output result. The amount of morphological change includes at least wear depth, wear area and surface morphological change.
[0105] That is to say, the present application can determine the component type of the cutter disc assembly according to the position and shape of the cutter disc assembly in the three-dimensional model of the cutter disc assembly, and generate a rectangular area frame that matches the shape of the cutter disc assembly, and the rectangular area frame can contain the corresponding cutter disc assembly image. The server uses each rectangular area frame to select the corresponding cutter disc assembly, and determines the first shape of a cutter disc assembly in the three-dimensional model of the cutter disc assembly before the update, and determines the second shape of the cutter disc assembly in the three-dimensional model of the cutter disc assembly after the update. Then, the pre-trained wear prediction model is used to calculate the change in the second shape relative to the first shape to obtain the shape change.
[0106] The wear state of the cutterhead assembly of the tunnel boring machine can be fully monitored and accurately predicted through the morphological change, thus improving the safety and efficiency of the equipment operation. The 3D model of the cutterhead assembly displayed on the user terminal can include the morphological change of the cutterhead assembly, so that the user can more intuitively see the deformation and wear morphological change of the cutterhead assembly.
[0107] This application will also use the morphological change amount to match the analysis parameters corresponding to each cutter disc component from a preset analysis parameter update list, so as to update the analysis parameters when the matched analysis parameters are inconsistent with the previous analysis parameters. This allows for a more flexible formulation of shutdown maintenance strategies. If the morphological change amount is greater than a preset change amount threshold, a cutter disc component monitoring alarm message can be generated to replace the cutter disc component. The cutter disc component monitoring alarm message can be text or sound, and this application does not specifically limit this.
[0108] For example, if the original analysis parameter is A, and the analysis parameter corresponding to the morphological change is B, if the original analysis parameter is not updated, the next shutdown maintenance strategy may need to wait T1 hours. Maintenance in T1 hours may result in improper tool maintenance and affect efficiency. Using the updated analysis parameter B, it may take T2 hours to wait, which is more suitable for the tool maintenance time interval.
[0109] Additionally, it includes:
[0110] When the maintenance is completed, maintenance feedback information is received from the user terminal. According to the maintenance feedback information, it is determined whether to update and obtain the first visual data to update the three-dimensional model of the cutter head assembly.
[0111] The user terminal can be understood as the user's mobile phone, computer and other devices, and the management terminal can be understood as the computer and other devices operated by the management personnel. This application does not make specific limitations on this. After the maintenance of the cutter head assembly is completed, the user can send maintenance feedback information to the server to determine whether to replace the parts, so that the server determines whether to update and obtain the first visual data, thereby updating the three-dimensional model of the cutter head assembly. If there is no replacement, a second control instruction can be generated and sent to the management terminal to enter the second operating state.
[0112] Through the above scheme, the present application can collect visual data of the cutterhead assembly normally through visual monitoring equipment without being affected by the environment in the soil compartment, and then establish a highly accurate three-dimensional model of the cutterhead assembly. At the same time, each cutterhead assembly is analyzed and processed, and combined with the three-dimensional model of the cutterhead assembly and the visual monitoring equipment, the life of the cutterhead assembly is continuously and accurately evaluated during real-time operation, so as to formulate a reasonable shutdown maintenance strategy. Visual monitoring of the tunnel boring machine during operation and updating of the three-dimensional model of the cutterhead structure according to the actual construction situation are realized, which can meet the actual needs of the operator and improve the actual construction efficiency of the tunnel boring machine.
[0113] In addition, the use of visual monitoring equipment can realize the capture and analysis functions of fallen objects in the soil cabin, timely warn of abnormal conditions in the soil cabin, and improve construction safety.
[0114] Figure 4 A schematic diagram of the structure of a visual monitoring device for a cutterhead assembly of a tunnel boring machine provided in an embodiment of the present application, such as Figure 4 As shown, the device includes:
[0115] At least one processor; and a memory in communication with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:
[0116] Acquire first visual data from a plurality of visual monitoring devices. Each visual monitoring device is used to acquire visual data of a cutterhead assembly at different angles. The visual monitoring device at least includes an image acquisition unit provided with a multi-axis pan-tilt head. The first visual data includes visual data of a cutterhead assembly of a tunnel boring machine in a first operating state. Based on the first visual data, a three-dimensional model of the cutterhead assembly is constructed and displayed on a user terminal. Acquire second visual data in a second operating state in real time. Based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly, a shutdown maintenance strategy of the tunnel boring machine is generated. When the tunnel boring machine meets the corresponding conditions of the shutdown maintenance strategy, a first control instruction for entering the first operating state is generated, so as to update the acquired first visual data according to the execution state of the first control instruction, so as to update the three-dimensional model of the cutterhead assembly. Based on the updated three-dimensional model of the cutterhead assembly, the morphological change amount corresponding to each cutterhead assembly is determined, and synchronously displayed on the three-dimensional model of the cutterhead assembly, so as to update the analysis parameters corresponding to the first analysis and processing according to each morphological change amount and / or generate corresponding cutterhead assembly monitoring alarm information, until the second control instruction for entering the second operating state is generated after the maintenance is completed.
[0117] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0118] The device and method provided in the embodiments of the present application correspond one to one, and therefore, the device also has similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.
[0119] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0120] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A visual monitoring method for a tunnel boring machine cutterhead assembly, characterized in that: The method comprises: Acquire first visual data from a plurality of visual monitoring devices; wherein each of the visual monitoring devices is used to collect visual data of a cutterhead assembly at different angles; the visual monitoring device at least comprises an image acquisition unit provided with a multi-axis pan-tilt head; the first visual data comprises visual data of the cutterhead assembly of the tunnel boring machine in a first operating state; Based on the first visual data, construct a three-dimensional model of the cutter head assembly and display it on a user terminal; Acquire second visual data in a second operating state in real time; generating a shutdown maintenance strategy for the tunnel boring machine based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly; When the tunnel boring machine meets the conditions corresponding to the shutdown maintenance strategy, a first control instruction for entering the first operating state is generated, so as to update the first visual data acquired according to the execution state of the first control instruction, so as to update the three-dimensional model of the cutterhead assembly; Based on the updated three-dimensional model of the cutter disc assembly, the morphological change amount corresponding to each cutter disc assembly is determined and synchronously displayed on the three-dimensional model of the cutter disc assembly, so as to update the analysis parameters corresponding to the first analysis processing and / or generate corresponding cutter disc assembly monitoring and alarm information according to each morphological change amount, until a second control instruction is generated to enter the second operating state after maintenance is completed.
2. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: The multiple visualization monitoring devices are symmetrically installed in the soil compartment of the tunnel boring machine, and the visual field corresponding to each of the visualization monitoring devices at least includes the area of all cutterhead components of the tunnel boring machine; the visualization monitoring equipment at least includes a high-precision industrial camera and a laser scanning device for collecting visual data; the visual data at least includes a two-dimensional image and three-dimensional scanning data obtained based on structured light scanning technology.
3. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 2, characterized in that: Based on the first visual data, a three-dimensional model of the cutter head assembly is constructed, specifically including: Preprocessing the three-dimensional scanning data in the first visual data to generate an initial three-dimensional point cloud model; the preprocessing includes: point cloud denoising, point cloud filtering, and point cloud registration; The point cloud data in the initial three-dimensional point cloud model is converted into a surface model by using a preset B-spline surface fitting algorithm; Based on the Marching Cubes algorithm and the surface model, a triangular mesh model is generated; The two-dimensional image in the first visual data is used as texture information and mapped onto the surface of the triangular mesh model to construct a three-dimensional model of the cutter head assembly.
4. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: Based on the first analysis and processing of the second visual data and the three-dimensional model of the cutterhead assembly, a shutdown maintenance strategy for the tunnel boring machine is generated, specifically comprising: Performing point cloud alignment of the second visual data with the three-dimensional model of the cutter head assembly by an iterative closest point algorithm; After the point cloud alignment process, the deviation of the point cloud data corresponding to the second visual data and the three-dimensional model of the cutter head assembly is calculated, so as to determine the deformation data of each cutter head assembly according to the point cloud data deviation and the finite element analysis method; the point cloud data deviation is obtained based on the Hausdorff distance; Extracting visual features of the cutter head assembly through a convolutional neural network based on the second visual data and a pre-trained wear prediction model, and processing output results of the visual features of the cutter head assembly through a long short-term memory artificial neural network to determine a current predicted wear amount curve of each cutter head assembly; Based on the deformation data, the current predicted wear curves and the analysis parameter set corresponding to the three-dimensional model of the cutter disc assembly, the shutdown maintenance strategy of each cutter disc assembly is determined; wherein the analysis parameter set includes the analysis parameters for characterizing the remaining service life of the cutter disc assembly; the analysis parameters include at least a deformation threshold and a wear threshold; the shutdown maintenance strategy includes at least one or more of maintenance time, replacement of worn components and adjustment of operating parameters.
5. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 4, characterized in that: The method further comprises: Obtain excavation condition data from pre-deployed monitoring sensors; Inputting the excavation condition data into a pre-trained excavation condition analysis model to generate an excavation condition feature vector; Calculating the cosine similarity between the tunneling condition feature vector and the current control parameter feature vector; The working condition maintenance strategy is determined based on the cosine similarity and its changing trend and the preset working condition maintenance list, and is added to the shutdown maintenance strategy.
6. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: Updating the three-dimensional model of the cutter head assembly specifically includes: Preprocessing the point cloud data in the updated first visual data to obtain updated visual data; Determine a point cloud transformation matrix of the updated visual data and the three-dimensional model of the cutter head assembly by an iterative closest point algorithm, so as to perform point cloud registration of the updated visual data and the three-dimensional model of the cutter head assembly according to the point cloud transformation matrix; The three-dimensional model after point cloud registration is processed by an adaptive filtering algorithm, and the three-dimensional model after filtering and the corresponding two-dimensional image are fused to obtain an updated three-dimensional model of the cutter head assembly.
7. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: Based on the updated three-dimensional model of the cutter head assembly, the shape change amount corresponding to each cutter head assembly is determined, specifically including: Determine a rectangular area frame of a corresponding shape according to the component type corresponding to each cutter head component in the updated three-dimensional model of the cutter head component; Determine, according to the rectangular area frame, a first form of the corresponding cutter head assembly in the three-dimensional model of the cutter head assembly before updating, and a second form of the corresponding cutter head assembly in the three-dimensional model of the cutter head assembly after updating; The first morphology and the second morphology are input into a pre-trained wear prediction model to determine the morphology change amount according to the output result; the morphology change amount includes at least wear depth, wear area and surface morphology change.
8. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: After acquiring the second visual data in the second operating state in real time, the method further includes: Determine the occluder and the type of occluder in the second visual data by using a preset image recognition model; Generate abnormal warning information of the soil tank environment according to the type of the obstruction and the preset abnormal warning conditions; The blocked cutter disc components corresponding to the blocking object are removed from the second visual data, so as to perform the first analysis process on the unblocked cutter disc components in the current second visual data.
9. A visual monitoring method for a tunnel boring machine cutterhead assembly according to claim 1, characterized in that: The method further comprises: When the maintenance is completed, receiving maintenance feedback information from the user terminal; According to the maintenance feedback information, it is determined whether to update and obtain the first visual data to update the three-dimensional model of the cutter head assembly.
10. A visual monitoring device for a cutterhead assembly of a tunnel boring machine, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a visual monitoring method for a tunnel boring machine cutter head assembly as described in any one of claims 1 to 9.
