A method and device for detecting ground protection of cutting head of tunnel boring machine

Abstract

The present invention discloses a method and device for detecting the grounding protection of the cutting head of a roadheader, the method comprising: obtaining real-time point cloud data of the roadheader; determining the overlap between the real-time point cloud data and the point cloud data of the cutting head obtained in advance when the cutting head has landed; when the overlap exceeds a preset overlap threshold, calculating the distance between the cutting head and the ground; when the distance between the cutting head and the ground exceeds a preset distance threshold, determining that the cutting head has not landed, triggering an alarm that the cutting head has not landed; when the distance between the cutting head and the ground does not exceed a preset distance threshold, determining that the cutting head has landed. The present invention compares the real-time point cloud data of the roadheader with the point cloud data of the cutting head obtained in advance to obtain the overlap, and combines the calculation of the distance between the cutting head and the ground to comprehensively judge whether the cutting head has landed in two ways, ensuring that it can accurately judge whether the cutting head has landed in different environments, thereby improving the accuracy of the judgment result.

Description

Technical Field

[0001] The invention belongs to the technical field of coal mine disaster prevention and control, and in particular relates to a ground protection detection method and device for a cutting head of a roadheader. Background Art

[0002] A roadheader is a machine used to dig tunnels under flat ground. It can be used in mines, underground tunnels and other scenarios. After the roadheader is finished working, the cutting head of the roadheader is required to be on the ground as much as possible and a protective cover is installed.

[0003] The existing method of identifying whether the cutting head of a tunnel boring machine has landed and installing a protective cover mainly involves target detection through monitoring video, detecting the coordinates of the cutting head area and the protective cover area, judging the overlap, and alarming when the overlap is less than a threshold. Alternatively, an inclination sensor is installed on the boom of the tunnel boring machine to monitor the inclination of the boom in real time, and alarming when the inclination is greater than a threshold. The above method has low accuracy and cannot adapt to the different working environments of the tunnel boring machine. Summary of the invention

[0004] An object of the present invention is to provide a method and device for detecting the grounding protection of a cutting head of a roadheader, which can solve the technical problem of low accuracy in identifying whether the cutting head of a roadheader has fallen to the ground in the prior art.

[0005] According to a first aspect of the present invention, a method for detecting the ground protection of a cutting head of a tunnel boring machine is provided, the method comprising: acquiring real-time point cloud data of the tunnel boring machine; determining the degree of overlap between the real-time point cloud data of the tunnel boring machine and the pre-acquired point cloud data of the cutting head when the cutting head falls to the ground; when the degree of overlap does not exceed a preset overlap threshold, determining that the cutting head has not fallen to the ground, and triggering a cutting head non-ground alarm; when the degree of overlap exceeds the preset overlap threshold, determining the overlapping part between the real-time point cloud data of the tunnel boring machine and the pre-acquired point cloud data of the cutting head when the cutting head falls to the ground as the real-time point cloud data of the cutting head, and calculating the distance between the cutting head and the ground according to the real-time point cloud data of the cutting head and the pre-acquired ground point cloud data; when the distance between the cutting head and the ground exceeds a preset distance threshold, determining that the cutting head has not fallen to the ground, and triggering a cutting head non-ground alarm; when the distance between the cutting head and the ground does not exceed the preset distance threshold, determining that the cutting head has fallen to the ground.

[0006] Optionally, determining the degree of overlap between the real-time point cloud data of the tunnel boring machine and the pre-acquired point cloud data of the cutting head when the cutting head lands, comprises: determining optimal transformation parameters based on the pre-acquired point cloud data of the part close to the cutting head when the cutting head lands and the real-time 3D point cloud data space, wherein the optimal transformation parameters are used to convert the pre-acquired point cloud data into the real-time 3D point cloud data space; converting the pre-acquired point cloud data of the cutting head when the cutting head lands into the real-time 3D point cloud data space by using the optimal transformation parameters to obtain the converted point cloud data; determining a first circumscribed cylindrical space corresponding to the converted point cloud data; determining the real-time point cloud data contained in the first circumscribed cylindrical space based on the first circumscribed cylindrical space and the real-time point cloud data of the tunnel boring machine; determining a second circumscribed cylindrical space corresponding to the real-time point cloud data in the first circumscribed cylindrical space; and determining the degree of overlap based on the first circumscribed cylindrical space and the second circumscribed cylindrical space.

[0007] Optionally, determining the first circumscribed cylindrical space corresponding to the converted point cloud data includes: processing the converted point cloud data by principal component analysis to obtain the principal component direction of the cutting head; projecting the converted point cloud data according to the principal component direction to obtain the effective length in the principal component direction; determining the maximum normal vector from the converted point cloud data to the principal component direction; determining the first circumscribed cylindrical space according to the principal component direction, the effective length in the principal component direction and the maximum normal vector, wherein the radius of the first circumscribed cylindrical space is the length of the maximum normal vector, the axial direction of the first circumscribed cylindrical space is the principal component direction, and the length of the first circumscribed cylindrical space along the principal component direction is the effective length in the principal component direction.

[0008] Optionally, determining the degree of overlap based on the first circumscribed cylindrical space and the second circumscribed cylindrical space includes: calculating the volume of the intersection of the first circumscribed cylindrical space and the second circumscribed cylindrical space; and taking the ratio between the volume of the intersection and the volume of the first circumscribed cylindrical space as the degree of overlap.

[0009] Optionally, when the overlap exceeds a preset overlap threshold, the overlapping part between the real-time point cloud data of the tunnel boring machine and the pre-acquired point cloud data of the cutting head when the cutting head lands is determined as the real-time point cloud data of the cutting head, and the distance between the cutting head and the ground is calculated based on the real-time point cloud data of the cutting head and the pre-acquired ground point cloud data, including: determining the ground 3D plane based on the ground point cloud data; determining the real-time point cloud data in the first circumscribed cylindrical space as the real-time point cloud data of the cutting head, calculating the normal vector from each point in the real-time point cloud data of the cutting head to the ground 3D plane, and taking the modulus; calculating the expected value of the modulus of the normal vector from each point to the ground 3D plane; and determining the expected value as the distance between the cutting head and the ground.

[0010] Optionally, a monitoring device is installed at a position where the cutting head of the tunnel boring machine can be observed on the excavation working face, the monitoring device includes a monitoring camera and a laser radar, the observation direction of the monitoring camera is the same as the observation direction of the laser radar, and after determining that the cutting head has landed, the method further includes: projecting the real-time point cloud data of the cutting head onto the real-time monitoring image taken by the monitoring camera according to pre-acquired projection parameters, and obtaining the first area of ​​the real-time point cloud data of the tunnel boring machine on the real-time monitoring image; analyzing the real-time monitoring image through a pre-trained target detection model to obtain an analysis result, wherein the analysis result is whether there is a protective cover of the cutting head in the real-time monitoring image; if the analysis result is that the protective cover of the cutting head does not exist in the real-time monitoring image, an alarm is triggered that the protective cover of the cutting head is not installed.

[0011] Optionally, the method also includes: if the analysis result is that the protective cover of the cutting head exists in the real-time monitoring image, determining a second area of ​​the protective cover of the cutting head in the real-time monitoring image; determining a degree of matching between the first area and the second area, wherein the degree of matching is a ratio of an overlapping area between the first area and the second area to an area of ​​the second area; when the degree of matching is less than a preset matching threshold, determining that the cutting head is not equipped with a protective cover, and triggering an alarm that the cutting head is not equipped with a protective cover.

[0012] Optionally, the projection parameters are determined in the following manner: collecting the 2D image data of the surveillance camera and the 3D point cloud data of the laser radar at the same time, wherein the 2D image coordinate system includes an X-axis and a Y-axis, the 3D point cloud coordinate system includes an XL axis, a YL axis and a ZL axis, the 3D point cloud data of the laser radar includes an XL axis component, a YL axis component and a ZL axis component, and the YL axis is the observation direction of the laser radar; projecting the XL axis component and the ZL axis component of the 3D point cloud data of the laser radar along the observation direction to the surveillance camera head; determine the correspondence between the 3D point cloud data of the laser radar and the 2D image data of the monitoring camera according to the projection result, and determine n groups of corresponding coordinates through coordinate sampling, wherein the i-th group of corresponding coordinates is expressed as [(Xi, Yi), (XLi, ZLi)], Xi and Yi respectively represent the X-axis coordinate and the Y-axis coordinate of the 2D image data of the monitoring camera, and XLi and ZLi respectively represent the XL-axis coordinate and the ZL-axis coordinate of the 3D point cloud data of the laser radar; determine the projection parameters according to the n groups of corresponding coordinates.

[0013] Optionally, determining the projection parameters according to the n groups of corresponding coordinates includes: determining a minimum value Xmin and a maximum value Xmax of an X-axis coordinate, a minimum value Ymin and a maximum value Ymax of a Y-axis coordinate, a minimum value XLmin and a maximum value XLmax of an XL-axis coordinate, and a minimum value ZLmin and a maximum value ZLmax of a ZL-axis coordinate from the n groups of corresponding coordinates; and calculating a scaling ratio scaleX of the 3D point cloud data to the 2D image data on the X-axis and a scaling ratio scaleY of the Y-axis, wherein:

[0014] ,

[0015] ,

[0016] The XL axis component and the ZL axis component of the 3D point cloud data in the n groups of corresponding coordinates are respectively multiplied by the X axis scaling ratio scaleX and the Y axis scaling ratio scaleY to obtain the scaled point cloud coordinates XLs and ZLs; the differences between the X axis coordinates and the Y axis coordinates of the 2D image data in the n groups of corresponding coordinates and the corresponding scaled point cloud coordinates XLs and ZLs are respectively calculated, and the expected values ​​dX and dY of the differences are calculated, wherein:

[0017] ,

[0018] ,

[0019] Among them, XLsi and ZLsi represent the coordinates of the i-th group of point clouds after scaling; the conversion formula from 3D point cloud data coordinates (XL, ZL) to 2D image data coordinates (X, Y) is:

[0020] ,

[0021] ,

[0022] The projection parameters include a scaling factor scaleX of the X axis, a scaling factor scaleY of the Y axis, and expected values ​​dX and dY of the difference values.

[0023] Optionally, the method also includes: collecting a preset number of pictures of cutting head protective covers; marking the protective cover area for each picture of the cutting head protective cover; splitting the marked pictures of the cutting head protective cover into a training set, a validation set and a test set according to a preset ratio; establishing a convolutional neural network model, and training the convolutional neural network model through the training set and the validation set; testing the trained convolutional neural network model through the test set with the F1 score as an indicator; and when the F1 score of the trained convolutional neural network model meets a preset condition, determining the trained convolutional neural network model as the target detection model.

[0024] According to a second aspect of the present invention, there is provided a device for detecting a ground-falling protection of a cutting head of a tunnel boring machine, comprising a processor and a memory, wherein an executable code is stored in the memory, and when the executable code is executed by the processor, the processor executes the method for detecting a ground-falling protection of a cutting head of a tunnel boring machine as described in the first aspect of the present invention.

[0025] The beneficial effect of the present invention is that the present invention compares the real-time 3D point cloud data of the tunnel boring machine with the pre-acquired point cloud data of the cutting head when the cutting head falls to the ground, obtains the overlap, and combines the calculated distance between the cutting head and the ground to comprehensively judge whether the cutting head has fallen to the ground in two ways, thereby ensuring that it is possible to accurately judge whether the cutting head has fallen to the ground in different environments, thereby improving the accuracy of the judgment result. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention is a flowchart of a method for detecting a ground protection of a cutting head of a tunnel boring machine according to an embodiment of the present invention.

[0027] Figure 2 Schematic diagram of a monitoring device in an embodiment of the present invention.

[0028] Figure 3 It is a schematic diagram of pre-scanning a tunnel boring machine at multiple angles in an embodiment of the present invention.

[0029] Figure 4 It is a schematic diagram of determining the principal component direction of the cutting head point cloud data in an embodiment of the present invention.

[0030] Figure 5 It is a schematic diagram of determining the circumscribed cylindrical space in an embodiment of the present invention.

[0031] Figure 6 It is a schematic diagram comparing the 3D plane of the ground point cloud and the real-time 3D point cloud of the cutting head in an embodiment of the present invention.

[0032] Figure 7 It is a schematic diagram of projecting a 3D point cloud onto a 2D plane in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0034] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the present invention and its application or use. Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, they should be considered part of the specification. In all examples shown and discussed herein, any specific value should be interpreted as being merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0035] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0036] In the specification and claims of the present invention, if the term "first" or "second" is involved, it may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the related objects are in an "or" relationship.

[0037] like Figure 1 As shown, this embodiment introduces a method for detecting ground protection of a cutting head of a tunnel boring machine, and the method includes steps 1100-1600.

[0038] Step 1100: Acquire real-time point cloud data of the tunnel boring machine.

[0039] Laser scanning equipment is installed on the excavation working surface to observe the position of the cutting head when the excavator stops working or is under maintenance or when the shift is changed. Figure 2 As shown, the laser scanning device may be a laser radar, and the laser radar may be used to scan the tunnel boring machine to obtain real-time point cloud data of the tunnel boring machine.

[0040] Step 1200: Determine the degree of overlap between the real-time point cloud data of the roadheader and the pre-acquired point cloud data of the cutting head when the cutting head lands.

[0041] The overlap degree is obtained by aligning the real-time point cloud data of the tunnel boring machine with the pre-acquired point cloud data of the cutting head when the cutting head lands. The higher the overlap degree, the greater the possibility that the cutting head of the tunnel boring machine has landed at the current moment.

[0042] The specific steps of pre-acquiring the point cloud data of the cutting head when the cutting head falls to the ground include steps S1-S10.

[0043] Step S1: Control the tunnel boring machine to put the cutting head in a grounded state. Figure 3 As shown, in this state, the laser scanner is used to scan the tunnel boring machine from multiple angles to ensure that the overall picture and details of the tunnel boring machine can be covered. During the scanning process, attention should be paid to the distance and angle between the laser radar and the tunnel boring machine to avoid measurement errors. In order to improve the accuracy of the scanning results, the overlapping scanning method needs to be adopted.

[0044] Step S2: import the point cloud data obtained by multiple scans into a computer, and preprocess the point cloud data group, including denoising, removing outliers, completing missing parts, etc., to improve data quality.

[0045] Step S3: Use the 3D-NDT algorithm to process the first set of point cloud data. Subdivide the space occupied by the point cloud into cubic 3D grids, and calculate the probability density function of the point cloud data appearing in each 3D grid .

[0046] .

[0047] .

[0048] .

[0049] in, and Represents the point cloud space points in The distribution mean and covariance matrix of the 3D grid where , The part represents the scaling factor, is the point in the point cloud space within this 3D grid, and m is the number of points in the point cloud space within this 3D grid.

[0050] Step S4: For the point cloud data of the second scan, the coordinate transformation parameters are calculated according to the coordinates and orientation of the laser radar during the scan, and the coordinate transformation parameters are used to map the point cloud data of the first scan to the space where the point cloud data is located.

[0051] Step S5: Obtain the probability distribution function of the second scanning point cloud using the 3D-NDT algorithm.

[0052] Step S6: The sum of the probability distribution of each point in the second scan point cloud is used as the score S(p) of the coordinate transformation parameter for evaluation.

[0053] .

[0054] in, x is mapped to the coordinates corresponding to the first scan point cloud space according to the transformation parameters, q i and C represent The corresponding mean vector and covariance matrix.

[0055] The 3D transformation vector T(p,x) can be expressed as:

[0056] .

[0057] in, , , , , .

[0058] Step S7: Use the Hessian matrix method combined with the Newton iteration method to find the minimum value of s(p). In order to minimize the function f, the following formula should be used in each iteration:

[0059] .

[0060] Where g is the transposed gradient of f, and the element g in the matrix g i It can be expressed as , the matrix H is the Hessian matrix of f, and its elements can be expressed as .

[0061] Step S8: Return to step S4 and continue the cycle until the convergence requirement is met.

[0062] Step S9: Optimize the registered and spliced ​​3D point cloud model. The optimization process includes removing noise, filling missing parts, removing overlaps, etc.

[0063] Step S10: extracting the point cloud data of the cutting head part and the point cloud data of the part close to the cutting head from the optimized 3D point cloud model.

[0064] Step 1300: When the overlap does not exceed a preset overlap threshold, it is determined that the cutting head has not landed, and a cutting head not landing alarm is triggered.

[0065] The preset overlap threshold is used to determine whether the cutting head has landed. If the overlap does not exceed the preset overlap threshold, it is considered that the cutting head has not landed. For example, the preset overlap threshold can be 75%. If the overlap is less than 75%, the cutting head has not landed alarm is triggered.

[0066] Step 1400: When the overlap exceeds the preset overlap threshold, the overlapping part between the real-time point cloud data of the tunnel boring machine and the pre-acquired point cloud data of the cutting head when the cutting head lands is determined as the real-time point cloud data of the cutting head, and the distance between the cutting head and the ground is calculated based on the real-time point cloud data of the cutting head and the pre-acquired ground point cloud data.

[0067] Due to the working environment of the tunnel boring machine, when the overlap exceeds the preset overlap threshold, it cannot be determined that the cutting head is definitely in the ground state. The tunnel boring machine can be used in working scenarios such as mines and tunnels. In the working scenarios of the tunnel boring machine, it cannot be guaranteed that the ground is always flat. For example, the tunnel boring machine works on the ground with a pit. In this case, when the overlap exceeds the preset overlap threshold, the cutting head is still in the state of not landing.

[0068] The tunnel boring machine includes a cutting head and a cutting head arm, and the cutting head is mounted on the cutting head arm. The cutting head can be driven by adjusting the angle of the cutting head arm. For example, in a flat ground scenario, when the cutting head arm is at a first angle, the cutting head is in a grounded state. If the tunnel boring machine is in a scenario where there is a pit on the ground, and the cutting head arm is still at the first angle, in this case it is judged that the overlap exceeds the preset overlap threshold. However, since there is a pit on the ground, the cutting head is in the pit at this time and is not in contact with the ground, and the cutting head is still in a non-grounded state. Therefore, when it is judged that the overlap exceeds the preset overlap threshold, it cannot be considered that the cutting head is in a grounded state.

[0069] In order to accurately determine whether the cutting head has landed, the distance between the cutting head and the ground is calculated when the overlap exceeds the preset overlap threshold. The ground here refers to a 3D plane fitted by pre-acquired ground point cloud data. The ground where the tunnel boring machine does not exist can be scanned by laser radar to obtain ground point cloud data, and then fitted into a 3D plane based on the ground point cloud data. Usually, the tunnel boring machine needs to be moved to a designated location before it can stop working. In order to ensure safety, the cutting head of the tunnel boring machine needs to be ensured to land after the tunnel boring machine stops working. The pre-acquired ground point cloud data is the ground point cloud data of the designated location where the tunnel boring machine stops working.

[0070] Step 1500: When the distance between the cutting head and the ground exceeds a preset distance threshold, it is determined that the cutting head has not landed, and a cutting head not landing alarm is triggered.

[0071] The preset distance threshold is used to determine whether the cutting head has landed when the overlap exceeds the preset overlap threshold. The preset distance threshold can be set according to the actual parameters of the roadheader, such as the size of the cutting head. The larger the cutting head, the larger the preset distance threshold. For example, the preset distance threshold can be 55 cm.

[0072] When the distance between the cutting head and the ground exceeds the preset distance threshold, it means that the distance between the cutting head and the ground is far, indicating that the cutting head has not landed, and the cutting head has not landed alarm is triggered.

[0073] Step 1600: When the distance between the cutting head and the ground does not exceed a preset distance threshold, determine that the cutting head has landed.

[0074] When the distance between the cutting head and the ground does not exceed the preset distance threshold, it means that the distance between the cutting head and the ground is close, indicating that the cutting head has fallen to the ground.

[0075] The present invention obtains the overlap by comparing the real-time 3D point cloud data of the tunnel boring machine with the pre-acquired point cloud data of the cutting head when the cutting head falls to the ground, and combines the calculated distance between the cutting head and the ground to comprehensively judge whether the cutting head falls to the ground in two ways, thereby ensuring that it is possible to accurately judge whether the cutting head falls to the ground in different environments, thereby improving the accuracy of the judgment result.

[0076] In this embodiment, step 1200 includes steps 1210-1260.

[0077] Step 1210: Determine optimal transformation parameters based on the pre-acquired point cloud data of the part close to the cutting head when the cutting head lands and the real-time 3D point cloud data space, wherein the optimal transformation parameters are used to convert the pre-acquired point cloud data into the real-time 3D point cloud data space.

[0078] The transformation parameters are randomly generated, and the pre-acquired point cloud near the cutting head is converted into the real-time 3D point cloud data space. The random transformation parameters are continuously optimized through the 3D-NDT algorithm and the Hessian matrix until the convergence requirements are met and the optimal transformation parameters are obtained.

[0079] The real-time 3D point cloud data space is a 3D point cloud data space generated when the real-time point cloud data of the tunnel boring machine is acquired through a laser radar.

[0080] Step 1220: convert the pre-acquired point cloud data of the cutting head when the cutting head lands into a real-time 3D point cloud data space using the optimal transformation parameters to obtain converted point cloud data.

[0081] Step 1230: Determine a first circumscribed cylindrical space corresponding to the converted point cloud data.

[0082] The first circumscribed cylinder is the circumscribed cylinder corresponding to the converted point cloud data. The converted point cloud data are all contained in the space of the first circumscribed cylinder.

[0083] Specifically, the step 1230 includes steps 1231-1234.

[0084] Step 1231: Process the converted point cloud data by principal component analysis to obtain the principal component direction of the cutting head.

[0085] The principal component direction of the cutter head represents the extension direction of the cutter head arm.

[0086] Step 1232: Project the converted point cloud data along the principal component direction to obtain the effective length along the principal component direction.

[0087] like Figure 4 As shown, after the converted point cloud data is projected along the principal component direction, the distance between the two endpoints of the projected point cloud data along the principal component direction is the effective length along the principal component direction.

[0088] Step 1233: Determine the maximum normal vector of the converted point cloud data to the principal component direction.

[0089] Calculate the normal vector from each point cloud to the principal component direction in the converted point cloud data. Determine the maximum normal vector based on the normal vector from each point cloud to the principal component direction. The modulus of the normal vector from each point cloud to the principal component direction can be calculated, and the normal vector with the largest modulus is taken as the maximum normal vector.

[0090] Step 1234: Determine the first circumscribed cylindrical space based on the principal component direction, the effective length in the principal component direction, and the maximum normal vector, wherein the radius of the first circumscribed cylindrical space is the length of the maximum normal vector, the axial direction of the first circumscribed cylindrical space is the principal component direction, and the length of the first circumscribed cylindrical space along the principal component direction is the effective length in the principal component direction.

[0091] like Figure 5 As shown, the generated first circumscribed cylindrical space contains all the converted point cloud data.

[0092] Step 1240: Determine the real-time point cloud data contained in the first circumscribed cylindrical space based on the first circumscribed cylindrical space and the real-time point cloud data of the tunnel boring machine.

[0093] After obtaining the first circumscribed cylindrical space, all real-time point cloud data contained in the first circumscribed cylindrical space are determined from the real-time point cloud data of the tunnel boring machine. Specifically, whether each real-time point cloud of the tunnel boring machine is in the first circumscribed cylindrical space can be determined based on the coordinates of each real-time point cloud of the tunnel boring machine.

[0094] Step 1250: Determine a second circumscribed cylindrical space corresponding to the real-time point cloud data in the first circumscribed cylindrical space.

[0095] After obtaining the real-time point cloud data contained in the first circumscribed cylindrical space, the same method as that for generating the first circumscribed cylindrical space may be used to obtain the second circumscribed cylindrical space corresponding to the real-time point cloud data in the first circumscribed cylindrical space.

[0096] Step 1260: Determine the degree of overlap based on the first circumscribed cylindrical space and the second circumscribed cylindrical space.

[0097] Specifically, step 1260 includes: calculating the volume of the intersection of the first circumscribed cylindrical space and the second circumscribed cylindrical space; and taking the ratio between the volume of the intersection and the volume of the first circumscribed cylindrical space as the overlap degree.

[0098] After obtaining the first circumscribed cylindrical space and the second circumscribed cylindrical space, the overlap is determined according to the first circumscribed cylindrical space and the second circumscribed cylindrical space. The overlap is used to indicate whether the cutting head has landed. The lower the overlap, the lower the possibility of the cutting head falling.

[0099] The first circumscribed cylindrical space is the cylindrical space corresponding to the point cloud data of the cutting head when the cutting head is grounded, and the second circumscribed cylindrical space is the cylindrical space corresponding to the real-time point cloud data of the tunnel boring machine in the first circumscribed cylindrical space. If the cutting head is in the grounded state, most of the real-time point cloud data of the tunnel boring machine contained in the first circumscribed cylindrical space should be the point cloud data of the cutting head.

[0100] The greater the overlap, the greater the proportion of the cutting head point cloud data in the real-time point cloud data of the tunnel boring machine contained in the first cylindrical space, and the more the first cylindrical space and the second cylindrical space intersect. The volume of the intersection of the first cylindrical space and the second cylindrical space can be calculated first, and the ratio between the volume of the intersection and the volume of the first cylindrical space is used as the overlap.

[0101] In this embodiment, step 1400 includes steps 1410-1440.

[0102] Step 1410: Determine the ground 3D plane based on the ground point cloud data.

[0103] Step 1420: Determine the real-time point cloud data in the first circumscribed cylindrical space as the real-time point cloud data of the cutting head, calculate the normal vector from each point in the real-time point cloud data of the cutting head to the ground 3D plane, and take the modulus.

[0104] Step 1430: Calculate the expected value of the modulus of the normal vector from each point to the ground 3D plane.

[0105] Step 1440: Determine the expected value as the distance between the cutting head and the ground.

[0106] like Figure 6 As shown, the ground 3D plane is obtained by fitting the pre-acquired ground point cloud data. After obtaining the real-time point cloud data of the tunnel boring machine in the first circumscribed cylindrical space, the normal vector from each real-time point cloud to the ground 3D plane is calculated, and the modulus of the normal vector is calculated. That is, the distance between each real-time point cloud and the ground 3D plane is calculated. Then the expected value of the modulus of the normal vector from each real-time point cloud to the ground 3D plane is calculated, that is, the average value of the distance from all real-time point clouds in the first circumscribed cylindrical space to the ground 3D plane is calculated, and the calculation result is used as the distance between the cutting head and the ground. If the overlap exceeds the threshold, all real-time point clouds in the first circumscribed cylindrical space are used as the real-time point clouds of the cutting head.

[0107] Since the cutting head is large in size, the distance between different positions of the cutting head and the ground is different. If only one point on the cutting head is selected to calculate the distance between the cutting head and the ground, the accuracy of the judgment result of whether the cutting head has landed will be affected.

[0108] Therefore, the present invention uses the expected value of the modulus of the normal vector from each point in the real-time point cloud data in the first circumscribed cylindrical space to the ground 3D plane as the distance between the cutting head and the ground, thereby improving the accuracy of the judgment result.

[0109] In this embodiment, a monitoring device is installed at a position where the cutting head of the tunnel boring machine can be observed on the tunneling working surface. Figure 2 As shown, the monitoring device includes a monitoring camera and a laser radar, the observation direction of the monitoring camera is the same as the observation direction of the laser radar, and after determining that the cutting head has landed, the method also includes steps 2100-2300.

[0110] Step 2100: Project the real-time point cloud data of the cutting head onto the real-time monitoring image captured by the monitoring camera according to the pre-acquired projection parameters, and obtain the first area of ​​the real-time point cloud data of the roadheader on the real-time monitoring image.

[0111] When real-time point cloud data is acquired by the laser radar, a real-time monitoring image is also captured by the monitoring camera. The projection parameters are used to project the point cloud acquired by the laser radar into the monitoring image captured by the monitoring camera. After the real-time point cloud data of the cutting head is projected, the corresponding first area of ​​the real-time point cloud data of the cutting head on the real-time monitoring image is obtained.

[0112] Step 2200: Analyze the real-time monitoring image using a pre-trained target detection model to obtain an analysis result, wherein the analysis result is whether a protective cover of the cutting head exists in the real-time monitoring image.

[0113] The pre-trained target detection model is used to detect the real-time monitoring image to determine whether there is a protective cover of the cutting head in the real-time monitoring image. The protective cover of the cutting head is an object that is covered on the outside of the cutting head. When the roadheader stops working, the protective cover needs to be installed on the outside of the cutting head.

[0114] Step 2300: If the analysis result is that the protective cover of the cutting head does not exist in the real-time monitoring image, an alarm is triggered that the cutting head is not equipped with a protective cover.

[0115] If the pre-trained target detection model determines that there is no protective cover for the cutting head in the real-time monitoring image, the alarm for the cutting head not being equipped with a protective cover is triggered to remind the staff to install the protective cover. Only when it is determined that the cutting head has fallen to the ground will it be determined whether the cutting head is equipped with a protective cover. If it is determined that the cutting head has not fallen to the ground, it will not be determined whether there is a protective cover.

[0116] In this embodiment, the method further includes steps 2400-2600.

[0117] Step 2400: If the analysis result is that the protective cover of the cutting head exists in the real-time monitoring image, determine a second area of ​​the protective cover of the cutting head in the real-time monitoring image.

[0118] If the target detection model determines that there is a protective cover of the cutting head in the real-time monitoring image, then the position of the protective cover of the cutting head in the real-time monitoring image can be obtained according to the recognition result of the target detection. The second area is the area where the protective cover is located in the real-time monitoring image obtained by the target detection model.

[0119] Step 2500: Determine a degree of matching between the first region and the second region, wherein the degree of matching is a ratio of an overlapping area between the first region and the second region to an area of ​​the second region.

[0120] The first area is the corresponding area of ​​the real-time point cloud data of the cutting head on the real-time monitoring image, and the second area is the corresponding area of ​​the protective cover identified by the target detection model in the real-time monitoring image. If the cutting head is equipped with a protective cover, since the protective cover is installed outside the cutting head, the overlap area of ​​the second area and the first area should be larger.

[0121] After obtaining the first area and the second area, the overlap area between the first area and the second area is calculated. Then the ratio of the overlap area to the area of ​​the protective cover area, that is, the ratio of the overlap area to the area of ​​the second area is calculated. The ratio is used as the matching degree of the first area and the second area. When the protective cover is installed, the matching degree is high. If the protective cover is not installed, the matching degree is low.

[0122] Step 2600: When the matching degree is less than a preset matching degree threshold, it is determined that the cutting head is not equipped with a protective cover, and an alarm for the cutting head not being equipped with a protective cover is triggered.

[0123] If the matching degree is less than the preset matching degree threshold, it means that the overlap between the first area and the second area is small, and it can be considered that the protective cover is not installed, thereby triggering the alarm of the cutting head not installing the protective cover, reminding the staff to install the protective cover. The matching degree threshold is an empirical value obtained through actual experiments, for example, the matching degree threshold can be 75%.

[0124] In this embodiment, the step of determining the projection parameters includes steps 3100-3400.

[0125] Step 3100: Collect the 2D image data of the surveillance camera and the 3D point cloud data of the laser radar at the same time, wherein the 2D image coordinate system includes an X-axis and a Y-axis, the 3D point cloud coordinate system includes an XL axis, a YL axis, and a ZL axis, and the 3D point cloud data of the laser radar includes an XL axis component, a YL axis component, and a ZL axis component, and the YL axis is the observation direction of the laser radar.

[0126] At the same time, the monitoring camera and the laser radar are used to collect 2D image data and 3D point cloud data respectively. The data can be collected in a scene with a tunnel boring machine, in which case the collected 2D image data and 3D point cloud data both contain the data of the tunnel boring machine. The data can also be collected in a scene without a tunnel boring machine, in which case the collected 2D image data and 3D point cloud data do not contain the data of the tunnel boring machine.

[0127] 2D image data uses a plane rectangular coordinate system, which includes the X-axis and the Y-axis. The collected image data includes the X-axis coordinate and the Y-axis coordinate of each pixel in the 2D image. 3D point cloud data uses a left-handed coordinate system, including the XL axis, the YL axis, and the ZL axis. The YL axis represents forward, which is the observation direction of the laser radar. The XL axis represents rightward, and the ZL axis represents upward.

[0128] Step 3200: Project the XL axis component and the ZL axis component of the 3D point cloud data of the laser radar along the observation direction to the 2D image of the surveillance camera.

[0129] like Figure 7 As shown in the figure, the collected 3D point cloud data is projected into the 2D image. Since the observation direction of the surveillance camera is the same as that of the laser radar, the 3D point cloud data is projected along the observation direction. Therefore, only the XL axis component and the ZL axis component of the 3D point cloud data are selected during the projection, and the YL axis component is discarded.

[0130] Step 3300: Determine the correspondence between the 3D point cloud data of the laser radar and the 2D image data of the monitoring camera based on the projection result, and determine n groups of corresponding coordinates through coordinate sampling, wherein the i-th group of corresponding coordinates is expressed as [(Xi, Yi), (XLi, ZLi)], Xi and Yi respectively represent the X-axis coordinate and the Y-axis coordinate of the 2D image data of the monitoring camera, and XLi and ZLi respectively represent the XL-axis coordinate and the ZL-axis coordinate of the 3D point cloud data of the laser radar.

[0131] By projecting the 3D point cloud onto the 2D image, the correspondence between the 3D point cloud and the pixels in the 2D image is determined. The coordinates of n corresponding 3D point clouds and the coordinates of the pixels in the 2D image are collected by coordinate sampling. Since the YL axis coordinates of the 3D point cloud are discarded during projection, the coordinates of the 3D point cloud only include the XL axis coordinates and the ZL axis coordinates in the n corresponding coordinates determined by coordinate sampling.

[0132] Step 3400: Determine the projection parameters according to the n groups of corresponding coordinates.

[0133] Specifically, the step 3400 includes steps 3410-3440.

[0134] Step 3410: Determine the minimum value Xmin and maximum value Xmax of the X-axis coordinate, the minimum value Ymin and maximum value Ymax of the Y-axis coordinate, the minimum value XLmin and maximum value XLmax of the XL-axis coordinate, and the minimum value ZLmin and maximum value Zlmax of the ZL-axis coordinate from the n groups of corresponding coordinates.

[0135] Step 3420: Calculate the scaling ratio scaleX of the 3D point cloud data to the 2D image data on the X-axis and the scaling ratio scaleY of the Y-axis.

[0136] .

[0137] .

[0138] Step 3430: Multiply the XL axis component and the ZL axis component of the 3D point cloud data in the n groups of corresponding coordinates by the X axis scaling scaleX and the Y axis scaling scaleY respectively to obtain the scaled point cloud coordinates XLs and ZLs.

[0139] Step 3440: Calculate the differences between the X-axis coordinates and the Y-axis coordinates of the 2D image data in the n groups of corresponding coordinates and the corresponding scaled point cloud coordinates XLs and ZLs, and calculate the expected value dX of the X-axis difference and the expected value dY of the Y-axis difference.

[0140] .

[0141] .

[0142] Among them, XLsi and ZLsi represent the scaled coordinates of the i-th group of point clouds.

[0143] The conversion formula from 3D point cloud data coordinates (XL, ZL) to 2D image data coordinates (X, Y) is:

[0144] .

[0145] .

[0146] The projection parameters include a scaling factor scaleX of the X axis, a scaling factor scaleY of the Y axis, and expected values ​​dX and dY of the difference values.

[0147] In this embodiment, the method also includes a target detection model training step, specifically including steps 4100-4600.

[0148] Step 4100: Collect a preset number of images of the cutting head protective cover.

[0149] The image of the cutting head protective cover can be collected by the monitoring camera. The preset number can be set by yourself, for example, the preset number can be 10000.

[0150] Step 4200: Mark the protective cover area for each picture of the cutting head protective cover.

[0151] After collecting the images of the cutting head protective cover, the protective cover area in each image of the cutting head protective cover is manually labeled for subsequent model training.

[0152] Step 4300: Split the labeled images of the cutting head protective cover into a training set, a validation set, and a test set according to a preset ratio.

[0153] The labeled images of the cutting head protective cover are split into a training set, a validation set, and a test set. The training set and the validation set are used for model training, and the test set is used to test the trained model. The images are split according to the preset ratio. For example, if the preset ratio is 8:1:1 and the number of images of the cutting head protective cover collected is 10,000, then the training set contains 8,000 images, the validation set contains 1,000 images, and the test set contains 1,000 images.

[0154] Step 4400: Establish a convolutional neural network model, and train the convolutional neural network model using the training set and the validation set.

[0155] The convolutional neural network model in the present invention selects the VGG16 structure, the convolution layer and the fully connected layer use the ReLU activation function, the pooling layer uses the maximum pooling, and the output layer uses the softmax function. The training set and the validation set are combined into a cross-validation data set, and the mini-batch method is used for training. The overall training is repeated for 5000 rounds before testing.

[0156] Step 4500: Test the trained convolutional neural network model using the test set with the F1 score as an indicator.

[0157] Step 4600: When the F1 score of the trained convolutional neural network model meets a preset condition, the trained convolutional neural network model is determined as the target detection model.

[0158] During the test, the F1 score is used as the test indicator. If the F1 score of the trained convolutional neural network model meets the requirements, it will be output as a target detection model to determine whether there is a protective cover in the picture taken by the surveillance camera.

[0159] The beneficial effects of the present invention are:

[0160] (1) Using a 2D-3D combination approach, an innovative method for identifying the scene of the cutting head of a tunnel boring machine falling to the ground and installing a protective cover is proposed.

[0161] (2) The 3D point cloud data is analyzed using the 3D-NDT algorithm, the Hessian matrix method, and the principal component analysis method. By analyzing the pre-acquired point cloud and the real-time point cloud, it is determined whether the cutting head of the tunnel boring machine has landed, which is a more accurate result than 2D target detection.

[0162] (3) The 3D point cloud normal vector calculation is used to obtain the actual distance between the cutting head of the tunnel boring machine and the ground, and then determine whether the cutting head of the tunnel boring machine has landed, which makes up for the problem of insensitivity to distance in 2D target detection.

[0163] (4) A projection algorithm from 3D point cloud to 2D image was used to determine the coordinate range of the cutting head of the tunnel boring machine in the 2D image, solving the problem that 2D target detection cannot be effectively identified when the cutting head is blocked by a protective cover.

[0164] (5) A convolutional neural network is used to establish a detection model for the cutting head protective cover of a tunnel boring machine. The existence and location of the protective cover are determined through 2D target detection, which makes up for the problem that the point cloud of the protective cover is too sparse in 3D scanning to be effectively identified.

[0165] (6) The present invention, through the above method, greatly improves the accuracy of identifying whether the cutting head has fallen to the ground and whether the protective cover is installed when the tunnel boring machine stops working or is under maintenance or when the shift is changed.

[0166] This embodiment introduces a cutting head falling protection detection device, including a processor and a memory, wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes a cutting head falling protection detection method as described in any embodiment of the present invention.

[0167] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

[0168] It should be understood that the size of the sequence number of each step in the content of the invention and the embodiments of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. For the purpose of example and description, the foregoing description of the implementation of the present disclosure has been given. The foregoing description is not exhaustive and is not intended to limit the present disclosure to the exact form disclosed. Various deformations and modifications may exist according to the above teachings, or various deformations and modifications may be obtained from the practice of the present disclosure. These embodiments are selected and described to illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can use the present disclosure in various embodiments and various modifications suitable for the specific purpose conceived.

Claims

1. A method for detecting the grounding protection of a cutting head of a tunnel boring machine, characterized in that: The method comprises: Obtain real-time point cloud data of the roadheader; Determining the degree of overlap between the real-time point cloud data of the roadheader and the pre-acquired point cloud data of the cutting head when the cutting head lands; When the overlap does not exceed a preset overlap threshold, it is determined that the cutting head has not landed, and a cutting head not landing alarm is triggered; When the overlap exceeds the preset overlap threshold, the overlapping portion between the real-time point cloud data of the roadheader and the pre-acquired point cloud data of the cutting head when the cutting head lands is determined as the real-time point cloud data of the cutting head, and the distance between the cutting head and the ground is calculated based on the real-time point cloud data of the cutting head and the pre-acquired ground point cloud data; When the distance between the cutting head and the ground exceeds a preset distance threshold, it is determined that the cutting head has not landed, and a cutting head not landing alarm is triggered; When the distance between the cutting head and the ground does not exceed a preset distance threshold, it is determined that the cutting head has landed.

2. The method according to claim 1, characterized in that Determining the degree of overlap between the real-time point cloud data of the roadheader and the pre-acquired point cloud data of the cutting head when the cutting head lands comprises: Determine optimal transformation parameters based on pre-acquired point cloud data of a portion close to the cutting head when the cutting head lands and the real-time 3D point cloud data space, wherein the optimal transformation parameters are used to convert the pre-acquired point cloud data into the real-time 3D point cloud data space; The point cloud data of the cutting head when the cutting head lands, which is acquired in advance, is converted into a real-time 3D point cloud data space by using the optimal transformation parameters to obtain the converted point cloud data; Determine a first circumscribed cylindrical space corresponding to the converted point cloud data; Determining the real-time point cloud data contained in the first circumscribed cylindrical space according to the first circumscribed cylindrical space and the real-time point cloud data of the tunnel boring machine; Determine a second circumscribed cylindrical space corresponding to the real-time point cloud data in the first circumscribed cylindrical space; The degree of overlap is determined according to the first circumscribed cylindrical space and the second circumscribed cylindrical space.

3. The method according to claim 2, characterized in that The determining of the first circumscribed cylindrical space corresponding to the converted point cloud data includes: Processing the converted point cloud data by principal component analysis to obtain the principal component direction of the cutting head; Projecting the converted point cloud data along the principal component direction to obtain an effective length in the principal component direction; Determine the maximum normal vector of the converted point cloud data to the direction of the principal component; The first circumscribed cylindrical space is determined according to the principal component direction, the effective length in the principal component direction and the maximum normal vector, wherein the radius of the first circumscribed cylindrical space is the length of the maximum normal vector, the axial direction of the first circumscribed cylindrical space is the principal component direction, and the length of the first circumscribed cylindrical space along the principal component direction is the effective length in the principal component direction.

4. The method according to claim 2, characterized in that: The determining the overlap degree according to the first circumscribed cylindrical space and the second circumscribed cylindrical space includes: Calculating the volume of the intersection of the first circumscribed cylindrical space and the second circumscribed cylindrical space; The ratio between the volume of the intersection portion and the volume of the first circumscribed cylindrical space is taken as the overlap degree.

5. The method according to claim 2, characterized in that: When the overlap exceeds a preset overlap threshold, the overlapping portion between the real-time point cloud data of the roadheader and the pre-acquired point cloud data of the cutting head when the cutting head lands is determined as the real-time point cloud data of the cutting head, and the distance between the cutting head and the ground is calculated according to the real-time point cloud data of the cutting head and the pre-acquired ground point cloud data, including: Determine a ground 3D plane according to the ground point cloud data; Determine the real-time point cloud data in the first circumscribed cylindrical space as the real-time point cloud data of the cutting head, calculate the normal vector from each point in the real-time point cloud data of the cutting head to the ground 3D plane, and take the modulus; Calculate the expected value of the modulus of the normal vector from each point to the ground 3D plane; The desired value is determined as the distance between the cutting head and the ground.

6. The method according to claim 1, characterized in that A monitoring device is installed at a position where the cutting head of the tunnel boring machine can be observed on the tunneling working face, the monitoring device includes a monitoring camera and a laser radar, the observation direction of the monitoring camera is the same as the observation direction of the laser radar, and after determining that the cutting head has landed, the method further includes: Projecting the real-time point cloud data of the cutting head onto the real-time monitoring image captured by the monitoring camera according to the pre-acquired projection parameters, to obtain the first area of ​​the real-time point cloud data of the roadheader on the real-time monitoring image; Analyzing the real-time monitoring image by using a pre-trained target detection model to obtain an analysis result, wherein the analysis result is whether a protective cover of the cutting head exists in the real-time monitoring image; If the analysis result is that the protective cover of the cutting head does not exist in the real-time monitoring image, an alarm that the cutting head is not equipped with a protective cover is triggered.

7. The method according to claim 6, characterized in that The method further comprises: If the analysis result is that the protective cover of the cutting head exists in the real-time monitoring image, determining a second area of ​​the protective cover of the cutting head in the real-time monitoring image; Determining a degree of matching between the first region and the second region, wherein the degree of matching is a ratio of an overlapping area between the first region and the second region to an area of ​​the second region; When the matching degree is less than a preset matching degree threshold, it is determined that the cutting head is not equipped with a protective cover, and a cutting head not equipped with a protective cover alarm is triggered.

8. The method according to claim 6, characterized in that The projection parameters are determined in the following way: Collecting the 2D image data of the surveillance camera and the 3D point cloud data of the laser radar at the same time, wherein the 2D image coordinate system includes an X-axis and a Y-axis, the 3D point cloud coordinate system includes an XL axis, a YL axis, and a ZL axis, and the 3D point cloud data of the laser radar includes an XL axis component, a YL axis component, and a ZL axis component, and the YL axis is the observation direction of the laser radar; Projecting the XL axis component and the ZL axis component of the 3D point cloud data of the laser radar to the 2D image of the surveillance camera along the observation direction; Determine the correspondence between the 3D point cloud data of the laser radar and the 2D image data of the surveillance camera according to the projection result, and determine n groups of corresponding coordinates by coordinate sampling, wherein the i-th group of corresponding coordinates is expressed as [(Xi, Yi), (XLi, ZLi)], Xi and Yi respectively represent the X-axis coordinate and the Y-axis coordinate of the 2D image data of the surveillance camera, and XLi and ZLi respectively represent the XL-axis coordinate and the ZL-axis coordinate of the 3D point cloud data of the laser radar; The projection parameters are determined according to the n groups of corresponding coordinates.

9. The method according to claim 8, characterized in that The determining the projection parameters according to the n groups of corresponding coordinates includes: Determine the minimum value Xmin and the maximum value Xmax of the X-axis coordinate, the minimum value Ymin and the maximum value Ymax of the Y-axis coordinate, the minimum value XLmin and the maximum value XLmax of the XL-axis coordinate, and the minimum value ZLmin and the maximum value ZLmax of the ZL-axis coordinate from the n groups of corresponding coordinates; Calculate the scaling ratio scaleX of the 3D point cloud data to the 2D image data on the X axis and the scaling ratio scaleY of the Y axis, wherein: ， ， Multiplying the XL axis component and the ZL axis component of the 3D point cloud data in the n groups of corresponding coordinates by the X axis scaling scale scaleX and the Y axis scaling scale scaleY respectively, to obtain the scaled coordinates XLs and ZLs of the point cloud; Calculate the differences between the X-axis coordinates and the Y-axis coordinates of the 2D image data in the n sets of corresponding coordinates and the corresponding scaled point cloud coordinates XLs and ZLs, respectively, and calculate the expected values ​​dX and dY of the differences, ， ， Among them, XLsi and ZLsi represent the coordinates of the i-th group of point clouds after scaling; The conversion formula from 3D point cloud data coordinates (XL, ZL) to 2D image data coordinates (X, Y) is: ， ， The projection parameters include a scaling factor scaleX of the X axis, a scaling factor scaleY of the Y axis, and expected values ​​dX and dY of the difference values.

10. The method according to claim 6, characterized in that The method further comprises: Collecting a preset number of images of the cutting head protective cover; Marking the protective cover area on each picture of the cutting head protective cover; The labeled images of the cutting head protective cover are split into a training set, a validation set, and a test set according to a preset ratio; Establishing a convolutional neural network model, and training the convolutional neural network model using the training set and the validation set; Testing the trained convolutional neural network model using the test set using the F1 score as an indicator; When the F1 score of the trained convolutional neural network model meets a preset condition, the trained convolutional neural network model is determined as the target detection model.

11. A ground protection detection device for a cutting head of a tunnel boring machine, characterized in that: The invention comprises a processor and a memory, wherein the memory stores executable codes, and when the executable codes are executed by the processor, the processor executes the method according to any one of claims 1 to 10.

Images