Pile driver working condition identification method, device and system

Abstract

The invention discloses a pile driver working condition recognition method, device and system. The pile driver working condition recognition method comprises the steps that two parallax pictures of a pile body are obtained; feature operator extraction is carried out on the two obtained parallax images, and matching points in the two parallax images are recognized based on feature operator extraction results; calculating three-dimensional coordinates of the matching points based on the recognized two-dimensional coordinates of the matching points and an internal reference matrix of the camera, and further generating a three-dimensional source point cloud of the pile body; and the three-dimensional source point cloud is registered with a preset three-dimensional target point cloud, and the pile lowering angle of the pile body is obtained. According to the invention, the angle measurement of the pile driving posture is indirectly completed, and the measurement precision is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering machinery, and in particular relates to a method, device and system for identifying working conditions of a pile driver. Background Art

[0002] The vertical installation of piles has always been an important means of measuring construction quality. The existing methods of pile measurement are divided into manual and non-manual methods. In the manual method, the on-site commander reminds the operator of the pile deflection angle through visual measurement. The operator makes slight adjustments based on the feedback from the on-site commander. This process relies entirely on personal experience and there is no precise measurement. The non-manual method mostly uses indirect measurement. For example, BuiltRobotics uses additional piling tools to directly control the angle of the pile, ensuring the verticality of the piling angle from a structural perspective.

[0003] In the Chinese utility model patent application with publication number CN219416180U, an automatic detection device for the verticality of steel sheet pile driving is disclosed, which belongs to the field of civil engineering technology. It aims at the limitations of the use of the detection device and the problem that the hanging hammer is easily damaged by collision, and includes a stabilizing frame, fixed rods are fixed on the left and right sides of the stabilizing frame, stabilizing rods are fixed on the lower sides of the two fixed rods, and the outer surfaces of the two stabilizing rods are rotatably connected to support plates, one end of the two support plates is fixed with a first cone, the upper ends of the two fixed rods are fixed with a connecting plate, the lower ends of the two fixed rods are fixed with a second cone, the middle part of the upper end of the stabilizing frame is fixed with a limit frame, the middle part of the limit frame is fixed with a hanging line, the lower end of the hanging line is fixed with a hanging hammer, and the right side of the stabilizing frame is fixed with a stabilizing column. This solution requires the addition of additional mechanisms to ensure that the steel sheet piles can be driven vertically into the ground, which adds additional costs.

[0004] In the Chinese utility model patent application with publication number CN215766919U, a device for measuring the lateral deformation of steel sheet piles in a foundation pit is provided, which belongs to the technical field of deformation test of foundation or foundation components using optical methods. It includes a fixing system, a laser ranging system and an information processing system. The laser ranging system and the information processing system are located on the fixing system. The laser ranging system includes a laser ranging receiving mechanism and a laser ranging transmitting mechanism. The information processing system includes a central processing unit and a data interface. The laser ranging transmitting mechanism and the laser ranging receiving mechanism are respectively connected to the central processing unit signal, and the angle of the laser ranging transmitting mechanism is controlled by the central processing unit. The data of the laser ranging receiving mechanism is transmitted to the central processing unit and output through the data interface. This scheme only measures the lateral deformation of the steel sheet pile by adding additional optical measuring equipment, and cannot measure the piling angle of the steel sheet pile. Summary of the invention

[0005] In view of the above problems, the present invention proposes a method, device and system for identifying the working condition of a pile driver, which extracts the feature operators of two parallax pictures of the pile body, identifies the matching points in the two parallax pictures, calculates the three-dimensional coordinates of the matching points based on the two-dimensional coordinates of the identified matching points and the intrinsic parameter matrix of the camera, and then generates a three-dimensional source point cloud of the pile body, and matches the generated three-dimensional source point cloud according to the preset target cloud point cloud, thereby indirectly completing the angle measurement of the pile driving posture.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions: In a first aspect, the present invention provides a method for identifying a working condition of a pile driver, comprising: Get two parallax images of the pile; Perform feature operator extraction on the two obtained disparity pictures respectively, and identify matching points in the two disparity pictures based on the feature operator extraction results; Based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, the three-dimensional coordinates of the matching points are calculated, and then the three-dimensional source point cloud of the pile body is generated; The three-dimensional source point cloud is registered with a preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

[0007] In combination with the first aspect, optionally, the two disparity pictures are respectively recorded as a first image and a second image; and the three-dimensional coordinates of the matching point are obtained by the following method: Calculate a projection matrix based on the two-dimensional coordinates of the identified matching points, the intrinsic parameter matrix of the camera and a preset first calculation formula; Based on the two-dimensional coordinates of the matching points, the intrinsic parameter matrix of the camera and the projection matrix, the three-dimensional coordinates of the matching points in the first image and the second image are calculated, and then the three-dimensional source point cloud of the pile body is generated.

[0008] In combination with the first aspect, optionally, the first calculation formula is expressed as: s1a*(x1a, y1a) / (K1*T1)- s1b*(x1b, y1b) / (K2*T2) = 0; s2a* (x2a, y2a) / (K1*T1) - s2b* (x2b, y2b) / (K2*T2) = 0; s3a*(x3a, y3a) / (K1*T1)- s3b* (x3b, y3b) / (K2*T2) = 0; s4a* (x4a, y4a) / (K1*T1) - s4b* (x4b, y4b) / (K2*T2) = 0; s5a* (x5a, y5a) / (K1*T1) - s5b* (x5b, y5b) / (K2*T2) = 0; s6a* (x6a, y6a) / (K1*T1) - s6b* (x6b, y6b) / (K2*T2) = 0; s7a* (x7a, y7a) / (K1*T1) - s7b* (x7b, y7b) / (K2*T2) = 0; s8a* (x8a, y8a) / (K1*T1) - s8b* (x8b, y8b) / (K2*T2) = 0; Wherein, s1a, s2a, s3a, s4a, s5a, s6a, s7a, s8a, s1b, s2b, s3b, s4b, s5b, s6b, s7b, and s8b represent the depths of the eight matching points in the first image and the second image respectively, K1 and K2 represent the intrinsic parameter matrices of the cameras corresponding to the first image and the second image respectively, T1 and T2 represent the projection matrices corresponding to the first image and the second image respectively, (x1a, y1a), (x2a, y2a), (x3a, y3a), (x4a, y4a), (x5a, y5a), (x6a, y6a), (x7a, y7a), and (x8a, y8a) represent the coordinates of the eight matching points in the first image respectively; (x1b, y1 b), (x2 b, y2 b), (x3 b, y3 b), (x4 b, y4 b), (x5 (x5 b, y5 b), (x6 b, y6 b), (x7 b, y7 b), and (x8 b, y8 b) respectively represent the two-dimensional coordinates of the eight matching points in the second image.

[0009] In combination with the first aspect, optionally, the three-dimensional source point cloud is generated by a BundleAdjustment algorithm.

[0010] In combination with the first aspect, optionally, before the step of respectively performing feature operator extraction on the two acquired disparity pictures, the method further includes: Grayscale the two obtained disparity pictures to generate a grayscale image; The grayscale value of each pixel in the grayscale image is compared with a preset grayscale threshold, and the grayscale value of each pixel is set to 0 or 255 according to the comparison result.

[0011] In combination with the first aspect, optionally, feature operator extraction is performed on the two obtained disparity pictures respectively, and matching points in the two disparity pictures are identified based on the feature operator extraction results, including: An improved FAST algorithm is used to extract FAST corner points in an image, wherein the FAST corner points have rotation invariance and scale invariance. Use the BRIEF algorithm to describe FAST corner points and generate feature point descriptors; The Hamming distance is used to match the feature point descriptors in the two images and identify the matching points in the two disparity images.

[0012] In combination with the first aspect, optionally, registering the three-dimensional source point cloud with a preset three-dimensional target point cloud to obtain a pile lowering angle of the pile body includes: Extract key points from 3D source point cloud and 3D target point cloud; Based on the extracted key points, the corresponding point pairs between the 3D source point cloud and the 3D target point cloud are found through feature matching; Eliminate erroneous corresponding points and complete the screening of corresponding point pairs; Based on the screened corresponding point pairs, the optimal transformation matrix is ​​estimated; The estimated optimal transformation matrix is ​​applied to the 3D source point cloud to complete the registration and obtain the installation angle of the pile.

[0013] In combination with the first aspect, optionally, the method for estimating the optimal transformation matrix includes: Create an ICP object; Set the 3D source point cloud and 3D target point cloud to the ICP object through the setInputSource and setInputTarget methods; Set the maximum number of iterations, the maximum corresponding distance, and the convergence threshold of the transformation matrix; Call the align method to perform the alignment and store the alignment result in a new point cloud object; Check whether the registration has converged through the hasConverged method, use getFitnessScore to obtain the fitness score of the registration, and use getFinalTransformation to obtain the final transformation matrix.

[0014] In a second aspect, the present invention provides a pile driver working condition identification device, comprising: An image acquisition module is used to acquire two parallax images of the pile body; A matching point recognition module is used to extract feature operators from the two obtained disparity pictures respectively, and identify matching points in the two disparity pictures based on the feature operator extraction results; A three-dimensional source point cloud generation module is used to calculate the three-dimensional coordinates of the matching points based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, and then generate a three-dimensional source point cloud of the pile body; The registration module is used to register the three-dimensional source point cloud with a preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

[0015] In a third aspect, the present invention provides a pile driver working condition identification system, including a storage medium and a processor; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the method according to any one of the first aspects.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method, device and system for identifying the working condition of a pile driver. The method extracts feature operators of two parallax pictures of a pile body, identifies matching points in the two parallax pictures, calculates the three-dimensional coordinates of the matching points based on the two-dimensional coordinates of the identified matching points and the intrinsic parameter matrix of the camera, and then generates a three-dimensional source point cloud of the pile body. The generated three-dimensional source point cloud is matched according to a preset target cloud point cloud, thereby indirectly completing the angle measurement of the pile driving posture, which can improve the efficiency of pile driving posture recognition and reduce costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor, among which: Figure 1 The present invention is a flowchart of a method for identifying working conditions of a pile driver according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention. Example 1

[0020] In an embodiment of the present invention, a method for identifying working conditions of a pile driver is provided. Figure 1 As shown, the following steps are included: (1) Obtain two parallax images of the pile; (2) Perform feature operator extraction on the two obtained disparity images respectively, and identify the matching points in the two disparity images based on the feature operator extraction results; (3) Based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, the three-dimensional coordinates of the matching points are calculated, and then the three-dimensional source point cloud of the pile is generated; (4) The three-dimensional source point cloud is aligned with a preset three-dimensional target point cloud to obtain the pile installation angle of the pile body.

[0021] Based on the pile driver working condition recognition method in the embodiment of the present invention, by extracting the feature operator of the two parallax pictures of the pile body, the matching points in the two parallax pictures are identified, and the three-dimensional coordinates of the matching points are calculated based on the two-dimensional coordinates of the identified matching points and the intrinsic parameter matrix of the camera, and then the three-dimensional source point cloud of the pile body is generated, and the generated three-dimensional source point cloud is matched according to the preset target cloud point cloud, thereby indirectly completing the angle measurement of the pile driving posture, which can effectively improve the efficiency of pile driving posture recognition and reduce costs.

[0022] In a specific implementation of the embodiment of the present invention, the two disparity pictures are respectively the first image and the second image; the three-dimensional coordinates of the matching point are obtained by the following method: Calculate a projection matrix based on the two-dimensional coordinates of the identified matching points, the intrinsic parameter matrix of the camera and a preset first calculation formula; Based on the two-dimensional coordinates of the matching points, the camera's intrinsic parameter matrix, and the projection matrix, the three-dimensional coordinates of the matching points in the first image and the second image are calculated (this calculation process is prior art and will not be described in detail in the present invention), thereby generating a three-dimensional source point cloud of the pile body.

[0023] In a specific implementation of the embodiment of the present invention, the expression of the first calculation formula is: s1a*(x1a, y1a) / (K1*T1)- s1b*(x1b, y1b) / (K2*T2) = 0; s2a* (x2a, y2a) / (K1*T1) - s2b* (x2b, y2b) / (K2*T2) = 0; s3a*(x3a, y3a) / (K1*T1)- s3b* (x3b, y3b) / (K2*T2) = 0; s4a* (x4a, y4a) / (K1*T1) - s4b* (x4b, y4b) / (K2*T2) = 0; s5a* (x5a, y5a) / (K1*T1) - s5b* (x5b, y5b) / (K2*T2) = 0; s6a* (x6a, y6a) / (K1*T1) - s6b* (x6b, y6b) / (K2*T2) = 0; s7a* (x7a, y7a) / (K1*T1) - s7b* (x7b, y7b) / (K2*T2) = 0; s8a* (x8a, y8a) / (K1*T1) - s8b* (x8b, y8b) / (K2*T2) = 0; Wherein, s1a, s2a, s3a, s4a, s5a, s6a, s7a, s8a, s1b, s2b, s3b, s4b, s5b, s6b, s7b, s8b represent the depth of each matching point in the first image and the second image respectively, K1 and K2 represent the intrinsic parameter matrices of the cameras corresponding to the first image and the second image respectively, T1 and T2 represent the projection matrices corresponding to the first image and the second image respectively, (x1a, y1a), (x2a, y2a), (x3a, y3a), (x4a, y4a), (x5a, y5a), (x6a, y6a), (x7a, y7a), (x8a, y8a) represent the coordinates of the 8 matching points in the first image respectively; (x1b, y1 b), (x2 b, y2 b), (x3 b, y3 b), (x4 b, y4 b), (x5 b, y5 b), (x6 b, y6 b), (x7 b, y7 b), and (x8 b, y8 b) respectively represent the coordinates of the eight matching points in the second image.

[0024] In order to highlight the feature points and use them as the material for feature extraction, the embodiment of the present invention proposes to process the acquired disparity images by a binarization method. Specifically, in a specific implementation of the embodiment of the present invention, before the step of extracting the feature operators from the two acquired disparity images, the following steps are further included: Grayscale the two obtained disparity pictures to generate a grayscale image; The grayscale value of each pixel in the grayscale image is compared with a preset grayscale threshold, and the grayscale value of each pixel is set to 0 or 255 according to the comparison result. In the specific implementation process, the preset grayscale threshold can be set according to actual needs.

[0025] In a specific implementation of the embodiment of the present invention, feature operators are respectively extracted from the two obtained disparity pictures, and matching points in the two disparity pictures are identified based on the feature operator extraction results, including: An improved FAST algorithm is used to extract FAST corner points in an image, wherein the FAST corner points have rotation invariance and scale invariance. Use the BRIEF algorithm to describe FAST corner points and generate feature point descriptors; The Hamming distance is used to match the feature point descriptors in the two images and identify the matching points in the two disparity images. Specifically, when the Hamming distance between the two feature point descriptors is less than the set threshold, they can be considered to be matched.

[0026] In a specific implementation of the embodiment of the present invention, the registering the three-dimensional source point cloud with the preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body includes: Extract key points from 3D source point cloud and 3D target point cloud; Based on the extracted key points, the corresponding point pairs between the 3D source point cloud and the 3D target point cloud are found through feature matching; Eliminate erroneous corresponding points and complete the screening of corresponding point pairs; Based on the screened corresponding point pairs, the optimal transformation matrix is ​​estimated; The estimated optimal transformation matrix is ​​applied to the source point cloud to complete the registration and obtain the installation angle of the pile.

[0027] In a specific implementation of the embodiment of the present invention, the method for estimating the optimal transformation matrix includes: Create an ICP object; Use the setInputSource and setInputTarget methods to set the 3D source point cloud and 3D target point cloud to the ICP object; Set the maximum number of iterations, the maximum corresponding distance, and the convergence threshold of the transformation matrix; Call the align method to perform the alignment and store the alignment result in a new point cloud object; Check whether the registration has converged through the hasConverged method, use getFitnessScore to obtain the fitness score of the registration, and use getFinalTransformation to obtain the final transformation matrix.

[0028] The pile driver working condition identification method in the embodiment of the present invention is described in detail below in conjunction with a specific implementation.

[0029] Step (1) uses two high-definition cameras installed above the cab to collect images of the pile in front and obtain two parallax pictures. The two parallax pictures are used for posture estimation, which includes the following steps: 1) Lighting: First, the target pile (such as steel sheet pile) needs to be properly illuminated because image acquisition relies on light reflected from the surface of the object. The lighting can be ambient light or a specially designed light source such as LED light, halogen light, etc. to ensure the quality and contrast of the image.

[0030] 2) High-definition camera (i.e. imaging system): usually consists of a lens and an image sensor. The lens focuses the image of the target pile onto the image sensor. The image sensor (such as CCD or CMOS) is responsible for capturing the focused light image and converting it into an electrical signal.

[0031] 3) Signal conversion: The electrical signal generated by the image sensor is an analog signal, which needs to be converted into a digital signal by an analog-to-digital converter (ADC). The digital signal can be processed, stored or transmitted by a computer.

[0032] 4) Image digitization: The digitization process involves converting analog image signals into discrete digital values. This is usually done line by line or pixel by pixel, with each pixel being assigned a numerical value representing its brightness or color information.

[0033] 5) Image data formatting: The digitized image data needs to be organized in a certain format, such as bitmap (BMP), Joint Photographic Experts Group (JPEG) or Portable Network Graphics (PNG), for easy storage and transmission.

[0034] 6) Image processing: After image acquisition, some preprocessing steps are usually required, such as denoising, contrast enhancement, color correction, etc., to improve the image quality and make it more suitable for subsequent analysis and recognition tasks.

[0035] 7) Data transfer and storage: The processed image data can be transferred to a computer or other processing device through various interfaces (such as USB, GigE, CameraLink, etc.). In the computer, the image data can be further analyzed or stored for later use.

[0036] Step (2) Preprocess the two disparity images: In the embodiment of the present invention, the two parallax images collected are processed by a binarization method to highlight the feature points, which are used as the material for feature extraction. The following steps are included: 1) Grayscale: First, if it is a color image, it needs to be converted to a grayscale image. Each pixel value in a grayscale image represents the brightness information of the pixel, usually ranging from 0 (black) to 255 (white).

[0037] 2) Selecting a threshold: The core of binarization is to select a threshold to distinguish the foreground and background in the image. Pixels with values ​​above the threshold will be classified into one category (usually white, representing the foreground), while those below the threshold will be classified into another category (usually black, representing the background).

[0038] 3) Applying threshold (i.e. binarization): Applying the selected threshold to each pixel in the grayscale image. This process involves comparing the grayscale value of each pixel with the threshold and then setting the pixel value to 0 or 255 (in an 8-bit image) based on the comparison result.

[0039] 4) Post-processing: After binarization, some post-processing steps may be required, such as removing noise, filling small holes, removing small objects, etc., to improve the quality of the image.

[0040] Step (3) extracts feature operators from the two disparity images after image preprocessing: Feature operator extraction is a key step in image processing and computer vision. It is used to extract features from images that are helpful for subsequent analysis and recognition. These features are usually some significant and distinguishable information in the image, which can be points, lines, edges, corners, etc.

[0041] The embodiment of the present invention adopts the ORB (Oriented FAST and Rotated BRIEF) feature extraction algorithm, which is an efficient algorithm that combines FAST corner detection and BRIEF descriptor, and has significant advantages in computing speed and performance. The steps include: 1) FAST corner extraction: The ORB algorithm uses the improved FAST (Features from Accelerated Segment Test) algorithm to detect corners in an image.

[0042] Corner points are determined by selecting a pixel in the image and comparing the brightness difference between it and the pixels in its surrounding neighborhood.

[0043] If a pixel has a significant brightness difference with enough pixels in its surrounding neighborhood, the pixel is considered to be a FAST corner point.

[0044] 2) Rotational invariance: The ORB algorithm defines a direction for each FAST corner point through the grayscale centroid method to achieve rotation invariance of feature points.

[0045] The moment of the image is calculated in the neighborhood of the FAST corner point to obtain the position of the center of mass, and then a direction is determined based on the position of the center of mass and the FAST corner point.

[0046] 3) Scale invariance: By constructing an image pyramid, corner points are repeatedly detected on images of different scales to achieve scale invariance.

[0047] 4) BRIEF descriptor: The ORB algorithm uses the BRIEF algorithm to describe feature points and generates a binary string as the feature point descriptor of the feature point. Point pairs are randomly selected in the neighborhood of the feature point, the grayscale values ​​of these point pairs are compared, and the comparison results are combined into the feature point descriptor.

[0048] Improved BRIEF (rBRIEF): The ORB algorithm improves the BRIEF descriptor to make it rotation invariant. By selecting point pairs in the rotated coordinate system, the descriptor is ensured to remain consistent when the image is rotated.

[0049] 5) Feature point matching: The Hamming distance is used to match the feature point descriptors in the two images. When the Hamming distance between the two feature point descriptors is less than the set distance threshold, they can be considered to match.

[0050] Step (4) calculates the three-dimensional coordinates of the matching points based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, and then generates a three-dimensional source point cloud of the pile body to complete the three-dimensional geometric structure restoration of the corresponding pile body. It includes the following steps: 1. Using the parallax and the camera's internal parameters, the 3D coordinates of the Pw point in the working scene can be calculated. The formula is usually as follows: s1*p1=K1*T1*Pw; s2*p2=K2*T2*Pw; Among them, p1 and p2 are the coordinates of the corresponding points in the coordinate systems of the two cameras, K1 and K2 are the internal parameter matrices of the two cameras, T1 and T2 are the external parameter matrices of the two cameras relative to the origin of space (in the specific implementation process, the midpoint above the cab can be set as the origin of space). s1 and s2 are the depth information (i.e. Z) of the point Pw calculated based on p1, p2, T1, T2, K1 and K2, respectively. The following equation can be established: Pw = s1*p1 / (K1*T1) = s2*p2 / (K2*T2), i.e. s1*p1 / (K1*T1)- s2*p2 / (K2*T2) = 0; Among them, K1 and K2 can be obtained during calibration, which is converted into the external parameter matrix of T1 and T2. Select 8 groups of non-collinear corresponding points to establish the equation system as follows: s1a*(x1a, y1a) / (K1*T1)- s1b*(x1b, y1b) / (K2*T2) = 0; s2a* (x2a, y2a) / (K1*T1) - s2b* (x2b, y2b) / (K2*T2) = 0; s3a*(x3a, y3a) / (K1*T1)- s3b* (x3b, y3b) / (K2*T2) = 0; s4a* (x4a, y4a) / (K1*T1) - s4b* (x4b, y4b) / (K2*T2) = 0; s5a* (x5a, y5a) / (K1*T1) - s5b* (x5b, y5b) / (K2*T2) = 0; s6a* (x6a, y6a) / (K1*T1) - s6b* (x6b, y6b) / (K2*T2) = 0; s7a* (x7a, y7a) / (K1*T1) - s7b* (x7b, y7b) / (K2*T2) = 0; s8a* (x8a, y8a) / (K1*T1) - s8b* (x8b, y8b) / (K2*T2) = 0; Wherein, s1a, s2a, s3a, s4a, s5a, s6a, s7a, s8a, s1b, s2b, s3b, s4b, s5b, s6b, s7b, s8b represent the depth of each matching point in the first image and the second image respectively, K1 and K2 represent the intrinsic parameter matrices of the cameras corresponding to the first image and the second image respectively, T1 and T2 represent the projection matrices corresponding to the first image and the second image respectively, (x1a, y1a), (x2a, y2a), (x3a, y3a), (x4a, y4a), (x5a, y5a), (x6a, y6a), (x7a, y7a), (x8a, y8a) represent the coordinates of the 8 matching points in the first image respectively; (x1b, y1 b), (x2 b, y2 b), (x3 b, y3 b), (x4 b, y4 b), (x5 b, y5 b), (x6 b, y6 b), (x7 b, y7 b), and (x8 b, y8 b) respectively represent the coordinates of the eight matching points in the second image.

[0051] Based on the two-dimensional coordinates of the matching points, the camera's intrinsic parameter matrix (K1 and K2) and the projection matrix (T1 and T2), the three-dimensional coordinates of the matching points in the first image and the second image are calculated to generate a three-dimensional source point cloud of the pile.

[0052] Based on the calculated three-dimensional coordinates of the matching points, a three-dimensional source point cloud of the pile is generated. In the specific implementation process, the present invention adopts the BundleAdjustment algorithm to generate a three-dimensional source point cloud of the pile. The BundleAdjustment algorithm is an advanced real-time three-dimensional reconstruction algorithm that combines sparse and dense image features and achieves high-precision three-dimensional reconstruction through a hierarchical local to global optimization strategy. The BundleAdjustment algorithm can handle large-scale scenes and can update the model in real time to reflect new observation data. The advantages of this algorithm include real-time, global optimization, robust tracking and repositioning capabilities, and efficient operation on the GPU. It can generate high-quality 3D models and maintain good reconstruction effects even under complex lighting and texture conditions, providing an efficient and consistent solution for the three-dimensional reconstruction of this scene.

[0053] Step (5) aligns the three-dimensional source point cloud with the preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

[0054] Pile body working condition recognition is to obtain the three-dimensional posture of the pile body and determine whether the pile body is driven into the ground in a vertical position. In the embodiment of the present invention, a point cloud registration algorithm is used. Specifically, the following steps are included: 1) Key point extraction: Extract key points from the 3D source point cloud and the 3D target point cloud. These key points can represent the features of the point cloud.

[0055] 2) Feature description: Calculate the feature point descriptors of key points, which are used in the subsequent matching process.

[0056] 3) Corresponding point estimation: The feature point descriptor based on the key points finds the corresponding point pairs between the 3D source point cloud and the 3D target point cloud through feature matching.

[0057] 4) Corresponding point screening: Use different strategies (such as distance threshold, RANSAC, etc.) to remove incorrect corresponding points to improve the accuracy of registration.

[0058] 5) Transformation estimation: Based on the screened corresponding point pairs, the optimal transformation matrix (including rotation and translation) is estimated so that the source point cloud can be aligned with the target point cloud.

[0059] 6) Transformation application: Apply the estimated optimal transformation matrix to the source point cloud to complete the registration and obtain the actual three-dimensional posture and installation angle of the pile.

[0060] There are many registration algorithms, such as ICP (Iterative Closest Point), Coherent Point Drift (CPD), Normal Distributions Transform (NDT), etc. Each algorithm has its specific application scenarios and advantages.

[0061] In the specific implementation, the ICP (Iterative Closest Point) algorithm in PCL (Point Cloud Library) can be used. This algorithm is a classic method for point cloud registration. It finds the best rigid body transformation between two point clouds in an iterative way to minimize the distance between the three-dimensional source point cloud (input source) and the target point cloud (input target). The following is the relevant algorithm flow: 1) Initialization: First, create two point cloud objects, one as the source point cloud (input source) and the other as the target point cloud (input target).

[0062] 2) Create an ICP object: Then, create a pcl::IterativeClosestPoint object, which is the class that implements the ICP algorithm in PCL.

[0063] 3) Set input: Set the source point cloud and target point cloud to the ICP object through the setInputSource and setInputTarget methods.

[0064] 4) Configure ICP parameters: You can set parameters such as the maximum number of iterations (setMaximumIterations), the maximum corresponding distance (setMaxCorrespondenceDistance), and the convergence threshold of the transformation matrix (setTransformationEpsilon).

[0065] 5) Perform alignment: Call the align method to perform the alignment and store the result in a new point cloud object.

[0066] 6) Check the results: Check whether the registration has converged through the hasConverged method, use getFitnessScore to obtain the fitness score of the registration, and use getFinalTransformation to obtain the final optimal transformation matrix. Example 2

[0067] Based on the same inventive concept as that of Embodiment 1, a device for identifying working conditions of a pile driver is provided in an embodiment of the present invention, comprising: An image acquisition module is used to acquire two parallax images of the pile body; A matching point recognition module is used to extract feature operators from the two obtained disparity pictures respectively, and identify matching points in the two disparity pictures based on the feature operator extraction results; A three-dimensional source point cloud generation module is used to calculate the three-dimensional coordinates of the matching points based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, and then generate a three-dimensional source point cloud of the pile body; The registration module is used to register the three-dimensional source point cloud with a preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

[0068] The rest are the same as in Example 1. Example 3

[0069] An embodiment of the present invention provides a pile driver working condition identification system, including a storage medium and a processor; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the method according to any one of the embodiments 1.

[0070] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

[0075] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for identifying working conditions of a pile driver, characterized in that: include: Get two parallax images of the pile; Perform feature operator extraction on the two obtained disparity pictures respectively, and identify matching points in the two disparity pictures based on the feature operator extraction results; Based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, the three-dimensional coordinates of the matching points are calculated, and then the three-dimensional source point cloud of the pile body is generated; The three-dimensional source point cloud is registered with a preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

2. A pile driver working condition identification method according to claim 1, characterized in that: The two disparity images are respectively the first image and the second image; the three-dimensional coordinates of the matching points are obtained by the following method: Calculate a projection matrix based on the two-dimensional coordinates of the identified matching points, the intrinsic parameter matrix of the camera and a preset first calculation formula; Based on the two-dimensional coordinates of the matching points, the intrinsic parameter matrix of the camera and the projection matrix, the three-dimensional coordinates of the matching points in the first image and the second image are calculated, and then the three-dimensional source point cloud of the pile body is generated.

3. A pile driver working condition identification method according to claim 2, characterized in that: The expression of the first calculation formula is: s1a*(x1a, y1a) / (K1*T1)- s1b*(x1b, y1b) / (K2*T2) = 0; s2a* (x2a, y2a) / (K1*T1) - s2b* (x2b, y2b) / (K2*T2) = 0; s3a*(x3a, y3a) / (K1*T1)- s3b* (x3b, y3b) / (K2*T2) = 0; s4a* (x4a, y4a) / (K1*T1) - s4b* (x4b, y4b) / (K2*T2) = 0; s5a* (x5a, y5a) / (K1*T1) - s5b* (x5b, y5b) / (K2*T2) = 0; s6a* (x6a, y6a) / (K1*T1) - s6b* (x6b, y6b) / (K2*T2) = 0; s7a* (x7a, y7a) / (K1*T1) - s7b* (x7b, y7b) / (K2*T2) = 0; s8a* (x8a, y8a) / (K1*T1) - s8b* (x8b, y8b) / (K2*T2) = 0; Wherein, s1a, s2a, s3a, s4a, s5a, s6a, s7a, s8a, s1b, s2b, s3b, s4b, s5b, s6b, s7b, and s8b represent the depths of the eight matching points in the first image and the second image respectively, K1 and K2 represent the intrinsic parameter matrices of the cameras corresponding to the first image and the second image respectively, T1 and T2 represent the projection matrices corresponding to the first image and the second image respectively, (x1a, y1a), (x2a, y2a), (x3a, y3a), (x4a, y4a), (x5a, y5a), (x6a, y6a), (x7a, y7a), and (x8a, y8a) represent the coordinates of the eight matching points in the first image respectively; (x1b, y1 b), (x2 b, y2 b), (x3 b, y3 b), (x4 b, y4 b), (x5 (x5 b, y5 b), (x6 b, y6 b), (x7 b, y7 b), and (x8 b, y8 b) respectively represent the two-dimensional coordinates of the eight matching points in the second image.

4. A pile driver working condition identification method according to claim 2, characterized in that: The three-dimensional source point cloud is generated by using the BundleAdjustment algorithm.

5. A pile driver working condition identification method according to claim 1, characterized in that: Before the step of extracting feature operators from the two acquired disparity pictures respectively, the method further includes: Grayscale the two obtained disparity pictures to generate a grayscale image; The grayscale value of each pixel in the grayscale image is compared with a preset grayscale threshold, and the grayscale value of each pixel is set to 0 or 255 according to the comparison result.

6. A pile driver working condition identification method according to claim 1, characterized in that: Perform feature operator extraction on the two obtained disparity pictures respectively, and identify the matching points in the two disparity pictures based on the feature operator extraction results, including: An improved FAST algorithm is used to extract FAST corner points in an image, wherein the FAST corner points have rotation invariance and scale invariance. Use the BRIEF algorithm to describe FAST corner points and generate feature point descriptors; The Hamming distance is used to match the feature point descriptors in the two images and identify the matching points in the two disparity images.

7. A pile driver working condition identification method according to claim 1, characterized in that: The step of registering the three-dimensional source point cloud with a preset three-dimensional target point cloud to obtain a pile lowering angle of the pile body includes: Extract key points from 3D source point cloud and 3D target point cloud; Based on the extracted key points, the corresponding point pairs between the 3D source point cloud and the 3D target point cloud are found through feature matching; Eliminate erroneous corresponding points and complete the screening of corresponding point pairs; Based on the screened corresponding point pairs, the optimal transformation matrix is ​​estimated; The estimated optimal transformation matrix is ​​applied to the 3D source point cloud to complete the registration and obtain the installation angle of the pile.

8. A pile driver working condition identification method according to claim 7, characterized in that: The method for estimating the optimal transformation matrix includes: Create an ICP object; Set the 3D source point cloud and 3D target point cloud to the ICP object through the setInputSource and setInputTarget methods; Set the maximum number of iterations, the maximum corresponding distance, and the convergence threshold of the transformation matrix; Call the align method to perform the alignment and store the alignment result in a new point cloud object; Check whether the registration has converged through the hasConverged method, use getFitnessScore to obtain the fitness score of the registration, and use getFinalTransformation to obtain the final transformation matrix.

9. A pile driver working condition identification device, characterized in that: include: An image acquisition module is used to acquire two parallax images of the pile body; A matching point recognition module is used to extract feature operators from the two obtained disparity pictures respectively, and identify matching points in the two disparity pictures based on the feature operator extraction results; A three-dimensional source point cloud generation module is used to calculate the three-dimensional coordinates of the matching points based on the identified two-dimensional coordinates of the matching points and the intrinsic parameter matrix of the camera, and then generate a three-dimensional source point cloud of the pile body; The registration module is used to register the three-dimensional source point cloud with a preset three-dimensional target point cloud to obtain the pile lowering angle of the pile body.

10. A pile driver working condition identification system, characterized in that: including storage media and processors; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the method according to any one of claims 1-8.

Images