Vertical rod double-machine perpendicularity judgment method and device

Abstract

The invention relates to the technical field of rod body perpendicularity detection, and discloses a vertical rod double-machine perpendicularity judgment method and device, and the method comprises the following steps: collecting an image and a roll angle through two devices, segmenting a rod body image through a deep neural network, fitting an axis, calculating an inclination angle under an image coordinate system, and obtaining an actual inclination angle after roll angle compensation, and finally, comparing the angles of the two machines to judge whether the rod body is vertical or not, thereby realizing non-contact real-time detection. The device comprises a triangular bracket, a carrying circular truncated cone, a device box shell, a USB camera, a Jetson nano embedded computing platform, an inertial measurement unit and a touch liquid crystal screen. According to the invention, through combination of deep neural network segmentation and image processing, efficient identification of the inclination angle of the rod body is realized, single-angle misjudgment is avoided by adopting a dual-computer system, installation deviation is compensated through a fusion algorithm, manual leveling and contact operation are not needed, and the detection efficiency and safety are improved; and meanwhile, good universality is achieved, and the device adapts to various rod bodies and environments.

Description

Technical Field

[0001] The present invention relates to the technical field of pole verticality detection, and particularly to a method and device for judging the verticality of an upright pole by using two machines. Background Art

[0002] During the installation and construction process of various poles such as transmission poles, street lamp poles, traffic sign poles, and signal poles, how to ensure the vertical state of the pole after installation is one of the key factors affecting the structural stability and safety. The pole not only bears functional components but also plays an important structural support role in its application environment. If the pole is skewed during the installation stage, risks such as uneven stress on the pole, structural fatigue, and even toppling may occur during subsequent use, and in severe cases, it may cause facility damage or personal injury. Therefore, real-time and accurate detection and adjustment of the verticality of the pole are necessary means to improve the project quality and construction safety level.

[0003] Currently, the commonly used verticality detection methods at the construction site include visual inspection method, total station measurement, inertial sensor detection, traditional level or plumb line method, and a small number of special mechanical auxiliary devices. However, these methods have many limitations in on-site operations: The visual inspection method relies on subjective judgment of personnel, with large errors and limited directions, and it is difficult to provide quantitative data; Although the total station has high measurement accuracy, its operation is complex and multiple fixed-point observations are required, making it difficult to meet the needs of rapid adjustment during the construction stage; Traditional sensor-based devices and mechanical devices usually need to be fixed on the pole, which not only increases the operation process but also introduces safety risks brought by contact installation. In addition, these methods generally do not have the ability of long-distance non-contact, and lack an efficient human-machine interface feedback mechanism, restricting the improvement of construction automation level; For this reason, a method and device for judging the verticality of an upright pole by using two machines are proposed to solve the above problems. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a method and device for judging the verticality of an upright pole by using two machines, which use a dual-view device to collect images from a distance and calculate the tilt angle, combine with an IMU for angle compensation, and realize data synchronization through WiFi. The host interface displays the angle information of the two views in real time, realizing single-person, non-contact, real-time, and high-precision verticality judgment, effectively improving construction efficiency and safety.

[0005] To achieve the above object, the present invention is realized through the following technical solutions: A method for judging the verticality of an upright pole by using two machines includes the following steps:

[0006] S1. Use two verticality judgment devices to perform real-time image acquisition of the pole from two directions with an included angle of nearly 90 degrees through their respective USB cameras, and at the same time obtain the roll angle of the camera corresponding to the device through an inertial measurement unit, obtaining image data and roll angle data;

[0007] S2. Transmit the image data and roll angle data at the same moment to the embedded computing platform;

[0008] S3. Run a deep neural network on the embedded computing platform to perform instance segmentation on the image, extract the rod body area, and output a binary image;

[0009] S4. Perform a row scan on the rod body pixels in the binary image to obtain the midpoints of the axial pixels, and obtain the axis of the rod body through a fitting algorithm;

[0010] S5. Calculate the tilt angle of the rod body axis in the image coordinate system based on the fitting algorithm and convert it to the tilt angle relative to the opposite direction of the Y-axis;

[0011] S6. Perform angle compensation in combination with the roll angle to obtain the tilt angle of the rod body in the device coordinate system;

[0012] S7. Compare the tilt angle values of the two plumb line determination devices. When both are zero, it is determined that the rod body is vertical.

[0013] Preferably, the deep neural network is a lightweight improved YOLOv8s-seg model, and the YOLOv8s-seg model is used to perform instance segmentation on the rod body image and output a binary mask image of the rod body area.

[0014] Preferably, the embedded computing platform is a Jetson nano embedded computing platform, which is used to receive image and roll angle data and perform deep neural network instance segmentation and rod body axis fitting calculations.

[0015] Preferably, the angle compensation adds the tilt angle calculated from the image to the roll angle measured by the inertial measurement unit to obtain the compensated tilt angle, where the roll angle is positive when tilting to the right and negative when tilting to the left, that is:

[0016] α = θ + β;

[0017] Among them, α is the angle compensation; θ is the tilt angle; β is the axis tilt angle.

[0018] Preferably, the two plumb line determination devices transmit the tilt angle data and image processing results to each other through a wireless communication method for the master control device to comprehensively judge whether the rod body is vertical.

[0019] Preferably, each plumb line determination device includes a USB camera, an inertial measurement unit, a Jetson nano embedded computing platform, and a touch display module, and the touch display module is used to display the image and the calculated tilt angle of the rod body.

[0020] The present invention also provides a verticality judging device for a vertical pole with two machines, which includes a triangular bracket. A load-bearing round table is arranged at the top of the triangular bracket, and a device box housing is arranged at the top of the load-bearing round table. An embedded computing platform Jetson nano and an inertial measurement unit are horizontally arranged on the inner bottom wall of the device box housing respectively, and the inertial measurement unit is used to measure the roll angle data of the bottom surface of the box in real time. A USB camera and a touch liquid crystal display screen are arranged on the inner wall of the device box housing respectively, and the USB camera and the touch liquid crystal display screen are symmetrically distributed.

[0021] Preferably, the Jetson nano embedded computing platform is connected to the control signal transmission ends of the inertial measurement unit, the USB camera and the touch liquid crystal display screen through a USB cable, and the Jetson nano embedded computing platform is connected to the touch liquid crystal display screen through an HDMI cable to send image signals to it.

[0022] Preferably, the USB camera faces the pole to collect images of the pole.

[0023] Preferably, the touch liquid crystal display screen faces the operator to display the captured pole images and pole inclination parameter information.

[0024] The present invention provides a method and device for judging the verticality of a vertical pole with two machines. It has the following beneficial effects:

[0025] 1. The present invention adopts an instance segmentation method based on YOLOv8s-seg, combines image processing to calculate the inclination angle of the pole, and realizes fast and high-precision pole attitude recognition. Compared with traditional image algorithms based on edge detection or simple threshold segmentation, this technical solution significantly reduces the segmentation error, no longer relies on manual adjustment and complex post-processing, and solves the technical problems of slow recognition speed, low accuracy and easy background interference in the past.

[0026] 2. The present invention calculates the angle of the pole by setting up a two-machine system respectively. In terms of structural design, the present invention avoids multiple positioning or manual rotation of the same device, and the data synchronization judgment in two directions is more direct. Compared with the existing method that requires a single device to measure at multiple angles, it avoids the reference error caused by position switching and no longer requires manual judgment of the pole inclination, thus solving the problem of incomplete acquisition of direction information.

[0027] 3. The device of the present invention does not require any manual leveling action, and all installation deviations can be automatically compensated by the fusion algorithm, and accurate angles can still be output on the premise that the device is not leveled. This design enables the device to directly work on non-flat surfaces without relying on any external platform or horizontal reference. Compared with the traditional total station or optical instrument that requires complex angle adjustment, it effectively overcomes the strong dependence on the equipment placement environment and greatly improves the portability and deployment efficiency.

[0028] 4. The present invention relies on automated image processing and attitude fusion algorithms to achieve a full closed-loop detection process. Users do not need to contact the target rod body, nor manually measure or review the judgment results. The detection process is completed quickly, and all calculations are automatically executed by the system. Compared with existing detection devices that require manual auxiliary reading or operating a robotic arm, it greatly reduces the intensity of manual operation and skill requirements, and solves the technical shortcomings of high labor cost, low efficiency, and easy error in the measurement link.

[0029] 5. The present invention combines a deep learning model with a general computing process. It does not depend on the specific shape, material, or site conditions of the rod body, and there is no need to perform pre-set modeling or hardware adjustment on the detection object. In some cases, only the model needs to be retrained to adapt to different rod types or environments. Different from existing detection schemes that require re-calibration or structural modification for the target, this technology overcomes the problems of poor versatility and limited applicable scenarios, and has good flexibility and expandability. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is the method flow chart of the present invention;

[0031] Figure 2 is the schematic diagram of the plumb judgment method of the present invention;

[0032] Figure 3 is the device structure diagram of the present invention;

[0033] Figure 4 is the internal structure diagram of the device of the present invention;

[0034] Figure 5 is the schematic diagram of the connection of internal modules of the device of the present invention;

[0035] Figure 6 is the schematic diagram of the process of extracting the axis of the rod body of the present invention;

[0036] Figure 7 is the human-machine interface of the device of the present invention;

[0037] Figure 8 is the schematic diagram of the structure of the improved lightweight YOLOv8s-seg of the present invention.

[0038] Among them, 1. Jetson nano embedded computing platform; 2. Inertial measurement unit; 3. USB camera; 4. Touch liquid crystal screen; 5. Device box shell; 6. Carrying turntable; 7. Triangular bracket. DETAILED DESCRIPTION OF THE INVENTION

[0039] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] Please refer to the attached Figure 1 - attached Figure 2 , the embodiment of the present invention provides a method for judging the verticality of a vertical pole with two machines, including the following steps:

[0041] S1. Use two verticality judgment devices to perform real-time image acquisition of the pole body from two directions with an included angle of nearly 90 degrees through their respective USB cameras, and at the same time obtain the roll angle of the camera corresponding device through the inertial measurement unit to obtain image data and roll angle data;

[0042] In the present invention, in order to realize the spatial perception and accurate determination of the verticality of the pole body, the image acquisition and attitude perception links of the two-machine combination constitute the precondition basis of the entire verticality judgment method. This step not only relates to the accuracy of subsequent image processing and angle calculation, but also directly determines whether the final judgment logic has spatial geometric significance.

[0043] Generally, it is difficult to accurately judge the inclination direction and angle of the pole body by only collecting the pole body image from a single perspective in space. Especially in the two-dimensional image coordinate system, there is a misjudgment phenomenon caused by the shooting angle. Therefore, the present invention adopts the idea of separating two devices and collecting images in different directions at the initial stage of system design to construct a stable spatial dual-view input model.

[0044] Specifically, the two verticality judgment devices are respectively arranged on both sides of the pole body, and the included angle between the perspectives is controlled between 85° and 95°, with an included angle of nearly 90 degrees as the standard angle. While not affecting the human operation area, it ensures that the imaging areas fully overlap. The two devices respectively perform synchronous image acquisition through the USB cameras configured by themselves.

[0045] In this embodiment, each verticality judgment device is equipped with a set of USB cameras. The camera selection supports automatic exposure and wide dynamic compensation functions, and the resolution is preferably 1280×720, and the frame rate is maintained above 20fps; the shooting direction faces the pole body, and the optical axis of the lens is kept as perpendicular to the surface of the pole body as possible to avoid image distortion caused by too large a shooting angle. In actual use, the camera is usually fine-tuned through an adjustable bracket to adapt to different pole body shapes on site.

[0046] During image acquisition, the Inertial Measurement Unit (IMU) built into each device is used to obtain the attitude data of the device body in real time, especially the roll angle information. As an option, the present invention uses a nine-axis sensing unit, including a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer, which outputs attitude angles in real time through a fusion algorithm. Among them, the roll angle (RollAngle), denoted as θ, is the key for the present invention.

[0047] The roll angle θ refers to the rotation angle of the device body around its forward axis (front-back direction axis), with the unit of degree system (°), and it can be positive or negative. The defined direction of θ is: when the right side of the device is lifted, θ > 0; when the left side of the device is lifted, θ < 0. This angle is used for the subsequent compensation calculation of the image axis tilt angle β.

[0048] In a possible implementation, to ensure the time alignment of image data and attitude data, the system binds and stores each frame of image and the corresponding roll angle value through timestamps at the software layer, that is, uses a unified system clock to add marked timestamps to the image frames and IMU data. In this way, in the subsequent processing module, the image and its corresponding attitude data can be called one by one, avoiding errors caused by sampling delays.

[0049] As an optional implementation, each verticality judgment device forms an "image-attitude pair" by packing the collected image frames and the corresponding θ values through an intranet connection structure, and uniformly transmits them to the Jetson nano embedded computing platform to support all steps of subsequent segmentation, fitting, and angle calculation.

[0050] In some embodiments, to further improve the data transmission stability, the camera data acquisition uses a USB2.0 interface for transmission, and the IMU data acquisition is transmitted through a serial bus. A scheduling thread runs inside the Jetson nano platform to manage the reading process of the two-way data in a time-sharing manner, ensuring that the image stream and angle data are completely passed into the processing pipeline.

[0051] Generally speaking, this step completes the basic acquisition tasks of images and roll angles, forming the following two key data objects:

[0052] Image frame I(x, y, t): where x and y are the pixel coordinates of the image, and t is the frame time;

[0053] Attitude angle θ(t): It is the roll angle data corresponding to the image frame time t, with the unit of °;

[0054] Subsequently, based on this as the input basis, it will enter stages such as deep neural network instance segmentation and image axis angle extraction, and perform fusion compensation with θ(t), and then complete the final verticality judgment.

[0055] The nearly vertical deployment of two machines used in this step provides sufficient redundancy and compensation capabilities in spatial angle distribution, which can significantly improve the fault tolerance of pole posture perception, especially in complex scenes with slender poles, unclear boundaries or strong background interference, and still has good stability.

[0056] S2, transmitting the image data and the rolling angle data at the same time to the embedded computing platform;

[0057] After completing the synchronous acquisition of image and posture information, in order to realize the subsequent image recognition and angle fusion processing, the original data needs to be reliably and efficiently transmitted to the core processing module of the system, that is, the embedded computing platform. As the receiving node of the data processing link, this step plays a key role in accurate data alignment, time consistency, and bandwidth scheduling. This process is directly related to the input integrity of the image segmentation algorithm and the credibility of the final calculation results.

[0058] Generally, the amount of image data is large, while the roll angle data collected by the inertial measurement unit is small. Therefore, in terms of data organization structure, this embodiment prioritizes binding the image frame and the attitude angle to form a single-frame synchronous data packet, thereby simplifying the data path design.

[0059] In this embodiment, the image data from the USB camera is a three-channel color image in RGB format with a size of 1280×720 pixels; while the roll angle from the inertial measurement unit is a single floating-point data in degrees (°) with a resolution of 0.1°. When the image frame arrives, the system scheduling thread reads the θ value from the IMU and records the timestamp.

[0060] As an option, the system encapsulates each frame image I(x, y, t) and the roll angle θ(t) at the corresponding moment into a structured data object D:

[0061] D = {I(x, y, t), θ(t)};

[0062] Where I(x,y,t) represents the image frame captured at time t; x,y are the pixel coordinates of the image, in pixels; θ(t) is the roll angle corresponding to the image, in degrees; θ>0 indicates that the device is tilted to the right, and θ<0 indicates that the device is tilted to the left.

[0063] In one possible implementation, the D data object is transmitted to the Jetson nano embedded computing platform via a built-in bus interface; the specific interface may be USB2.0, USB3.0 or a serial bus, depending on the device port compatibility and on-site transmission requirements.

[0064] Specifically, in the system structure of the present invention, the Jetson nano embedded computing platform serves as the main processing unit, integrating a GPU module and a main control CPU, which can directly run lightweight neural network models and simultaneously complete data synchronization and cache scheduling for multiple threads.

[0065] In this embodiment, the transmission mechanism adopts a double-buffer structure. While the main thread receives the current frame D, the sub-thread processes the data of the previous frame to avoid blocking the main process; the image frame data is temporarily stored in the high-speed buffer, and the IMU data is read through the serial port interrupt. Each frame of data is marked with a high-precision system timestamp, and a time difference threshold ≤ 5ms is used as the image and attitude matching window.

[0066] In some embodiments, to address possible frame loss or instantaneous IMU data loss problems, the system is embedded with a data fault tolerance mechanism. When a certain frame lacks the θ value or the image frame is abnormal, the frame is marked as an invalid frame and excluded in subsequent fitting to ensure the stability of the calculation results.

[0067] As an optional mechanism, data transmission can also be achieved wirelessly. In a deployment scenario with a WiFi network, two devices can package the D object through the TCP protocol and transmit it to the Jetson nano. The main processing platform marks the device number according to the source address of the transmission, so as to identify whether the data source is from the left device or the right device.

[0068] Furthermore, in some feasible expansion schemes, to reduce the wireless transmission bandwidth pressure, the system also supports compressing and encoding the image data I(x,y,t), such as using the JPEG or H.264 encoding format, and then decoding it at the platform end to restore it to a processable image.

[0069] It should be noted that this transmission structure is not limited to the simple synchronization of image frames and roll angles, but can also be extended to the intermediate state feedback of subsequent image segmentation results and angle calculation values, thereby supporting system self-checking and process monitoring, and providing interface reservation capabilities for subsequent deployment to an automated operation system.

[0070] S3. Run a deep neural network on the embedded computing platform to perform instance segmentation on the image, extract the rod body area, and output a binary image;

[0071] After receiving the image data and roll angle data, the system enters the processing stage based on visual semantic understanding. The core task of this stage is to accurately separate the area related to the target rod from the background in the input original image and output a clear mask result in the form of an image. As a deep learning-based image semantic recognition technology, instance segmentation not only needs to identify the target location but also segment the specific pixel area. In the technical path of the present invention, this module undertakes the responsibility of extracting the rod area in the complex scene as the input for subsequent structural feature analysis, playing a bridging role.

[0072] Generally, traditional image segmentation methods have poor adaptability to complex scenes and are easily interfered by the background. Especially when the color of the rod is similar to the background or the edge is blurred, the recognition accuracy is not high. The present invention selects a lightweight deep neural network model and completes model inference on an embedded platform, ensuring both computational efficiency and recognition accuracy.

[0073] In this embodiment, the embedded computing platform is Jetson nano. The trained and optimized YOLOv8s-seg model runs inside the platform. After the model structure is pruned and the number of parameters is adjusted, it can achieve high-frame-rate segmentation inference in the embedded GPU environment.

[0074] Specifically, the YOLOv8s-seg model uses the backbone network Backbone to extract image features, performs multi-scale feature fusion through the neck structure Neck, and finally outputs the InstanceMask result. The training of the model adopts the data annotation method in COCO format, and is specifically trained for the rod target category. The loss function includes three items: category loss, bounding box regression loss, and mask segmentation loss.

[0075] As an option, to improve the robustness of the model, data augmentation strategies such as random scaling, brightness perturbation, and mirror augmentation are introduced in the training stage, enabling the model to still be recognizable under complex backgrounds and various lighting conditions.

[0076] In this embodiment, the embedded platform receives the input image I(x, y, t), and outputs a binary image M(x, y, t) after being processed by the neural network. Its definition is as follows:

[0077]

[0078] Among them, M(x, y, t) represents the segmentation result image corresponding to time t; (x, y) is the pixel point coordinate; R(t) is the domain of the rod area in the image, that is, the set of predicted rod pixels.

[0079] In a possible implementation, to avoid the influence of noise interference on subsequent fitting, the system performs morphological processing on the output M(x, y, t), such as dilation, erosion, connected component filtering, etc., to remove isolated pixels and false target interference.

[0080] Further, in some embodiments, contour extraction operations are performed on M(x, y, t) to obtain the rod edge information C(t); the edge can be used to determine whether the rod is deformed or has a discontinuous structure, which is suitable for subsequent anomaly detection expansion.

[0081] In addition, in this embodiment, the original image I and the segmentation result M are pixel-by-pixel fused to form a three-channel visible image I′ for on-site display and manual verification. The display image is defined as follows:

[0082] I′(x, y, t) = I(x, y, t) × (1 - M(x, y, t)) + [0, 255, 0] × M(x, y, t);

[0083] Among them, I′(x, y, t) represents the pseudo-color image after adding the mask; the green [0, 255, 0] is the marked rod area; other areas maintain the original image color.

[0084] This mask image can be used both as the input for subsequent axis fitting and as the visual feedback of the on-site result, improving the interaction efficiency.

[0085] S4. Perform a row scan on the rod pixels in the binary image to obtain the midpoints of the axial pixels, and obtain the rod axis through a fitting algorithm;

[0086] After completing the segmentation of the rod area, the system needs to further extract geometric information from the segmentation result to establish a structural description of the rod in the image coordinate system. The key objective here is to obtain the position of the main axis of the rod from the image and calculate the spatial orientation of this axis. Since the rod pixel area has been peeled off from the background in the instance segmentation stage, the current operation is based on this binary mask image, performs pixel-by-pixel analysis row by row, and obtains the central axis through a fitting algorithm. The processing result of this step will participate in the inclination calculation in the next stage. Therefore, its accuracy and stability have a direct impact on the final verticality judgment result.

[0087] Generally, the rod presents long and narrow, upright geometric features, and is distributed as a continuous, vertical or nearly vertical pixel area in the image. Therefore, by using the method of row-by-row scanning, the central trend of the rod contour can be effectively located. By extracting the midpoint pixel coordinates on each scanned row, a representative axial point set is formed, and then the main axis trend is restored through a fitting algorithm.

[0088] In this embodiment, the input data is a mask image M(x, y, t), where the part with a pixel value of 1 represents the rod body area. The system scans all pixel points in each row y in sequence from the top to the bottom in the longitudinal dimension of the image to identify the boundary coordinates of the white area.

[0089] As an option, only the leftmost and rightmost pixel points in the continuous pixel block are retained in each row, denoted as L(y) and R(y), then the center point C(y) of this row is calculated as follows:

[0090] C(y) = ((L(y) + R(y)) / 2, y);

[0091] Among them, L(y) is the abscissa of the leftmost white pixel in the y-th row; R(y) is the abscissa of the rightmost white pixel in the y-th row; C(y) is the horizontal midpoint coordinate of the rod body in the y-th row.

[0092] Generally, to eliminate the interference of abnormal rows, the system sets a height threshold and only extracts the center points for the rows with a white area of sufficient width to avoid being misled by pseudo-points caused by edge sawtooth or light interference for fitting.

[0093] After the set of center points C = {C1, C2,..., C} of all valid rows is extracted, the system performs a linear fitting on this point set. The fitting method uses the least squares linear regression, and the goal is to solve the optimal linear function:

[0094] x = k · y + b;

[0095] Among them, k is the slope of the axis of the rod body in the image coordinate system; b is the abscissa intercept of the straight line when y = 0, and x, y are the pixel point positions in the image coordinate system.

[0096] In a possible implementation, the system performs denoising processing on the point set C, removes outliers, and then performs linear fitting to improve robustness. The fitting process satisfies the derivation of the standard linear least squares formula, and the objective function is:

[0097] min∑(x i -(k · y i + b)) 2 ;

[0098] Among them, (x i , y i ) is the coordinate of the i-th center point; n is the number of fitting sample points.

[0099] Once the fitting is completed, the system will output the linear function x = k · y + b and use the slope k as the input parameter for subsequent angle calculation.

[0100] Specifically, this slope represents the degree of deviation of the rod body in the image plane relative to the Y-axis direction. This slope will be fused with the roll angle θ in the subsequent angle calculation stage for compensation.

[0101] In some embodiments, for the convenience of display and manual verification, the system can draw the fitted main axis on the original image to form a superimposed image for screen display and archiving.

[0102] Furthermore, to support anomaly detection, the system records the axis fitting residual, that is, the mean square error of all points to the line. If this error exceeds the set threshold, it will prompt the user "abnormal rod body shape" or "unreliable fitting" to enhance the system stability.

[0103] S5. Calculate the tilt angle of the rod axis in the image coordinate system based on the fitting algorithm and convert it to the tilt angle relative to the opposite direction of the Y-axis;

[0104] After completing the pixel fitting of the main axis of the rod body, the system needs to further analyze the fitting result to obtain the actual tilt angle information of the rod body in the image plane coordinate system. As an important quantitative parameter describing the direction of the rod body, this angle is not only used to determine whether it is vertical but also will be fused with the inertial attitude angle in the subsequent process to obtain the final compensation angle. The calculation of the tilt angle in the image coordinate system is derived from the slope of the fitted line. Therefore, this step logically directly inherits from the main axis fitting result in the previous step and realizes the angle conversion through the inverse trigonometric function.

[0105] Generally, the image coordinate system is represented by a two-dimensional Cartesian coordinate system with the upper left corner as the origin, the X-axis to the right, and the Y-axis downward. Therefore, the slope k of the fitted axis can be positive or negative, and the correct geometric meaning can be obtained only after parsing in combination with the coordinate direction rule. For the convenience of unified analysis, the present invention defines the tilt angle as the angle between the rod axis and the opposite direction of the image Y-axis, and the direction adopts the mathematical positive direction, that is, clockwise is positive.

[0106] In this embodiment, the expression of the fitted line has been obtained in the previous processing step:

[0107] x = k·y + b;

[0108] where k is the slope of the rod axis in the image coordinate system; b is the x-intercept when y = 0, and x, y are the pixel positions in the image coordinate system.

[0109] In this coordinate system, the tilt angle β of the rod axis relative to the opposite direction of the Y-axis can be calculated by the following formula:

[0110] β = arctan(k);

[0111] Among them, β is the inclination angle of the axis of the rod in the image coordinate system, with the unit of radian or degree (°), and the specific value is determined by the implementation method; k is the slope obtained in the previous step.

[0112] As an option, in the actual operation of the system, β is usually expressed in degrees, and the output result of the arctangent function needs to be multiplied by a conversion factor:

[0113] β = arctan(k) × (180 / π);

[0114] Among them, π is the constant of pi, approximately 3.1416; 180 / π is the coefficient for converting radians to degrees.

[0115] Specifically, if the rod is inclined to the right side of the image, the slope of the fitted line is positive, and the corresponding β is positive; conversely, if the rod is inclined to the left, β is negative; if the fitted line is approximately vertical, k approaches infinity, and β approaches 90° or -90°.

[0116] In a possible implementation method, to avoid numerical calculation errors caused by too large a value of the slope k, the system will limit the value range of the slope. For example, the absolute value of the slope is limited within ±100, and values outside this range will trigger an exception prompt.

[0117] In some embodiments, the system will simultaneously calculate the angle α' between the axis of the rod and the Y-axis of the image as a reference value, and its calculation formula is:

[0118] α' = 90° - |β|;

[0119] This angle α' is used to describe the degree of deviation of the axis of the rod from the vertical direction, which helps to manually determine whether the system output is credible.

[0120] Furthermore, in some implementations, the fitted slope will be converted into the form of a direction vector, that is, the fitted line is expressed as a set of vector coordinates for subsequent three-dimensional attitude expansion analysis; however, in the main solution of the present invention, β is used as the only characterization parameter of the image inclination angle.

[0121] S6. Perform angle compensation in combination with the roll angle to obtain the inclination angle of the rod in the device coordinate system;

[0122] After calculating the inclination angle of the rod in the image coordinate system, if this angle is directly used as the basis for determining the rod attitude, it may introduce deviations caused by the attitude change of the device itself. Therefore, in order to obtain a rod inclination angle closer to the true direction in space, the installation attitude of the device, that is, the roll angle parameter, needs to be introduced into the compensation process to form a fused angle quantity. This angle compensation link is a key step for the system to achieve spatial verticality measurement, and its core lies in mathematically combining the image recognition result with the sensor output to complete the attitude correction under coordinate transformation.

[0123] In general, it is impossible to ensure that the device is completely parallel or perpendicular to the ground at the use site, and slight inclination is inevitable. If the image angle β is not compensated, there will be significant errors. Especially when the deviation angle of the device is large, this error can no longer be ignored. Therefore, in the structural design of the present invention, the roll angle data θ is introduced to participate in the fusion to compensate the systematic error of the image visual angle and further improve the spatial consistency of the rod body attitude estimation.

[0124] In this embodiment, on the premise of knowing the image tilt angle β, the current device roll angle θ is obtained, and the fusion angle α is constructed to represent the actual tilt angle of the rod body in the device coordinate system. The compensation relationship is as follows:

[0125] α = θ + β;

[0126] Among them, α is the angle compensation; θ is the tilt angle; β is the axis tilt angle.

[0127] Specifically, if the installation direction of the device remains vertical but the rod body tilts to the right, then β > 0 and θ ≈ 0, and at this time α ≈ β;

[0128] If the rod body itself is vertical but the device tilts to the left, then θ < 0 and β ≈ 0, and at this time α ≈ 0, reflecting the influence of the system's own attitude;

[0129] If the rod body tilts to the right and the device tilts to the right at the same time, then α is the sum of the two, representing the true right tilt angle of the rod body in the world coordinate system.

[0130] As an option, the system internally expresses the fusion angle in a weighted form, allowing the user to adjust the weight coefficient according to the sensor accuracy or image credibility:

[0131] α = w1θ + w2β;

[0132] Among them: w1 and w2 are the weight coefficients of the image angle and the roll angle respectively, satisfying w1 + w2 = 1, and the unit is dimensionless real number;

[0133] Generally, it is set as w1 = 0.5 and w2 = 0.5, indicating that both participate in the compensation with equal weights;

[0134] In some embodiments, when the image quality deteriorates due to poor lighting, it can be adjusted to w1 < w2, so as to enhance the system's dependence on inertial sensing data.

[0135] In a possible implementation manner, in order to avoid the result jitter caused by the instantaneous angle jump, a low-pass filter module is introduced before the α output of the system, and the sliding average or first-order recursive form is used to smooth the compensation result. The filtering formula is:

[0136] α = γ·α -1 +(1 - γ)·α*;

[0137] Where: α is the output angle at time t; α -1 is the compensation angle result at the previous moment; γ is the filtering coefficient, and its value range is [0, 1], usually 0.7 - 0.9.

[0138] Furthermore, to enhance the anti-interference ability of the system, this embodiment also introduces an "invalid compensation detection mechanism". When either θ or β exceeds the allowable measurement range (such as ±90°) or the detection is empty, the compensation angle α of this frame will be marked as invalid, and the system will skip the subsequent processing to prevent the propagation of error values.

[0139] In some embodiments, the system will also record the historical sequence of α for dynamically judging the stability of the rod body attitude. For example, if the fluctuations of α in N consecutive frames are within the set threshold, it is prompted that the rod body has tended to be stable. S7. Compare the tilt angle values of the two plumb devices. When both are zero, it is judged that the rod body is vertical;

[0140] After obtaining the compensation angle α from the dual-machine system, the system enters the final judgment stage, that is, to judge whether the rod body is in a vertical state according to the measurement results in two directions. This link is the output logic end point of the whole method, and its core goal is to combine the measurement angles of the two devices respectively to obtain an overall conclusion about the attitude of the same rod body. Through the fusion judgment from two-way perspectives, not only the direction blind area that may exist in a single angle is overcome, but also the fault tolerance ability of the system to spatial errors and local deformations is improved.

[0141] Generally, the rod body may be tilted in multiple directions in space. If only relying on the measurement results of a single-side device, it is easy to cause deviations due to shooting angles, local occlusion, background interference, etc.; by independently calculating the compensation angles of the two devices respectively and then making a comprehensive comparison and judgment, the stability and reliability of the judgment results can be effectively enhanced.

[0142] In this embodiment, the dual-machine system respectively outputs its respective compensation angles α1 and α2, both of which are real numbers with the unit of angle (°), and the calculation methods have been defined in the previous steps, that is:

[0143] α1 = θ1 + β1;

[0144] α2 = θ2 + β2;

[0145] Where, θ1 and θ2 are the roll angles of the two devices respectively, with the unit of angle; β1 and β2 are the tilt angles obtained by image fitting of the two devices respectively, with the unit of angle; α1 and α2 are the compensation angle results calculated by the two devices.

[0146] The system takes these two angles as inputs for verticality judgment. The judgment criterion is: if the absolute values of α1 and α2 are both less than the set threshold ε, it is determined that the rod body is in a vertical state. The mathematical form is as follows:

[0147] If |α1| ≤ ε and |α2| ≤ ε, it is determined to be vertical;

[0148] where ε is the vertical judgment threshold set by the system, with the unit of angle (°), usually set to 1.0 or 0.5, and the specific value depends on the measurement accuracy requirements;

[0149] |α1| and |α2| represent the absolute values of two angles, avoiding the interference of direction positive and negative on the results.

[0150] Specifically, if α2 = -0.6° and ε = 1.0°, then |α1| = 0.8 < ε and |α2| = 0.6 < ε. At this time, it is judged as "vertical";

[0151] Conversely, if any angle exceeds the threshold, for example, α1 = -2.2°, the system outputs "not vertical" and provides the corresponding direction and angle prompt.

[0152] As an option, in this embodiment, on the basis of vertical judgment, an "inclination direction indication" function is additionally provided. When α1 and α2 have the same sign, it means that the whole rod body deviates in the same direction; when α1 and α2 have different signs but close values, there may be torsion or bending; such information helps on-site manual analysis of the deformation trend of the rod body.

[0153] In a possible implementation, the system introduces a communication module for angle data synchronization between two machines. The two devices establish a TCP communication link through the Wi-Fi network. Each device packs and sends its current frame's α value to the other party and receives the α data packet from the other end to complete the two-way exchange.

[0154] The data packet format is defined as:

[0155] P = {α, ID, T};

[0156] where α is the compensated angle value, with the unit of angle (°); ID is the device number, identifying the left or right device; T is the timestamp, used to match the frame time, and the general accuracy requirement is at the millisecond level.

[0157] The system simultaneously displays the α angle values on both the left and right sides and the vertical state of the rod body on the main control interface. The display interface can be marked in zones and is equipped with color prompts. For example, green represents "vertical", yellow represents "slightly skewed", and red represents "significantly skewed".

[0158] Furthermore, for the convenience of data archiving and retrospective analysis, the system will also record the judgment result of each frame and its corresponding angle value to generate a structured log file; this log can be used for quality traceability, construction acceptance, or subsequent improvement of the learning model.

[0159] Please refer to Appendix Figure 3 - AppendixFigure 5 , the present invention also provides a vertical pole dual - machine verticality judgment device, including a triangular bracket 7, which is used to support modules such as a load - carrying turntable 6. The top of the triangular bracket 7 is provided with a load - carrying turntable 6, and the load - carrying turntable 6 is used to install the housing of the device box 5. The top of the load - carrying turntable 6 is provided with the housing of the device box 5. On the inner bottom wall of the housing of the device box 5, a Jetson nano embedded computing platform 1 and an inertial measurement unit 2 are horizontally arranged respectively. And the inertial measurement unit 2 is used to measure the roll angle data of the bottom surface of the box in real time. On the inner wall of the housing of the device box 5, a USB camera 3 and a touch liquid crystal screen 4 are arranged respectively, and the USB camera 3 and the touch liquid crystal screen 4 are symmetrically distributed. The USB camera 3 is used to collect pole data, the USB camera 3 faces the pole to collect pole images, and the touch liquid crystal screen 4 is used to display the collected data and the human - machine interaction interface. The touch liquid crystal screen 4 faces the operator to display the captured pole images and pole inclination parameter information;

[0160] The Jetson nano embedded computing platform 1 is connected to the control signal transmission ends of the inertial measurement unit 2, the USB camera 3 and the touch liquid crystal screen 4 through a USB cable. The USB cable is used to transmit data. The Jetson nano embedded computing platform 1 is connected to the touch liquid crystal screen 4 through an HDMI cable to send image signals to it.

[0161] Please refer to the appendix Figure 6 - appendix Figure 8 , for the human - machine interaction interface displayed on the touch liquid crystal screen, the interface is divided into three areas. Two image display areas are the pole images captured by the camera of this device and the pole images captured by another device and transmitted to this device through the WIFI channel. The right - hand area of the interface simultaneously displays the roll angles (host inclinations) of the cameras on both devices, the inclination angles (pole angles) of the pole axes measured from different angles by the two devices in their image coordinate systems, and the calculated actual inclination angle of the pole (pole inclination).

[0162] The Jetson nano embedded computing platform uses a lightweight improved embedded YOLOv8s - seg instance segmentation model to perform target instance segmentation on the pole images. The structure of the lightweight improved YOLOv8s - seg is as Figure 8 shown. The outputs of each EfficientViTBlock stage passed by the input are added together as the input of the SPPF layer. The output of the SPPF is downsampled once and then concatenated with the output of the third stage of the EfficientViTBlock of the Backbone. After the second downsampling, it is concatenated with the output of the second stage of the EfficientViTBlock of the Backbone.

[0163] The rod images captured by the device are processed through the above steps and methods, and the inclination angle data of the rod is calculated and displayed on the touch liquid crystal screen in real time. The operator interacts with the system using the touch screen user interface. The Jetson nano embedded computing platform receives user commands and operations from the touch screen. According to the user's instructions. The control process of the entire system is a collaborative process, where the Jetson nano acts as the central processing unit, coordinating and integrating the data and operations of each module to complete the vertical judgment task. The operator can directly interact with the system through the touch screen interface to achieve convenient operation and control. Both devices are equipped with WIFI wireless communication. One of the devices is set as a WIFI hotspot using Hostapd, setting the network name, encryption method, password, permissions, etc., creating a TCP server Socket, binding it to a static IP address and port, and listening for connection requests. The other device connects to the hotspot of the system and the specified IP address and port to achieve data transmission and interaction.

[0164] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for judging the verticality of a vertical pole by using two machines, characterized in that, It includes the following steps: S1. Use two verticality judgment devices to perform real-time image acquisition on the rod body from two directions with an included angle of nearly 90 degrees through their respective USB cameras, and at the same time obtain the roll angle of the corresponding device of the camera through the inertial measurement unit to obtain image data and roll angle data; S2. Transmit the image data and roll angle data at the same moment to the embedded computing platform; S3. Run a deep neural network on the embedded computing platform to perform instance segmentation on the image, extract the rod body area and output a binary image; S4. Perform a row scan on the rod body pixels in the binary image to obtain the midpoint of the axial pixels, and obtain the axis of the rod body through a fitting algorithm; S5. Calculate the tilt angle of the rod body in the image coordinate system based on the fitting algorithm and convert it to the tilt angle relative to the reverse direction of the Y-axis; S6. Perform angle compensation in combination with the roll angle to obtain the tilt angle of the rod body in the device coordinate system; S7. Compare the tilt angle values of the two verticality judgment devices. When both are zero, it is determined that the rod body is vertical.

2. The double-machine plumb judgment method for the vertical pole according to claim 1, wherein The deep neural network is a lightweight improved YOLOv8s-seg model, and the YOLOv8s-seg model is used to perform instance segmentation on the rod body image and output a binary mask image of the rod body area.

3. A method for judging the verticality of a vertical pole by two machines according to claim 1, characterized in that, The embedded computing platform is a Jetson nano embedded computing platform, which is used to receive image and roll angle data and perform deep neural network instance segmentation and rod body axis fitting calculation.

4. A method for judging the verticality of a vertical pole by two machines according to claim 1, characterized in that, The angle compensation adds the tilt angle calculated from the image to the roll angle measured by the inertial measurement unit to obtain the compensated tilt angle, where the roll angle is positive when tilted to the right and negative when tilted to the left, that is: α = θ + β; Where, α is the angle compensation; θ is the tilt angle; β is the axis tilt angle.

5. A method for judging the verticality of a vertical pole with two machines according to claim 1, characterized in that, The two verticality judgment devices transmit tilt angle data and image processing results to each other through a wireless communication method for the main control device to comprehensively judge whether the rod body is vertical.

6. A method for judging the verticality of a vertical pole with two machines according to claim 1, characterized in that, Each verticality judgment device includes a USB camera, an inertial measurement unit, a Jetson nano embedded computing platform and a touch display module, and the touch display module is used to display images and the calculated tilt angle of the rod body.

7. A verticality judging device for a vertical pole with two machines, which is applied to the verticality judging method for a vertical pole with two machines according to any one of claims 1-6, and comprises a triangular bracket (7), and is characterized in that, The top of the triangular support (7) is provided with a load-carrying round table (6), the top of the load-carrying round table (6) is provided with a device box housing (5), the inner bottom wall of the device box housing (5) is horizontally provided with a Jetson nano embedded computing platform (1) and an inertial measurement unit (2) respectively, and the inertial measurement unit (2) is used to measure the roll angle data of the bottom surface of the box in real time. The inner walls of the device box housing (5) are respectively provided with a USB camera (3) and a touch liquid crystal screen (4), and the USB camera (3) and the touch liquid crystal screen (4) are symmetrically distributed.

8. A verticality determination device for double machines on a vertical pole according to claim 7, characterized in that, The Jetson nano embedded computing platform (1) is connected to the control signal transmission ends of the inertial measurement unit (2), the USB camera (3) and the touch liquid crystal screen (4) through a USB cable, and the Jetson nano embedded computing platform (1) is connected to the touch liquid crystal screen (4) through an HDMI cable to send an image signal to it.

9. A verticality judgment device for double machines on a vertical pole according to claim 7, characterized in that, The USB camera (3) faces the rod body to collect an image of the rod body.

10. A verticality determination device for two machines on a vertical pole according to claim 7, characterized in that, The touch liquid crystal screen (4) faces the operator to display the captured image of the rod body and the rod body tilt parameter information.

Images