A machine vision-based method and system for supervising a tower erection process

Abstract

The application discloses a kind of tower erection process supervision method and system based on machine vision, it is related to tower monitoring technical field, including historical capture position acquisition module, reference capture position range acquisition module, robot dog platform, image acquisition processing module, image acquisition identification module and analysis alarm module;Through illegal signal, it shows that the mechanical gripper is in the process of capturing construction components, serious deviation of spatial position and attitude angle occurs simultaneously, immediate stop operation and comprehensive inspection and correction are needed, errors of mechanical gripper in the process of tower erection can be found and corrected in time, the accuracy, safety and quality of construction are guaranteed, and it helps to prevent construction accidents caused by improper attitude, such as component falling caused by incorrect attitude of gripper, which may cause harm to on-site personnel and equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tower monitoring, in particular to a tower erection process supervision method and system based on machine vision. BACKGROUND

[0002] Tower erection refers to installing towers (including power poles and iron towers) at predetermined positions according to design requirements through specific methods and processes to support the conductors and ground wires of a power transmission line. In modern power transmission line construction, tower erection is a key and complex engineering and technical task, whether it is overall erection or disassembly and erection, which provides a solid foundation for the stable operation of the power system.

[0003] However, during the tower erection process, it is difficult to discover the deviation of the spatial position of the mechanical gripper when it grabs the construction components in time. For example, if the grabbing position of the mechanical gripper deviates from the reference range too much, it may lead to inaccurate installation of the components, affecting the overall structural stability of the tower. For example, when building a power tower, if the grabbing position of a key component is too high or too low, it may lead to difficulties in subsequent assembly. Therefore, a tower erection process supervision method and system based on machine vision are proposed. SUMMARY

[0004] The purpose of the present application is to provide a tower erection process supervision method and system based on machine vision, which solves the technical problem of being unable to discover the deviation of the spatial position of the mechanical gripper when it grabs the construction components in time during the tower erection process.

[0005] The purpose of the present application can be achieved through the following technical solutions:

[0006] A tower erection process supervision system based on machine vision, comprising:

[0007] A historical grabbing position acquisition module records the attitude angle and spatial coordinates of the mechanical gripper when it grabs each component of the tower.

[0008] A reference grabbing position range acquisition module analyzes the attitude angle and spatial coordinates of the mechanical gripper when it grabs each component of the tower to obtain the corresponding reference grabbing position range of each component of the tower.

[0009] A robot dog platform performs omnidirectional inspection of the tower erection site by using a robot dog as a mobile carrier.

[0010] An image acquisition and processing module processes the parameters of image shooting of the robot dog.

[0011] An image acquisition and recognition module acquires the spatial coordinates and attitude angle of the mechanical gripper when it grabs the construction components in real time.

[0012] The analysis alarm module obtains the spatial coordinates and the attitude angle of the mechanical gripper when the mechanical gripper is in real-time grabbing of the construction assembly, compares and analyzes the corresponding reference grabbing position range of the construction assembly, and generates a space warning signal, an attitude warning signal, or a violation signal according to the comparison and analysis result.

[0013] As a further scheme of the present application, the reference grabbing position range corresponding to each component of the tower is obtained in the following specific manner:

[0014] S1: randomly selecting one component from the components of the tower as a target component;

[0015] S2: obtaining the spatial coordinates Dn and the attitude angle En of the mechanical gripper when the mechanical gripper is in grabbing of the target component for a preset number n of times, calculating the standard deviation U of the horizontal coordinates Xn in the spatial coordinates when the mechanical gripper is in grabbing of the target component for the preset number n of times, analyzing the standard deviation U, obtaining the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component according to the analysis result, and generating the reference grabbing range AX1[BX-CX, BX+CX] corresponding to the horizontal coordinate of the target component according to the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component and the threshold value CX, wherein the threshold value CX is the absolute value of the maximum value and the minimum value of the horizontal coordinates Xn, and n is the serial number of the grabbing number, n≥1;

[0016] S3: repeating steps S1-S2, so as to obtain the reference grabbing ranges AXj, AYj, AZj, Aαj, Aβj, and Aγj corresponding to each parameter in the spatial coordinates and the attitude angle corresponding to each component of the tower, wherein j represents different components of the tower, and j is a positive integer, j≥1.

[0017] As a further scheme of the present application, the specific manner of analyzing the standard deviation U is as follows:

[0018] When the standard deviation is less than a preset threshold Q1, Xp is taken as the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component, and when the standard deviation is greater than or equal to the preset threshold Q1, the mean value of the maximum value and the minimum value of the horizontal coordinates Xn is taken as the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component, wherein Xp is the mean of the horizontal coordinates Xn.

[0019] As a further scheme of the present application, the specific manner of generating the space warning signal, the attitude warning signal, or the violation signal is as follows:

[0020] The spatial coordinates and attitude angles of the mechanical gripper when grasping the construction component in real time are marked as R(RX,RY,RZ) and T(Tα,Tβ,Tγ), respectively. The reference grasping position range corresponding to the construction component is obtained from the reference grasping position range acquisition module and marked as FX, FY, FZ, Fα, Fβ, and Fγ, respectively. The number of parameters of the spatial coordinates of the construction component that do not belong to their corresponding reference grasping position range is marked as e, and the number of parameters of the attitude angles of the construction component that do not belong to their corresponding reference grasping position is marked as r. When e is greater than the preset value Q1, a spatial warning signal is generated; when r is greater than the preset value Q2, an attitude warning signal is generated; otherwise, no action is taken.

[0021] When both space warning signals and attitude warning signals are generated simultaneously, a violation signal is generated and output; when only a space warning signal or an attitude warning signal is generated, the corresponding signal is output.

[0022] As a further aspect of the present invention, the specific method for processing the images captured by the robot dog is as follows:

[0023] The robot dog uses an integrated high-magnification fixed-focus digital camera to automatically adjust the focal length and exposure parameters to capture clear images of the tower erection process.

[0024] As a further aspect of the present invention: a grayscale correction method is used to preprocess the acquired images to remove noise and enhance contrast. At the same time, image segmentation technology is used to remove messy and useless background information and retain images of key construction areas.

[0025] A machine vision-based method for supervising the pole erection process, which is executed by a machine vision-based pole erection process supervision system, specifically includes the following steps:

[0026] Step 1: Record the posture angle and spatial coordinates of the mechanical gripper when it grasps each component of the tower;

[0027] Step 2: Obtain the reference gripping position range corresponding to each component of the tower;

[0028] Step 3: Conduct a comprehensive inspection of the tower erection site using a robot dog;

[0029] Step 4: Process the parameters used to capture images of the robot dog;

[0030] Step 5: Obtain the spatial coordinates and attitude angles of the mechanical gripper when it grasps the construction components in real time;

[0031] Step 6: Generate space warning signals, attitude warning signals, or violation signals.

[0032] Advantages of the present application:

[0033] (1) The present application helps to find the deviation of the spatial position of the mechanical gripper when grabbing the construction assembly in time through the spatial warning signal, reminds the operator to take measures to adjust, avoids the accumulation of spatial position deviation to cause greater problems, such as correcting the position of the gripper in time to ensure the accuracy of subsequent grabbing, helps to protect the construction quality and safety, if the spatial deviation is not found and corrected in time, it may lead to abnormal stress on the tower in use, increase the risk of collapse.

[0034] (2) The present application can prompt the abnormality of the attitude angle of the mechanical gripper when grabbing the construction assembly through the attitude warning signal, such as the too large inclination angle of the gripper, which may affect the docking accuracy of the assembly, prompting the operator to pay attention to the attitude of the gripper and adjust in time to ensure the accuracy and stability of the assembly during installation.

[0035] (3) The present application indicates that the mechanical gripper has serious deviation of spatial position and attitude angle when grabbing the construction assembly, which is a very serious situation that needs immediate attention; means that there may be a big mistake or risk in the construction process, which needs to be stopped immediately and checked and corrected comprehensively, providing an important basis for construction management, facilitating the analysis and improvement of similar problems to avoid recurrence; can find and correct the error of the mechanical gripper in the tower assembly process in time, ensure the accuracy, safety and quality of construction, and help to prevent construction accidents caused by improper attitude, such as assembly falling caused by incorrect attitude of the gripper, which may cause harm to the on-site personnel and equipment. BRIEF DESCRIPTION OF DRAWINGS

[0036] The present application will be further described below in conjunction with the drawings.

[0037] Figure 1 is a schematic diagram of the system framework structure of the present application;

[0038] Figure 2 is a schematic diagram of the method framework structure of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] Embodiment one

[0041] Please refer toFigure 1 The present application is a machine vision-based tower assembly process supervision system, comprising:

[0042] A historical grabbing position acquisition module acquires historical grabbing positions of the mechanical grabber for each component of the tower. Specifically, a global coordinate system is established in the working space, and all grabbing positions are recorded relative to the coordinate system. The global coordinate system is usually consistent with the base coordinate system of the mechanical arm. Position sensors (such as encoders) and vision systems (such as stereo cameras and laser scanners) are used to record the position and posture of the mechanical grabber. The data provided by these sensors can be used to calculate the specific position of the grabber in the global or local coordinate system. Each grabbing position not only records the spatial coordinates, but also records the posture angle to completely describe its position and direction in space.

[0043] A reference grabbing position range acquisition module acquires and analyzes the historical grabbing positions of the mechanical grabber for each component of the tower to obtain the corresponding reference grabbing position range of each component of the tower. Specifically:

[0044] S1: Randomly select one component from the components of the tower as the target component;

[0045] S2: Obtain the spatial coordinates Dn(Xn, Yn, Zn) and the posture angle En(αn, βn, γn) of the mechanical grabber when grabbing the target component for a preset number of times n, where n is the serial number of the grabbing times, and n≥1;

[0046] By Calculate the standard deviation U of the horizontal coordinate Xn in the spatial coordinates when the mechanical grabber grabs the target component of the tower for a preset number of times n, where n≥a≥1, and Xp is the mean of the horizontal coordinate Xn. When the standard deviation is less than a preset threshold Q1, Xp is taken as the reference horizontal coordinate BX corresponding to the spatial horizontal coordinate of the target component. When the standard deviation is greater than or equal to the preset threshold Q1, the mean of the maximum and minimum values of the horizontal coordinate Xn is taken as the reference horizontal coordinate BX corresponding to the spatial horizontal coordinate of the target component, i.e. BX=(Xmin+Xmax) / 2. The threshold value CX corresponding to the horizontal coordinate is calculated by CX=|Xmin-Xmax|, and the specific value of Q1 is determined by relevant personnel according to experience and actual application requirements.

[0047] Generate the reference grabbing range AX1[BX-CX, BX+CX] corresponding to the spatial horizontal coordinate of the target component according to the reference horizontal coordinate BX corresponding to the spatial horizontal coordinate of the target component and the threshold value CX.

[0048] S3: In the same way as the target component space horizontal coordinate corresponding to the reference grabbing range, each parameter in the space coordinates Dn(Xn, Yn, Zn) and the attitude angle En(αn, βn, γn) is analyzed one by one, and then the reference grabbing range AX1, AY1, AZ1, Aα1, Aβ1 and Aγ1 corresponding to each parameter in the space coordinates Dn(Xn, Yn, Zn) and the attitude angle En(αn, βn, γn) is obtained;

[0049] S4: Repeat steps S1-S3, that is, the reference grabbing range AXj, AYj, AZj, Aαj, Aβj and Aγj corresponding to each parameter in the space coordinates and the attitude angle of each component of the tower is obtained, wherein j represents different components of the tower, j is a positive integer, j≥1;

[0050] The robot dog platform, through the robot dog as a mobile carrier, carries a high-power fixed-focus digital camera, and records the position and attitude of the mechanical gripper in real time through the position sensor and vision system, and conducts omnidirectional inspection on the tower assembly site;

[0051] It should be noted that the robot dog can move flexibly and adapt to complex terrain environment to ensure comprehensive coverage of the tower assembly work area. The action path of the robot dog can be pre-set or adjusted in real time according to the site construction situation;

[0052] The image acquisition processing module, through the integrated high-power fixed-focus digital camera set on the robot dog, can automatically adjust the focal length and exposure parameters, capture clear images of the tower assembly process, and the camera has a built-in image stabilization system to ensure stable image quality during movement;

[0053] For example, when the mechanical gripper assembles each component of the tower, it can quickly capture clear images of the process. In order to obtain the best image quality, the camera will also automatically adjust the focus. Whether it is close-up assembly details or long-distance overall scene, it can be clearly imaged. In strong light environment, the camera can automatically reduce the exposure to avoid image overexposure; in weak light environment, the camera will automatically enhance the light sensitivity to ensure the clarity of the image;

[0054] At the same time, the gray scale correction method is used to preprocess the collected images to remove noise and enhance contrast. At the same time, image segmentation technology is used to remove useless background information and retain key construction area images. The above are existing and mature technologies, so no further description is made here;

[0055] For example, if the image is partially dark or bright due to uneven lighting, the gray scale correction method can balance the gray scale distribution of the image, making the tower components and mechanical gripper construction actions in the image more clear and identifiable, while effectively filtering out irrelevant image content such as background clutter, temporarily placed tools, and other clutter. For image blurring caused by weather conditions such as fog and dust, the visual correction module can also perform de-fogging and de-dusting to restore the true construction scene.

[0056] The image acquisition and recognition module identifies the tower components grasped by the mechanical gripper in the image, identifies the real-time construction components grasped by the mechanical gripper, and obtains the spatial coordinates and attitude angles of the mechanical gripper during real-time grasping of the construction components.

[0057] The analysis and alarm module compares the spatial coordinates and attitude angles of the mechanical gripper during real-time grasping of the construction components with the corresponding reference grasping position range of the construction components, and generates a spatial warning signal, an attitude warning signal, or a violation signal based on the comparison results. The specific method is as follows:

[0058] The spatial coordinates and attitude angles of the mechanical gripper during real-time grasping of the construction components are marked as R(RX, RY, RZ) and T(Tα, Tβ, Tγ), respectively. The reference grasping position range corresponding to the construction component is obtained from the reference grasping position range acquisition module, and is marked as FX, FY, FZ, Fα, Fβ, and Fγ, respectively. Each parameter RX, RY, RZ, Tα, Tβ, and Tγ in the spatial coordinates and attitude angles is compared with the corresponding reference grasping position range one by one. The number of parameters in the spatial coordinates of the construction component that do not belong to the corresponding reference grasping position range FX, FY, and FZ is obtained and marked as e. The number of parameters in the attitude angle of the construction component that do not belong to the corresponding reference grasping position range Fα, Fβ, and Fγ is obtained and marked as r.

[0059] When e is greater than a preset value Q1, a spatial warning signal is generated, otherwise no action is taken. When r is greater than a preset value Q2, an attitude warning signal is generated, otherwise no action is taken. The preset values Q1 and Q2 are determined by relevant personnel according to actual needs.

[0060] When both the spatial warning signal and the attitude warning signal are generated, a violation signal is generated and outputted. When only the spatial warning signal or the attitude warning signal is generated, the corresponding signal is outputted.

[0061] The spatial pre-warning signal helps to timely find the deviation of the mechanical gripper in the spatial position when grabbing the construction component, reminds the operator to take measures to adjust, avoids the accumulation of spatial position deviation and causes greater problems, such as timely correcting the position of the gripper, ensuring the accuracy of subsequent grabbing, helping to protect the construction quality and safety, if the spatial deviation is not timely discovered and corrected, it may cause the tower to bear abnormal stress during use, increasing the risk of collapse.

[0062] Through the posture pre-warning signal, the abnormality of the posture angle of the mechanical gripper when grabbing the construction component can be prompted, such as the too large inclination angle of the gripper, which may affect the docking accuracy of the component, prompting the operator to pay attention to the posture of the gripper and timely adjust to ensure the accuracy and stability of the component during installation.

[0063] Through the violation signal, it is indicated that the mechanical gripper has serious deviation in spatial position and posture angle when grabbing the construction component, which is a very serious situation that needs immediate attention; means that there may be a big mistake or risk in the construction process, which needs to be stopped immediately and checked and corrected comprehensively, such as the simultaneous violation of space and posture when assembling important tower key nodes, which may cause major hidden dangers to the entire tower structure; can timely find and correct the errors of the mechanical gripper during the tower erection process, ensure the accuracy, safety and quality of the construction, and help prevent construction accidents caused by improper posture, such as component falling caused by incorrect posture of the gripper, which may cause harm to on-site personnel and equipment.

[0064] Embodiment two

[0065] Please refer to Figure 2 The embodiment also provides a tower erection process supervision method based on machine vision, which is implemented by the above-mentioned tower erection process supervision system based on machine vision, and specifically includes the following steps:

[0066] Step one: record the posture angle and spatial coordinates of the mechanical gripper when grabbing each component of the tower;

[0067] Step two: obtain the reference grabbing position range corresponding to each component of the tower respectively;

[0068] Step three: use the mechanical dog as a mobile carrier, carry a high-magnification fixed-focus digital camera, and use the position sensor and vision system to record the position and posture of the mechanical gripper in real time, and conduct comprehensive inspection of the tower erection site;

[0069] Step four: through the integrated high-power focusing digital camera set on the mechanical dog, the focus and exposure parameters can be automatically adjusted, clear images of the tower assembly process can be captured, the image stabilization system is built-in the camera, the image quality is stable during the movement, the parameters of the image shooting of the mechanical dog are processed

[0070] Step five: the tower assembly grabbed by the mechanical gripper is identified, the construction assembly grabbed by the mechanical gripper in real time is identified, and the spatial coordinates and attitude angle of the mechanical gripper when grabbing the construction assembly in real time are obtained.

[0071] Step six: the spatial coordinates and attitude angle of the mechanical gripper when grabbing the construction assembly in real time are compared and analyzed with the reference grabbing position range corresponding to the construction assembly, and the space warning signal, the attitude warning signal or the violation signal is generated according to the comparison and analysis result.

[0072] Embodiment three

[0073] As an embodiment of the present application, compared with embodiment one and embodiment two, the technical scheme of the present embodiment is to combine the schemes of embodiment one and embodiment two.

[0074] The above formulas are dimensionless numerical calculations, the formulas are obtained by software simulation of a large amount of data to obtain the latest real situation, and the preset parameters and threshold values in the formula are set by the person skilled in the art according to the actual situation.

[0075] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A machine vision-based monitoring system for the tower erection process, characterized in that, include; The historical gripping position acquisition module records the posture angle and spatial coordinates of the mechanical gripper when gripping various components of the tower; The reference gripping position range acquisition module analyzes the attitude angles and spatial coordinates of the mechanical gripper on each component of the tower to obtain the reference gripping position range corresponding to each component of the tower. The robot dog platform uses robot dogs as mobile carriers to conduct comprehensive inspections of the tower erection site. The image acquisition and processing module processes the parameters for capturing images of the robot dog; The image acquisition and recognition module acquires the spatial coordinates and posture angles of the mechanical gripper when it grasps the construction components in real time. The analysis and alarm module obtains the spatial coordinates and attitude angles of the mechanical gripper when it grasps the construction components in real time, compares and analyzes them with the reference grasping position range corresponding to the construction components, and generates spatial warning signals, attitude warning signals or violation signals based on the comparison and analysis results. The specific method for obtaining the reference grab position range corresponding to each component of the tower is as follows: S1: Randomly select one of the various components of the tower as the target component; S2: Obtain the spatial coordinates Dn and attitude angle En of the mechanical gripper when gripping the target component in a preset number of n times. Calculate the standard deviation U of the horizontal coordinate Xn in the spatial coordinates of the mechanical gripper when gripping the target component of the tower in a preset number of n times. Analyze the standard deviation U and obtain the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component based on the analysis results. At the same time, generate the reference gripping range AX1[BX-CX, BX+CX] corresponding to the horizontal coordinate of the target component based on the reference horizontal coordinate BX and the threshold value CX. The threshold value CX is the absolute value of the maximum and minimum values ​​in the horizontal coordinate Xn, where n is the sequence number of gripping times, and n≥1. S3: Repeat steps S1-S2 to obtain the reference capture ranges AXj, AYj, AZj, Aαj, Aβj and Aγj corresponding to the parameters in the spatial coordinates and attitude angles of each component of the tower, where j refers to different components of the tower, j is a positive integer, j≥1.

2. The tower erection process monitoring system based on machine vision according to claim 1, characterized in that, The specific method for analyzing the standard deviation U is as follows: When the standard deviation is less than the preset threshold Q1, Xp is used as the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component space. When the standard deviation is greater than or equal to the preset threshold Q1, the mean of the maximum and minimum values ​​of the horizontal coordinate Xn is used as the reference horizontal coordinate BX corresponding to the horizontal coordinate of the target component space, where Xp is the mean of the horizontal coordinate Xn.

3. The tower erection process monitoring system based on machine vision according to claim 2, characterized in that, The specific methods for generating space warning signals, attitude warning signals, or violation signals are as follows: The spatial coordinates and attitude angles of the mechanical gripper when grasping the construction component in real time are marked as R(RX,RY,RZ) and T(Tα,Tβ,Tγ), respectively. The reference grasping position range corresponding to the construction component is obtained from the reference grasping position range acquisition module and marked as FX, FY, FZ, Fα, Fβ, and Fγ, respectively. The number of parameters of the spatial coordinates of the construction component that do not belong to their corresponding reference grasping position range is marked as e, and the number of parameters of the attitude angles of the construction component that do not belong to their corresponding reference grasping position is marked as r. When e is greater than the preset value Q1, a spatial warning signal is generated; when r is greater than the preset value Q2, an attitude warning signal is generated; otherwise, no action is taken. When both space warning signals and attitude warning signals are generated simultaneously, a violation signal is generated and output; when only a space warning signal or an attitude warning signal is generated, the corresponding signal is output.

4. The tower erection process monitoring system based on machine vision according to claim 1, characterized in that, The specific method for processing the images captured by the robot dog is as follows: The robot dog uses an integrated high-magnification fixed-focus digital camera to automatically adjust the focal length and exposure parameters to capture clear images of the tower erection process.

5. The tower erection process monitoring system based on machine vision according to claim 4, characterized in that, The grayscale correction method is used to preprocess the acquired images to remove noise and enhance contrast. At the same time, image segmentation technology is used to remove cluttered and useless background information and retain images of key construction areas.

6. A method for supervising the tower erection process based on machine vision according to any one of claims 1-5, characterized in that, This method is executed through a machine vision-based tower erection process monitoring system, and specifically includes the following steps: Step 1: Record the posture angle and spatial coordinates of the mechanical gripper when it grasps each component of the tower; Step 2: Obtain the reference gripping position range corresponding to each component of the tower; Step 3: Conduct a comprehensive inspection of the tower erection site using a robot dog; Step 4: Process the parameters used to capture images of the robot dog; Step 5: Obtain the spatial coordinates and attitude angles of the mechanical gripper when it grasps the construction components in real time; Step 6: Generate space warning signals, attitude warning signals, or violation signals.

Images