Visual inspection and path planning method for intelligent pasting robot for duct piece waterproof material
Through visual inspection and path planning methods, the problem of insufficient work efficiency and human labor pressure in the rail transit construction industry is solved, and the precise movement and efficient operation of the intelligent sticking robot of pipe sheet waterproof material is realized, which enhances its adaptability and learning ability.
Patent Information
- Application Number
- CN202411142526.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
There is little research on intelligent equipment manufacturing and intelligent level management of construction robots in the rail transit construction industry, resulting in insufficient work efficiency and human labor pressure for intelligent sticking robots for pipe sheet waterproof materials.
Visual detection and path planning methods are adopted, including grayscale processing, noise reduction processing, binary processing, filter feature extraction and extended Kalman filter positioning, combined with path planning algorithms, to realize the precise movement of the intelligent pasting robot of pipe sheet waterproof material .
The work efficiency of the intelligent sticking robot for pipe sheet waterproof materials is improved, the pressure of human labor is reduced, and its adaptability and learning ability is enhanced, achieving precise continuous homework.
Smart Images

Figure CN119088007A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a method for visual detection and path planning of a robot for intelligent pasting of pipe segment waterproof materials. Background Art
[0002] The development of the construction robot industry started late, and both research and development and application fields are in the initial stage. The types of construction robots put into use are also limited, and the degree of popularization is still very low. At present, there is little research on intelligent equipment manufacturing and intelligent level management in the rail transit construction industry. As part of the intelligent operation of the subway, the intelligent pasting robot for pipe segment waterproofing materials is expected to further improve the application field of robots in the construction industry, fill the gap in the industry, meet the needs of market development, improve production quality, and promote industrial production efficiency.
[0003] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention
[0004] The purpose of the present invention is to provide a method for visual inspection and path planning of an intelligent robot for pasting waterproof materials of pipe segments, so as to improve work efficiency and reduce the pressure of human labor.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] The present invention provides a method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials, which specifically comprises the following steps:
[0007] S1: First, the intelligent pasting robot for pipe segment waterproofing materials takes the surrounding environment captured by the camera as the input of the controller, performs grayscale processing on its input image, and for color images, grayscales the red component of each pixel. , Green component and blue component Take the weighted average respectively to get the corresponding gray value ;
[0008] S2: Perform denoising on the input image to effectively reduce the noise in the image and improve the visual quality of the image, enhancing the details and contrast of the image;
[0009] S3: Binarize the image and convert it into a binary image containing only two pixel values, so as to more easily detect the edge and contour of the target, thereby achieving more accurate and reliable target detection and recognition;
[0010] S4: After processing the image, use The filter extracts features from the image to capture the multi-scale and multi-directional texture features in the image;
[0011] S5: Use extended Kalman filter for segment positioning operation to perform state estimation and update in a real-time environment, thereby providing real-time positioning results with lower computational complexity;
[0012] S6: The final segment positioning result can be obtained through the state estimation value of the filter, which is represented by the position, speed and direction of the target;
[0013] S7: Realize the movement of the intelligent segment waterproof material pasting robot through the path planning algorithm;
[0014] S8: Randomly sample a point ; This point is randomly generated from the starting point toward the target point; iterate in this way until the target point is found; this will generate a node tree, and according to the currently generated tree structure, find the node closest to the random sampling point from the tree as the nearest neighbor node;
[0015] S9: Determine the correct node among the number of nodes so as to obtain the optimal path;
[0016] S10: After traversing the entire number of nodes, a path with the correct direction and the shortest distance is generated. Then the controller drives the chassis to move along the planned path, and moves the intelligent segment waterproofing material pasting robot to the top of the segment to be operated. After completing the current operation, the current position, that is, the nearest segment position, is used as the starting position. Repeat the above steps for the next operation.
[0017] Furthermore, the gray value The calculation formula is as follows:
[0018] .
[0019] Furthermore, in step S2, a mean filter is used to reduce noise on the input image. The image noise reduction process is implemented by the following formula:
[0020]
[0021] in is the pixel value after noise reduction processing, , are the original pixel values at different points.
[0022] Furthermore, in step S3, the image is binarized using the global threshold method, and the global threshold is defined. ,The binary processing calculation formula is as follows:
[0023]
[0024] in, is the pixel value after noise reduction processing, The pixel corresponds to white, The pixel corresponds to black.
[0025] Further, in step S4, The calculation method of the filter to extract features from the image is as follows:
[0026]
[0027] in for The filter response, represents the coordinates on the image, and represents the rotated and scaled coordinates, represents ellipticity, Indicates scale, Indicates frequency, Indicates phase; The filter uses multiple filters of different scales and directions. function, applying them to each pixel position of the input image, where different scales are obtained by Adjustment, direction through To adjust; for each scale and direction, the filter output texture response represents the texture characteristics of the input image at that scale and direction.
[0028] Further, the specific process of using the extended Kalman filter to perform segment positioning operation in step S5 is as follows:
[0029] First, at each time step, the state model is used to predict and estimate the state and covariance matrix at the next moment. The calculation formula is as follows:
[0030]
[0031]
[0032] in, Indicates at time The estimated value of the state; Represents the state transfer function, which transforms the state estimation and control input of the previous moment into Mapping to the state estimate at the current moment;
[0033] Indicates at time The state covariance matrix of Represents the state transition function The Jacobian matrix of is used to linearize the nonlinear state transfer function; represents the process noise covariance matrix, which represents the uncertainty or noise in the state transition process;
[0034] After the prediction, the observation model is used to compare the sensor measurement with the predicted state, and its state estimate and covariance matrix are updated. The calculation formula is as follows:
[0035] Calculate the innovation covariance:
[0036]
[0037] Calculate the Kalman gain:
[0038]
[0039] Computing Innovation:
[0040]
[0041] Update the state estimate:
[0042]
[0043] Update the covariance matrix:
[0044]
[0045] in Indicates at time Observed value of
[0046] represents the observation model function, which maps the current state estimate to the observation space;
[0047] Represents the observation model function The Jacobian matrix of is used to linearize the nonlinear observation model;
[0048] represents the observation noise covariance matrix, which represents the uncertainty or noise in the observation process;
[0049] represents innovation, i.e., the difference between the observed and predicted observations;
[0050] represents the innovation covariance matrix, which measures the uncertainty of observation noise and forecast error;
[0051] represents the Kalman gain, which is used to update the state estimate and covariance matrix;
[0052] represents the identity matrix;
[0053] Repeat the prediction and update steps to iteratively update the state estimate based on new sensor measurements and model predictions; each iteration updates the state estimate and covariance matrix, thereby improving positioning accuracy;
[0054] The above extended Kalman filter positioning algorithm is recursive and can perform state estimation and update in a real-time environment, providing real-time positioning results with lower computational complexity.
[0055] Furthermore, the specific process of realizing the movement of the intelligent waterproof material pasting robot for pipe segments by means of the path planning algorithm in step S7 is as follows:
[0056] First, the starting point and the target point are initialized, and the position of the segment intelligent waterproof material pasting robot is defined as the coordinate origin, that is, the starting position ; The detected segment position is used as the target position ; This initialization is used to clarify the starting point and the target point, so that the algorithm can start from the starting point and search towards the target point.
[0057] Further, in step S9, the specific method for determining the correct node in the number of nodes is as follows: define the nearest node as ; Define vector ; The orientation calculation formula is as follows: ;in, Representation vector The module length, For the current node To target point direction; in this way, the nodes with the correct direction can be obtained in the number of nodes, so as to obtain the optimal path.
[0058] By adopting the above technical solution, the present invention has the following beneficial effects:
[0059] The present invention can improve work efficiency, and the intelligent pasting robot for pipe segment waterproofing materials can perform operations accurately and continuously. Compared with manual operations, the invention greatly improves work efficiency and reduces the pressure of human labor. The intelligent pasting robot for pipe segment waterproofing materials can also enhance its adaptability and learning ability during continuous operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0061] Figure 1 It is a flow chart of the visual positioning part of the method of the present invention;
[0062] Figure 2 is a flow chart of path planning of the method of the present invention;
[0063] Figure 3 It is a flow chart of the overall scheme of the method of the present invention. DETAILED DESCRIPTION
[0064] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0065] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.
[0066] In this example, the pipe segment waterproof material intelligent pasting robot is equipped with four 270° panoramic cameras around it to achieve real-time environmental photography. The present invention only describes the visual algorithm and path planning, and does not involve the specific structure of the pipe segment waterproof material intelligent pasting robot.
[0067] Combination Figure 1-3 As shown, this embodiment provides a method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials, which specifically includes the following steps:
[0068] S1: First, the intelligent pasting robot for pipe segment waterproofing materials takes the surrounding environment captured by the camera as the input of the controller, and performs grayscale processing on its input image. For color images, the red component of each pixel can be grayscaled. , Green component and blue component Take the weighted average respectively to get the corresponding gray value The calculation formula is as follows:
[0069] .
[0070] S2: The input image is subjected to noise reduction processing. The present invention uses a mean filter for noise reduction. In the mean filter, the value of each pixel is the average value of the pixels in its field. The noise reduction processing of the image is achieved by the following formula:
[0071] ;
[0072] in is the pixel value after noise reduction processing, , is the original pixel value at different points. After filtering and denoising, the noise in the image can be effectively reduced, the visual quality of the image can be improved, and the details and contrast of the image can be enhanced.
[0073] S3: Binarize the image to convert it into a binary image containing only two pixel values. This example uses the global threshold method for processing. This case is used for the pipe segment intelligent waterproof material pasting robot to locate the pipe segment position through vision. It does not require a lot of details and texture information, so a lower global threshold can be selected. This case defines the global threshold ,The binary processing calculation formula is as follows:
[0074] ;
[0075] in is the pixel value after noise reduction processing, The pixel corresponds to white, Pixels correspond to black. Binarization of the image can provide a better basis for target detection and recognition, making it easier to detect the edges and contours of the target, thereby achieving more accurate and reliable target detection and recognition.
[0076] S4: After processing the image, use The filter extracts features from the image and the calculation method is as follows:
[0077]
[0078] in for The filter response, represents the coordinates on the image, and represents the rotated and scaled coordinates, represents ellipticity, Indicates scale, Indicates frequency, Indicates phase.
[0079] The filter uses multiple filters of different scales and directions. function, applying them to each pixel position of the input image, where different scales are obtained by Adjustment, direction through For each scale and direction, the filter output texture response represents the texture characteristics of the input image at that scale and direction. Filters can capture multi-scale and multi-directional texture features in images.
[0080] S5: After image processing, this example uses the extended Kalman filter to perform segment positioning operations. First, in each time step, the state model is used to predict and estimate the state and covariance matrix at the next moment. The calculation formula is as follows:
[0081] ;
[0082] ;
[0083] in, Indicates at time The estimated value of the state; Represents the state transfer function, which transforms the state estimation and control input of the previous moment into Mapping to the state estimate at the current moment; Indicates at time The state covariance matrix of Represents the state transition function The Jacobian matrix of is used to linearize the nonlinear state transfer function; represents the process noise covariance matrix, which represents the uncertainty or noise in the state transition process.
[0084] After the prediction, the observation model is used to compare the sensor measurement with the predicted state, and its state estimate and covariance matrix are updated. The calculation formula is as follows:
[0085] Calculate the innovation covariance:
[0086]
[0087] Calculate the Kalman gain:
[0088]
[0089] Computing Innovation:
[0090]
[0091] Update the state estimate:
[0092]
[0093] Update the covariance matrix:
[0094]
[0095] in Indicates at time Observed value of
[0096] represents the observation model function, which maps the current state estimate to the observation space;
[0097] Represents the observation model function The Jacobian matrix of is used to linearize the nonlinear observation model;
[0098] represents the observation noise covariance matrix, which represents the uncertainty or noise in the observation process;
[0099] represents innovation, i.e., the difference between the observed and predicted observations;
[0100] represents the innovation covariance matrix, which measures the uncertainty of observation noise and forecast error;
[0101] represents the Kalman gain, which is used to update the state estimate and covariance matrix;
[0102] Represents the identity matrix.
[0103] The prediction and update steps are repeated to iteratively update the state estimate based on new sensor measurements and model predictions. Each iteration updates the state estimate and the covariance matrix, thereby improving localization accuracy.
[0104] The above extended Kalman filter positioning algorithm is recursive and can perform state estimation and update in a real-time environment, and can provide real-time positioning results with low computational complexity.
[0105] S6: The final positioning result of the segment can be obtained through the state estimation value of the filter, which can be expressed as the position, speed, and direction of the target. However, in this example, the segment is stationary and only its two-dimensional position needs to be obtained. Subsequently, the segment waterproofing material intelligent pasting robot will be driven to the top of the segment for operation through the path planning algorithm.
[0106] S7: Specifically, the path planning algorithm will be used to realize the movement of the intelligent pasting robot for pipe segment waterproofing materials. First, the starting point and the target point are initialized, and the position of the intelligent pasting robot for pipe segment waterproofing materials is defined as the coordinate origin, that is, the starting position ; The detected segment position is used as the target position This initialization can clarify the starting point and the target point, so that the algorithm can start from the starting point and search towards the target point. Furthermore, the search range can be determined, which helps to limit the area searched by the algorithm, avoid searching in the entire space, and improve the efficiency of the algorithm.
[0107] S8: Randomly sample a point The point is randomly generated from the starting point toward the target point. Repeat this method in turn This will generate a node tree, and according to the currently generated tree structure, find the node closest to the random sampling point from the tree as the nearest neighbor node.
[0108] S9: First determine whether its direction is correct and define the nearest node as .
[0109] Defining vectors The orientation calculation formula is as follows:
[0110] ;
[0111] in Representation vector The module length, For the current node To target point In this way, the nodes with the correct orientation can be obtained in the number of nodes, so as to obtain the optimal path.
[0112] S10: Since the working space in this case is defined as an open space, the problem of obstacle detection is not considered. According to this step, the entire node tree is traversed from the current node in sequence.
[0113] After traversing the entire number of nodes, a path with the correct direction and the shortest distance is generated. Then the controller drives the chassis to move along the planned path and moves the intelligent pasting robot for the pipe segment waterproofing material to the top of the pipe segment to be worked on.
[0114] After the current operation is completed, the current position, i.e. the position of the nearest segment, is used as the starting position. Repeat the above steps for the next operation.
[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials, characterized in that: The specific steps include: S1: First, the intelligent pasting robot for pipe segment waterproofing materials takes the surrounding environment captured by the camera as the input of the controller, performs grayscale processing on its input image, and for color images, grayscales the red component of each pixel. , Green component and blue component Take the weighted average respectively to get the corresponding gray value ; S2: Perform denoising on the input image to effectively reduce the noise in the image and improve the visual quality of the image, enhancing the details and contrast of the image; S3: Binarize the image and convert it into a binary image containing only two pixel values, so as to more easily detect the edge and contour of the target, thereby achieving more accurate and reliable target detection and recognition; S4: After processing the image, use The filter extracts features from the image to capture the multi-scale and multi-directional texture features in the image; S5: Use extended Kalman filter for segment positioning operation to perform state estimation and update in a real-time environment, thereby providing real-time positioning results with lower computational complexity; S6: The final segment positioning result can be obtained through the state estimation value of the filter, which is represented by the position, speed and direction of the target; S7: Realize the movement of the intelligent segment waterproof material pasting robot through the path planning algorithm; S8: Randomly sample a point ; This point is randomly generated from the starting point toward the target point; iterate in this way Until the target point is found; This will generate a node tree. According to the currently generated tree structure, the node closest to the random sampling point is found in the tree as the nearest neighbor node; S9: Determine the correct node among the number of nodes so as to obtain the optimal path; S10: After traversing the entire number of nodes, a path with the correct direction and the shortest distance is generated. Then the controller drives the chassis to move along the planned path, and moves the intelligent segment waterproofing material pasting robot to the top of the segment to be operated. After completing the current operation, the current position, that is, the nearest segment position, is used as the starting position. Repeat the above steps for the next operation.
2. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: The gray value The calculation formula is as follows: 。 3. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: In step S2, a mean filter is used to reduce noise on the input image. The image noise reduction process is implemented by the following formula: ; in is the pixel value after noise reduction processing, , are the original pixel values at different points.
4. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: In step S3, the global threshold method is used to binarize the image and define the global threshold ,The binary processing calculation formula is as follows: ; in, is the pixel value after noise reduction processing, The pixel corresponds to white, The pixel corresponds to black.
5. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: In step S4, The calculation method of the filter to extract features from the image is as follows: ; in for The filter response, represents the coordinates on the image, and represents the rotated and scaled coordinates, represents ellipticity, Indicates scale, Indicates frequency, Indicates phase; The filter uses multiple different scales and directions function, applying them to each pixel position of the input image, where different scales are obtained by Adjustment, direction through To adjust; for each scale and direction, the filter output texture response represents the texture characteristics of the input image at that scale and direction.
6. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: The specific process of using the extended Kalman filter to perform segment positioning operation in step S5 is as follows: First, at each time step, the state model is used to predict and estimate the state and covariance matrix at the next moment. The calculation formula is as follows: ; ; in, Indicates at time The estimated value of the state; Represents the state transfer function, which transforms the state estimation and control input of the previous moment into Mapping to the state estimate at the current moment; Indicates at time The state covariance matrix of Represents the state transition function The Jacobian matrix of is used to linearize the nonlinear state transfer function; represents the process noise covariance matrix, which represents the uncertainty or noise in the state transition process; After the prediction, the observation model is used to compare the sensor measurement with the predicted state, and its state estimate and covariance matrix are updated. The calculation formula is as follows: Calculate the innovation covariance: ; Calculate the Kalman gain: ; Computing Innovation: ; Update the state estimate: ; Update the covariance matrix: ; in Indicates at time Observed value of represents the observation model function, which maps the current state estimate to the observation space; Represents the observation model function The Jacobian matrix of is used to linearize the nonlinear observation model; represents the observation noise covariance matrix, which represents the uncertainty or noise in the observation process; represents innovation, i.e., the difference between the observed and predicted observations; represents the innovation covariance matrix, which measures the uncertainty of observation noise and forecast error; represents the Kalman gain, which is used to update the state estimate and covariance matrix; represents the identity matrix; Repeat the prediction and update steps to iteratively update the state estimate based on new sensor measurements and model predictions; each iteration updates the state estimate and covariance matrix, thereby improving positioning accuracy; The above extended Kalman filter positioning algorithm is recursive and can perform state estimation and update in a real-time environment, providing real-time positioning results with lower computational complexity.
7. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: The specific process of realizing the movement of the intelligent segment waterproof material pasting robot by the path planning algorithm in step S7 is as follows: First, the starting point and the target point are initialized, and the position of the intelligent pasting robot for pipe segment waterproofing material is defined as the coordinate origin, that is, the starting position ; The detected segment position is used as the target position ; This initialization is used to clarify the starting point and the target point, so that the algorithm can start from the starting point and search towards the target point.
8. The method for visual inspection and path planning of a robot for intelligent pasting of pipe segment waterproof materials according to claim 1 is characterized in that: The specific method for determining the correct node in the number of nodes in step S9 is as follows: define the nearest node as ; Define vector ; The orientation calculation formula is as follows: ;in, Representation vector The module length, For the current node To target point direction; in this way, the nodes with the correct direction can be obtained in the number of nodes, so as to obtain the optimal path.
Citation Information
Patent Citations
Location and environment modeling method of intelligent movable robot
CN102706342A
Robot control method and electronic equipment
CN112417944A
Mobile transfer robot control method and system based on Cartograph algorithm
CN115407769A
Automatic welding robot control system based on artificial intelligence
CN115592324A
Robot path planning method based on RRT*- DR algorithm
CN116295393A
