Method and system for evaluating the quality of the automated painting of prefabricated surface coatings for trestles

Abstract

The present application relates to the technical field of computer, in particular to an automatic spraying quality evaluation method and system for assembled trestle surface coating, comprising: obtaining a BIM model of the assembled trestle, and performing meshing division on the surface of the BIM model; obtaining corresponding mesh surfaces on the surface of the assembled trestle; obtaining complex morphology mesh surfaces in the mesh surfaces; obtaining the best inclination and actual collection distance of each complex morphology mesh surface during image collection based on the position of each point cloud in the complex morphology mesh surface, the average value of the gradient of the point cloud depth and the gray histogram; the visual sensor completes image collection of each complex morphology mesh surface by using the best inclination and actual collection distance; the spraying quality on the entire surface of the assembled trestle is evaluated by using the mesh surface images of all mesh surfaces on the surface of the assembled trestle after spraying; the authenticity of the collected images is ensured, and the accuracy of the later quality evaluation is improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and coating quality inspection technology, and particularly to an automated method and system for evaluating the coating quality of assembled trestle surfaces. Background Technology

[0002] Prefabricated trestle bridges are an important component of temporary or permanent steel structure bridges, and the quality of their surface anti-corrosion coating directly affects the trestle bridge's service life and structural safety. Currently, the surface coating of prefabricated trestle bridges is usually completed using automated spraying equipment: engineers import the trestle bridge's 3D CAD / BIM model into a computer, use professional offline programming (OLP) software to automatically generate the spraying trajectory in a virtual environment, and then send the program to the on-site robotic arm for execution.

[0003] After spraying is completed, the coating quality needs to be evaluated to determine whether there are defects such as insufficient thickness, sagging, orange peel, or missed spraying. Existing automated spraying quality evaluation methods usually adopt visual inspection, that is, to acquire images of the trestle surface through a camera and then use image processing algorithms to identify coating defects.

[0004] However, prefabricated trestle structures are not simply flat surfaces; their components typically contain numerous seams, irregular edges, internal corners, and welded joints. When using a two-dimensional scanning method based on a fixed shooting mode to acquire images of these complex morphologies, optical interference such as shadow occlusion, overexposure, or focus failure can easily occur, causing local images to fail to accurately reflect the surface coating effect, thus leading to errors in the assessment results of local spraying quality.

[0005] In the prior art, Chinese patent application CN111998789B discloses a method for evaluating and controlling the spraying quality of thermal barrier coatings. This method uses a laser profilometer to perform a fixed two-dimensional morphological scan of the sample, but does not involve adaptive adjustment of the acquisition posture for the complex three-dimensional structure of the workpiece itself. Chinese patent application CN118115040A discloses a method and system for evaluating the spraying quality of decorative paper. This method acquires surface images before and after spraying and uses a predictive model for evaluation, but it targets relatively flat surfaces such as decorative paper and also does not involve adaptive image acquisition for complex three-dimensional morphologies.

[0006] It can be seen that although the existing technology can effectively evaluate the coating quality based on the acquired images, the evaluation is carried out by directly preprocessing the acquired images. This cannot change the problems encountered in the acquisition process of existing images, which result in defects in the acquired original images and inaccurate evaluation in the later stages. Summary of the Invention

[0007] To address the aforementioned technical problems, the present invention aims to provide an automated spraying quality assessment method and system for the surface coating of prefabricated trestle bridges. This system allows for the control of the acquisition device during image acquisition, ensuring that the acquired images more closely resemble real-world representations and providing more accurate assessments for subsequent quality evaluations.

[0008] In a first aspect, embodiments of the present invention provide an automated method for evaluating the spraying quality of coatings on the surface of assembled trestle bridges, comprising:

[0009] Obtain the BIM model of the prefabricated trestle and divide the surface of the BIM model into a grid; use the grid divided on the surface of the BIM model to map the corresponding grid surface on the surface of the prefabricated trestle after spraying.

[0010] The robotic arm moves according to the gridded surfaces. During the movement, the robotic arm uses a vision sensor to collect point cloud data and image data of the grid surfaces mapped on the surface of the assembled bridge that has been sprayed. Based on the point cloud elevation value and pixel gray value of each grid surface, the surface undulation of each grid surface is obtained. The undulation of each grid surface is used to determine the complex morphology grid surface among all grid surfaces.

[0011] The optimal tilt angle for each complex topography mesh surface during image acquisition is obtained based on the position of each point cloud, the average gradient of the point cloud depth, and the gray-level histogram.

[0012] The actual acquisition distance of each complex topography grid surface is obtained by using the optimal tilt angle of each complex topography grid surface during image acquisition, the surface undulation of the complex topography grid surface, and the acquisition distance of the vision sensor.

[0013] The robotic arm adjusts the vision sensor according to the optimal tilt angle and actual acquisition distance of each complex topographic mesh surface to complete the image acquisition of each complex topographic mesh surface, and obtains the mesh surface image of the complex topographic mesh surface;

[0014] The coating quality of the entire prefabricated trestle surface is evaluated using mesh images of all grid surfaces on the completed coating surface.

[0015] Furthermore, it also includes:

[0016] The image sharpness of each grid surface acquired by the vision sensor at each moment during the acquisition process and the vibration frequency of the robotic arm at each moment are obtained. The image quality factor at each moment is obtained using the image quality factor obtained at each moment. The trend convergence factor at each moment is calculated using the image factor obtained at each moment, and the trend convergence factors are sorted according to time sequence. When the trend convergence factor exceeds the set threshold for the first time, the grid surface image acquired at that moment is taken as the final acquired grid image.

[0017] Furthermore, methods for obtaining the surface undulation degree of each grid surface include:

[0018]

[0019] In the formula The degree of surface undulation of the grid surface; Therefore, the first in the mesh surface Elevation values ​​of point cloud locations This represents the average elevation of all point cloud locations on the grid surface. This results in the number of point clouds in the grid area; The variance of the grayscale values ​​of all pixels on this grid surface;

[0020] After obtaining the surface undulation degree of each grid surface, the obtained surface undulation degree of each grid surface is normalized, and the result of the normalization process is used to determine whether each grid surface is a complex topography grid surface.

[0021] Furthermore, the method for obtaining the optimal tilt angle for each complex topographic mesh surface during image acquisition is as follows:

[0022] Using the point cloud of each complex topographic mesh as the entire field of view, the points in the complex topographic mesh are transformed into a local coordinate system with the center of the lens of the vision sensor as the origin and the optical axis as the Z-axis.

[0023] The field of view coefficient of the optical axis vector at any tilt angle is obtained by using the gradient mean of the optical axis plane and the position of each point cloud position in the local coordinate system.

[0024] The sampling highlight saliency of the optical axis vector at any tilt angle is obtained by using the total area of ​​the gray-level histogram and the area of ​​the highlight peak of the gray-level histogram in each complex morphological grid surface;

[0025] The optimal tilt factor of the optical axis vector at any tilt angle is obtained by using the field of view coefficient and the sampling spectro saliency of the optical axis vector at any tilt angle.

[0026] Obtain the minimum optimal tilt factor, and use the tilt angle corresponding to the optimal tilt factor as the optimal tilt angle for image acquisition of complex topographic mesh surfaces.

[0027] Furthermore, methods for obtaining the field-of-view coefficients of the optical axis vector at arbitrary tilt angles include:

[0028]

[0029] In the formula: For any tilt angle of the optical axis vector The field of view coefficient below; In the local coordinate system, the first The location of a point cloud; This represents the position of the corresponding point on the optical axis plane. The number of point clouds in a complex topographic mesh surface; is the gradient mean of the point cloud depth.

[0030] Furthermore, methods for obtaining the actual acquisition distance for each complex topographic mesh surface include:

[0031] The acquisition distance adjustment coefficient for each complex mesh surface is obtained by utilizing the optimal tilt angle of the complex mesh surface during image acquisition, the surface undulation of the complex mesh surface, the average surface undulation of the normal mesh surface, and the maximum angle of the robotic arm driving the vision sensor.

[0032] The actual acquisition distance of the complex mesh surface is obtained based on the acquisition distance adjustment coefficient of the complex mesh surface and the acquisition distance of the vision sensor.

[0033] Furthermore, the method for calculating the image sharpness in each grid plane includes:

[0034]

[0035] In the formula, In order to be in The sharpness of the grid surface at any given time; In this grid image, the first The combined gradient magnitude of each pixel in the X and Y axes. This refers to the number of pixels in the grid image.

[0036] Secondly, this embodiment provides an automated spraying quality assessment system for the surface coating of prefabricated trestle bridges, including:

[0037] Mesh surface division unit is used to obtain the BIM model of the prefabricated trestle and to divide the surface of the BIM model into a mesh.

[0038] The corresponding mesh surface on the surface of the assembled trestle after spraying is mapped out using the mesh divided on the surface of the BIM model.

[0039] The complex topography mesh surface acquisition unit drives the robotic arm to move according to the meshed mesh surfaces. During the movement, the robotic arm uses a vision sensor to collect point cloud data and image data of the mesh surfaces mapped on the surface of the assembled trestle after spraying. Based on the point cloud elevation value and pixel gray value of each mesh surface, the surface undulation degree of each mesh surface is obtained. The undulation degree of each mesh surface is used to determine the complex topography mesh surfaces among all mesh surfaces.

[0040] The actual acquisition distance determination unit obtains the optimal tilt angle for each complex topography grid surface during image acquisition based on the position of each point cloud in the complex topography grid surface, the average gradient of the point cloud depth, and the gray-level histogram.

[0041] The actual acquisition distance of each complex topography grid surface is obtained by using the optimal tilt angle of each complex topography grid surface during image acquisition, the surface undulation of the complex topography grid surface, and the acquisition distance of the vision sensor.

[0042] The complex topography mesh surface image determination unit is a robotic arm that adjusts the vision sensor according to the optimal tilt angle and actual acquisition distance of each complex topography mesh surface to complete the image acquisition of each complex topography mesh surface and obtain the mesh surface image of the complex topography mesh surface.

[0043] The evaluation unit assesses the coating quality of the entire prefabricated trestle surface using images of all grid surfaces on the completed coating surface.

[0044] Thirdly, an electronic device includes:

[0045] One or more processors;

[0046] Memory, used to store one or more programs;

[0047] When the one or more programs are executed by the one or more processors, the one or more processors implement the automated spraying quality assessment method for the surface coating of an assembled trestle.

[0048] Fourthly, a computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned automated spraying quality assessment method for surface coatings of assembled trestle bridges.

[0049] The beneficial effects of this invention are as follows:

[0050] The automated spraying quality assessment method for the surface coating of prefabricated trestle provided in this application first divides the BIM model of the trestle structure into a grid to ensure comprehensive coverage and structural processing of complex morphologies. Based on the adaptive pose calculation of grid smoothness and normal features, the verticality and optimal working distance of the sampling robotic arm are calculated and adjusted in real time according to the smoothness and normal plane features of each grid surface. This ensures that the image acquisition device is always facing the surface being measured and maintains the best focal plane, thereby eliminating optical interference such as shadow occlusion and focus failure caused by surface abrupt changes.

[0051] Meanwhile, during the sampling and shooting process, the sampling trigger delay is determined by the real-time sharpness of the mesh surface image under the corresponding posture and the temporal convergence trend of the vibration frequency. The corresponding delay parameters are determined for the quality convergence characteristics of different mesh surfaces, ensuring that while improving efficiency during the shooting process, high consistency and high definition image sequences can still be obtained, thereby improving the accuracy of subsequent image-based spraying quality assessment.

[0052] By extracting coating surface features from optimized images acquired from each grid surface and performing local spraying quality assessment, all grid evaluation results are ultimately integrated into a complete trestle spraying quality distribution map, achieving the goal of comprehensive and reliable evaluation of coating quality for complex morphological components. Attached Figure Description

[0053] Figure 1 A flowchart illustrating an automated spraying quality assessment method for the surface coating of a prefabricated trestle, provided in an embodiment of the present invention.

[0054] Figure 2 This is a structural block diagram of an automated spraying quality assessment system for the surface coating of an assembled trestle, as described in an embodiment of the present invention.

[0055] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0057] Example 1

[0058] Currently, the surface coating of prefabricated trestle bridges is generally carried out using automated coating facilities. Engineers import the 3D CAD / BIM model of the trestle bridge into a computer and use professional OLP software to automatically generate the coating trajectory in a virtual environment. After the program is generated, it is directly sent to the robotic arm on site for execution. The robotic arm drives the spray head to completely coat the surface of the prefabricated trestle bridge. After the coating is completed, the robotic arm uses vision sensors to acquire images of the coated surface of the prefabricated trestle bridge, and uses the acquired images to evaluate the quality of the coating.

[0059] Currently, due to the complex morphology of steel trestle materials, such as splicing seams and inner corners, visual assessment of coating quality is prone to optical interference such as shadow occlusion, overexposure, or focus failure, making it impossible for local images to accurately reflect the surface coating effect. Therefore, this step optimizes the image acquisition quality of the surface coating by adjusting the acquisition posture.

[0060] To solve the above problems, such as Figure 1 As shown, this embodiment provides an automated method for evaluating the coating quality of a prefabricated trestle surface, which includes:

[0061] S1. Obtain the BIM model of the prefabricated trestle bridge, and mesh the surface of the BIM model; when meshing, based on the BIM model, use orthogonal projection and ray projection to divide the structural surface into square uniform grids; complete the meshing of the BIM model.

[0062] In this embodiment, an integrated sensor is installed on the robotic arm of the spraying facility. After spraying is completed, the integrated sensor samples data from the steel structure surface using the grid defined by the BIM model. To ensure that the integrated sensor on the robotic arm can sample data using the grid defined by the BIM model, the grid defined on the BIM model surface is mapped onto the surface of the prefabricated trestle after spraying to form a corresponding grid surface before data collection. Then, during data collection, data can be collected according to the grid surface.

[0063] The integrated sensor in this embodiment includes a vision sensor and a distance sensor, and the collected data includes the number and depth of point clouds on the grid surface, pixel grayscale and vibration frequency, etc.

[0064] In this embodiment, during mapping, the BIM model uses "virtual coordinates," while the robotic arm recognizes "physical coordinates of the base." Before starting data acquisition, global coordinate system calibration is required. Typically, high-precision feature holes on the trestle or preset reference targets are used to control the robotic arm's end effector to touch (or laser-mark) these known coordinate points. By calculating translation and rotation transformation matrices (homogeneous transformation matrices), the digital space of the BIM model and the working space of the sampling robotic arm are rigidly registered with high precision. This ensures that when the robotic arm moves to a point in the physical space, it precisely corresponds to a specific grid surface on the BIM model, completing the mapping of the grids divided on the BIM model onto the sprayed surface of the prefabricated trestle, forming the corresponding grid surface on the prefabricated trestle surface.

[0065] When sampling the coating effect on the steel surface of a trestle structure using a robotic arm, the steel surface is not perfectly flat, but has various textures, internal corners, and other complex uneven shapes. Therefore, using a fixed sampling mode is prone to optical interference such as shadow occlusion, overexposure, or focus failure, making local images unable to accurately reflect the surface coating effect, thus leading to errors in coating quality assessment. In view of the above problems, the main purpose of this embodiment is to improve sampling accuracy by adaptively adjusting the sampling posture.

[0066] In this embodiment, a visual sensor on a robotic arm is used for grid sampling. While the visual sensor can generally handle the sampling of flat grid surfaces, it does not perform well on grid surfaces with complex shapes. Therefore, it is necessary to adaptively adjust the sampling mode for grid surfaces with complex shapes so that the sampling posture can adapt to the changes in complex shapes in order to maximize the sampling accuracy.

[0067] S2. In order to distinguish all the collected mesh surfaces, this step categorizes them into normal mesh surfaces and complex-shaped mesh surfaces. The technical solution adopted is as follows:

[0068] Because complex mesh surfaces exhibit obvious undulations or curvatures, their undulations can be used for evaluation. Furthermore, these undulations represent uneven light reflection, resulting in high-frequency abrupt changes and chaotic gradients at the pixel level. Therefore, the selection process employs the following method:

[0069] By driving a robotic arm, vision and distance sensors mounted on the arm collect data from the grid surfaces on the steel structure of the prefabricated trestle. The collected data includes point cloud data and image data for each grid surface. After obtaining the point cloud data and image data for each grid surface, the surface undulation of each grid surface is calculated using the point cloud elevation value of each point cloud data in each grid surface and the grayscale value of each pixel in the grid surface. The specific calculation formula is as follows:

[0070]

[0071] In the formula The degree of surface undulation of the grid surface; Therefore, the first in the mesh surface Elevation values ​​of point cloud locations This represents the average elevation of all point cloud locations on the grid surface. This results in the number of point clouds in the grid area; This is the variance of the grayscale values ​​of all pixels on the grid surface; the method for calculating the variance of the grayscale values ​​of all pixels is existing technology, and its specific formula will not be described in detail here.

[0072] As can be seen from the above formula, This represents the difference in elevation values ​​of the point cloud positions on this grid surface. A larger difference indicates a greater degree of surface undulation, and vice versa. The degree of unevenness is measured by the variance of surface pixel grayscale. The larger the variance, the greater the degree of grayscale disorder of the corresponding pixels, and the more likely the surface is to have uneven shape, thus the greater its undulation.

[0073] To extract complex-shaped mesh surfaces after obtaining the surface undulation degree of each mesh surface, this embodiment normalizes the obtained surface undulation degree. The normalized surface undulation degree value is in the range [0,1]. This embodiment uses the normalized surface undulation degree for filtering. Mesh surfaces with a value greater than or equal to 0.7 are classified as complex topography mesh surfaces; other mesh surfaces are classified as normal mesh surfaces.

[0074] In this embodiment, after extracting the complex topography mesh surfaces from all mesh surfaces, since the corresponding steel surface of each complex topography mesh surface must have various textures, inner corners and other complex concave and convex features, in order to improve the sampling quality, if the sampling angle of the robotic arm and the acquisition distance of the vision sensor are not adjusted, the sampled image will affect the subsequent quality evaluation. Therefore, in order to ensure that the image acquisition of complex topography mesh surfaces can be accurate, it is necessary to adaptively adjust the acquisition angle and acquisition distance for each complex topography mesh surface.

[0075] To obtain a clear and complete image, it is generally necessary to keep the camera's optical axis (visual sensor) parallel to the grid surface normal vector and ensure that the physical distance between the sensor lens and the component surface is always equal to the lens's optimal working focal length during image acquisition. For complex grid surfaces, the above parameters need to be adaptively adjusted.

[0076] Generally, the camera optical axis is determined based on the normal vector of a flat mesh surface. For complex mesh surfaces, the parallel camera optical axis will be severely blurred because the difference in the field of view exceeds the "depth of field". Therefore, the angle offset needs to be adjusted, that is, the tilt angle needs to be optimized. The optimal tilt angle needs to take into account all surface points in the entire field of view.

[0077] This embodiment is performed according to the following method:

[0078] S3. Based on the position of each point cloud in the complex topography mesh surface, the average gradient of the point cloud depth, and the gray-level histogram, the optimal tilt angle of each complex topography mesh surface during image acquisition is obtained;

[0079] When calculating the optimal tilt angle for image acquisition, this embodiment uses the point cloud of each complex topography grid surface as the entire field of view, and transforms the points in the complex topography grid surface into a local coordinate system with the center of the lens of the vision sensor as the origin and the optical axis as the Z-axis.

[0080] The field of view coefficient of the optical axis vector at any tilt angle is obtained by using the gradient mean of the optical axis plane and the position of each point cloud position in the local coordinate system.

[0081] In this embodiment, the field of view coefficient is calculated according to the following formula:

[0082]

[0083] In the formula: For any tilt angle of the optical axis vector The field of view coefficient below; In the local coordinate system, the first The location of a point cloud; This represents the position of the corresponding point on the optical axis plane. The number of point clouds in a complex topographic mesh surface; is the gradient mean of the point cloud depth.

[0084] In the formula, This represents the first The depth value of a point cloud relative to the optical axis plane. The smaller the value, the better. The shorter the distance from the camera's optical axis to any point cloud at a tilt angle, the clearer the field of view; simultaneously The smaller the value, the more "gentle" the surface undulations are relative to the lens at that angle, thus maximizing the depth of field and resulting in better image quality.

[0085] Meanwhile, since the steel of the trestle bridge is relatively smooth after being freshly painted, if the camera's tilt angle just meets the condition that "the angle of incidence equals the angle of reflection", strong reflected light will directly enter the lens, causing large areas of overexposure in the image. Therefore, the degree of brightness needs to be controlled when adjusting the tilt angle. Since the presence of highlights will cause a significant peak in the direction of high grayscale values ​​in the grayscale histogram, this embodiment needs to use the total area of ​​the grayscale histogram in each complex topographic grid surface and the area of ​​the highlight peak in the grayscale histogram to obtain the sampling highlight significance of the optical axis vector at any tilt angle before calculating the optimal tilt angle.

[0086] The optimal tilt factor of the optical axis vector at any tilt angle is obtained by using the field of view coefficient and the sampling spectro saliency of the optical axis vector at any tilt angle.

[0087] Obtain the minimum optimal tilt factor, and use the tilt angle corresponding to the optimal tilt factor as the optimal tilt angle for image acquisition of complex topographic mesh surfaces;

[0088] Specifically, the saliency of the sampled specular highlight is calculated using the following formula:

[0089]

[0090] In the formula: For any tilt angle of the optical axis vector Sampling highlight saliency; This represents the total area of ​​the grayscale histogram of the grid surface. This represents the area of ​​the highlight peak in the grayscale histogram of the grid surface; The larger the angle, the larger the proportion of the highlight area at that tilt angle, and the greater the highlight prominence.

[0091] Meanwhile, for any camera and any optical axis vector tilt angle, in order to improve the image sharpness of complex topographic grid surfaces, it is necessary to combine the above two factors. That is, the optimal tilt angle should have the sharpest field of view and the smallest specular salience. Therefore, when calculating the optimal tilt angle factor at any tilt angle, it is necessary to use the field of view coefficient and the sampled specular salience at any tilt angle for calculation. The calculation formula is as follows:

[0092]

[0093] In the formula The optimal tilt factor for the optical axis vector of a complex-shaped mesh surface at any tilt angle; For any tilt angle of the optical axis vector The field of view coefficient below, For any tilt angle of the optical axis vector The sampling highlight salience is determined by the optimal tilt angle, which is associated with both the minimum field of view coefficient and the minimum sampling highlight salience.

[0094] This yields the optimal dip factor for all angles on each complex topographic mesh surface. To determine the optimal dip angle, the smallest optimal dip factor among all the optimal dip factors is selected. The corresponding tilt angle is the optimal tilt angle for the complex topography mesh surface. For example, if the angles included on each complex topography grid surface are 15°, 18°, 22°, 24°, and 28°, and their corresponding optimal tilt factors are 0.4, 0.36, 3.38, 0.31, and 0.34, then the optimal tilt angle for this complex topography grid surface is 24°.

[0095] S4. Obtaining the optimal tilt angle for complex topographic mesh surfaces Subsequently, because the optical axis of the camera (vision sensor) tilts to avoid obstacles or conform to the surface orientation, the actual physical optical path from the lens to the surface of the component will immediately change non-linearly. If the acquisition distance remains at the value before tilting, the image will inevitably be out of focus over a large area. Therefore, it is necessary to adjust the acquisition distance appropriately according to the optimal tilt angle to compensate for the loss of sharpness.

[0096] Since different complex morphological grid surfaces have different degrees of undulation, and the greater the degree of undulation, the greater the corresponding surface roughness, it is necessary to appropriately shorten the acquisition distance in order to maintain image clarity; at the same time, the existence of tilt angle will inevitably lead to a reduction in the sampling field of view, so it is necessary to appropriately increase the acquisition distance.

[0097] In this embodiment, the acquisition distance is obtained using the following method:

[0098] First, the acquisition distance adjustment coefficient for each complex mesh surface is obtained by utilizing the optimal tilt angle for image acquisition of the complex mesh surface, the surface undulation of the complex mesh surface, the average surface undulation of the normal mesh surface, and the maximum angle of the robotic arm driving the vision sensor; the specific calculation method is as follows:

[0099]

[0100] In the formula This is the adjustment coefficient for the acquisition distance of complex mesh surfaces; For the optimal tilt angle of a complex mesh surface, This represents the maximum angle range of the robotic arm; The degree of surface undulation of a complex mesh surface, This is the average surface undulation of all normal grid surfaces; then, the adjustment is made by comparing the ratio of these two values. If the value is greater than 0, the corresponding sampling distance is increased; if... If the value is less than 0, the corresponding sampling distance will be reduced; if the value is equal to 0, no adjustment will be made.

[0101] After obtaining the acquisition distance adjustment coefficient for the complex mesh surface, the actual acquisition distance of the complex topography mesh surface is obtained based on the acquisition distance adjustment coefficient and the acquisition distance of the vision sensor; the specific algorithm includes:

[0102]

[0103] In the formula: This refers to the actual acquisition distance for complex mesh surfaces. The camera's acquisition distance. This is the acquisition distance adjustment coefficient for complex mesh surfaces; then, as... The positive and negative changes, the acquisition distance of its robotic arm is based on Adaptive adjustments are made to avoid out-of-focus blur.

[0104] After obtaining the optimal tilt angle and actual acquisition distance for each complex grid surface, the robotic arm is adjusted to adjust the vision sensor according to the optimal tilt angle and actual acquisition distance, and then images are acquired for each complex grid surface.

[0105] S5. After acquiring images of all grid surfaces, register the spatial projection coordinates of all grid surface images with the BIM digital twin model to achieve projection stitching of discrete visual defect features on the three-dimensional grid surface.

[0106] Then, the local grayscale variance of the stitched image is calculated, and the grayscale variance is used to evaluate the spraying quality. Specifically, the calculated variance is mapped to a 0-1 health index space. The closer it is to 1, the better the spraying quality, and the closer it is to 0, the worse the quality. Finally, an overall spraying quality map is constructed based on this result.

[0107] In one embodiment, when the vision sensor controlled by the robotic arm takes a picture, it needs to adjust the tilt angle and acquisition distance when encountering complex grid surfaces. During the adjustment process, the robotic arm may suddenly stop, which can easily cause high-frequency residual vibrations during the shooting process. This can lead to image shaking and blurring, affecting the clarity of the image.

[0108] To address the aforementioned issues, this embodiment captures multiple sets of images during the shooting process for each grid surface. The best image is then selected as the final image for that grid surface. The selected final image is then used for quality evaluation. Specifically, the selection process is as follows:

[0109] Since the images in each grid face are captured in a time sequence by the vision sensor, an image is captured at each time t during the capture process. That is, the images captured in each grid face form a time sequence image set.

[0110] After obtaining the time series image set corresponding to each grid surface, the image quality factor at each moment is obtained by using the image sharpness of the image at each moment and the vibration frequency of the robotic arm at each moment. The trend convergence factor at each moment is calculated by using the image factor obtained at each moment, and the trend convergence factors are sorted according to the time series. When the trend convergence factor exceeds the set threshold for the first time, the image corresponding to that moment is taken as the final acquired grid image.

[0111] In this embodiment, the image sharpness is calculated using the following method:

[0112]

[0113] In the formula, In order to be in The sharpness of the grid surface at any given time; In this grid image, the first The combined gradient magnitude of each pixel in the X and Y axes. This refers to the number of pixels in the grid image;

[0114] This represents the average value of the overall gradient of all pixels in the image. The smaller the value, the stronger the blur and the lower the image sharpness; conversely, the higher the value, the higher the sharpness and the clearer the image.

[0115] When calculating the image quality factor, because vibrations gradually dissipate as the shooting time increases when the vision sensor is driven by a robotic arm, the corresponding image features gradually change from blurry and low sharpness to clear and high sharpness, thus gradually meeting the shooting conditions; therefore, the image quality factor is calculated using the following method:

[0116]

[0117] In the formula: In order to be in Image quality factor at any given moment. Let be the image sharpness of the grid surface at time t. Let be the vibration frequency of the robotic arm at time t;

[0118] Image sharpness gradually converges over time, while vibration frequency gradually approaches 0. This is reflected in the image quality factor, where its value converges over time.

[0119] Since a smaller image quality factor corresponds to greater stability and a clearer image, the optimal sampling point is when the quality factor just begins to converge. Therefore, for any given sampling point:

[0120]

[0121] In the formula For trend convergence factor; Let be the image quality factor at time t. and These are the image quality factors for the previous and next time moments, respectively; since the gradient of the quality factor slows down immediately when the curve suddenly converges, therefore... The value will immediately increase, and the convergence trend factor will increase. Here, the corresponding real-time convergence trend factor is calculated and normalized based on the real-time curve trend. When the first detection... When the trend convergence occurs, the image is considered to have stabilized, and the image at that moment is taken as the sampled image of the grid surface.

[0122] The images selected by the above method in this embodiment can effectively avoid the impact of vibration on image quality, thus improving the reliability of subsequent spraying quality assessment.

[0123] In one embodiment, such as Figure 2 As shown, an automated spraying quality assessment system for the surface coating of an assembled trestle is provided, comprising:

[0124] Mesh surface division unit is used to obtain the BIM model of the prefabricated trestle and to divide the surface of the BIM model into a mesh.

[0125] The corresponding mesh surface on the surface of the assembled trestle after spraying is mapped out using the mesh divided on the surface of the BIM model.

[0126] The complex topography mesh surface acquisition unit drives the robotic arm to move according to the meshed mesh surfaces. During the movement, the robotic arm uses a vision sensor to collect point cloud data and image data of the mesh surfaces mapped on the surface of the assembled trestle after spraying. Based on the point cloud elevation value and pixel gray value of each mesh surface, the surface undulation degree of each mesh surface is obtained. The undulation degree of each mesh surface is used to determine the complex topography mesh surfaces among all mesh surfaces.

[0127] The actual acquisition distance determination unit obtains the optimal tilt angle for each complex topography grid surface during image acquisition based on the position of each point cloud in the complex topography grid surface, the average gradient of the point cloud depth, and the gray-level histogram.

[0128] The actual acquisition distance of each complex topography grid surface is obtained by using the optimal tilt angle of each complex topography grid surface during image acquisition, the surface undulation of the complex topography grid surface, and the acquisition distance of the vision sensor.

[0129] The complex topography mesh surface image determination unit is a robotic arm that adjusts the vision sensor according to the optimal tilt angle and actual acquisition distance of each complex topography mesh surface to complete the image acquisition of each complex topography mesh surface and obtain the mesh surface image of the complex topography mesh surface.

[0130] Image filtering unit: It acquires the image sharpness of each grid surface acquired by the vision sensor at each moment during the acquisition process and the vibration frequency of the robotic arm at each moment. It uses the image sharpness of the grid surface acquired at each moment and the vibration frequency of the robotic arm at each moment to obtain the image quality factor at each moment. It uses the image factor obtained at each moment to calculate the trend convergence factor at that moment and sorts the trend convergence factors according to the time sequence. When the trend convergence factor exceeds the set threshold for the first time, the grid surface image acquired at that moment is taken as the final acquired grid image.

[0131] The evaluation unit assesses the coating quality of the entire prefabricated trestle surface using images of all grid surfaces on the completed coating surface.

[0132] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. For example... Figure 3 The diagram shows a structural block diagram of an electronic device. An embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement an automated spraying quality assessment method for the surface coating of an assembled bridge as described in any of the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0133] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0134] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0135] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0136] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in the automated spraying quality assessment method for the surface coating of any of the assembled trestle embodiments described above. The computer-readable storage medium can be volatile or non-volatile.

[0137] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described automated spraying quality assessment method for the surface coating of the assembled stack.

[0138] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0139] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0140] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0141] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0142] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0143] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0144] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0145] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0147] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. An automated method for evaluating the coating quality of a prefabricated trestle surface, characterized in that, include: Obtain the BIM model of the prefabricated trestle bridge, and divide the surface of the BIM model into a grid. The corresponding mesh surface on the surface of the assembled trestle after spraying is mapped out using the mesh divided on the surface of the BIM model. The robotic arm moves according to the gridded surfaces. During the movement, the robotic arm uses a vision sensor to collect point cloud data and image data of the grid surfaces mapped on the surface of the assembled bridge that has been sprayed. Based on the point cloud elevation value and pixel gray value of each grid surface, the surface undulation of each grid surface is obtained. The undulation of each grid surface is used to determine the complex morphology grid surface among all grid surfaces. The optimal tilt angle for each complex topography mesh surface during image acquisition is obtained based on the position of each point cloud, the average gradient of the point cloud depth, and the gray-level histogram. The actual acquisition distance of each complex topography grid surface is obtained by using the optimal tilt angle of each complex topography grid surface during image acquisition, the surface undulation of the complex topography grid surface, and the acquisition distance of the vision sensor. The robotic arm adjusts the vision sensor according to the optimal tilt angle and actual acquisition distance of each complex topographic mesh surface to complete the image acquisition of each complex topographic mesh surface, and obtains the mesh surface image of the complex topographic mesh surface; The coating quality of the entire prefabricated trestle surface is evaluated using mesh images of all grid surfaces on the completed coating surface.

2. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 1, characterized in that, Also includes: The image sharpness of each grid surface acquired by the vision sensor at each moment during the acquisition process and the vibration frequency of the robotic arm at each moment are obtained. The image quality factor at each moment is obtained using the image quality factor at each moment. The trend convergence factor at each moment is calculated using the image quality factor obtained at each moment. The trend convergence factors are sorted according to time sequence. When the trend convergence factor exceeds the set threshold for the first time, the grid surface image acquired at that moment is taken as the final acquired grid image.

3. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 1, characterized in that, Methods for obtaining the surface undulation of each grid surface include: ； In the formula The degree of surface undulation of the grid surface; Therefore, the first in the mesh surface Elevation values ​​of point cloud locations This represents the average elevation of all point cloud locations on the grid surface. This results in the number of point clouds in the grid area; The variance of the grayscale values ​​of all pixels on this grid surface; After obtaining the surface undulation degree of each grid surface, the obtained surface undulation degree of each grid surface is normalized, and the result of the normalization process is used to determine whether each grid surface is a complex topography grid surface.

4. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 1, characterized in that, The method for obtaining the optimal tilt angle for each complex topographic mesh surface during image acquisition is as follows: Using the point cloud of each complex topographic mesh as the entire field of view, the points in the complex topographic mesh are transformed into a local coordinate system with the center of the lens of the vision sensor as the origin and the optical axis as the Z-axis. The field of view coefficient of the optical axis vector at any tilt angle is obtained by using the gradient mean of the optical axis plane and the position of each point cloud position in the local coordinate system. The sampling highlight saliency of the optical axis vector at any tilt angle is obtained by using the total area of ​​the gray-level histogram and the area of ​​the highlight peak of the gray-level histogram in each complex morphological grid surface; The optimal tilt factor of the optical axis vector at any tilt angle is obtained by using the field of view coefficient and the sampling spectro saliency of the optical axis vector at any tilt angle. Obtain the minimum optimal tilt factor, and use the tilt angle corresponding to the optimal tilt factor as the optimal tilt angle for image acquisition of complex topographic mesh surfaces.

5. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 4, characterized in that, Methods for obtaining the field-of-view coefficients of the optical axis vector at any tilt angle include: ； In the formula: For any tilt angle of the optical axis vector The field of view coefficient below; In the local coordinate system, the first The location of a point cloud; This represents the position of the corresponding point on the optical axis plane. The number of point clouds in a complex topographic mesh surface; The gradient mean of the point cloud depth.

6. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 1, characterized in that, Methods for obtaining the actual acquisition distance of each complex topographic mesh surface include: The acquisition distance adjustment coefficient for each complex mesh surface is obtained by utilizing the optimal tilt angle of the complex mesh surface during image acquisition, the surface undulation of the complex mesh surface, the average surface undulation of the normal mesh surface, and the maximum angle of the robotic arm driving the vision sensor. The actual acquisition distance of the complex mesh surface is obtained based on the acquisition distance adjustment coefficient of the complex mesh surface and the acquisition distance of the vision sensor.

7. The automated spraying quality assessment method for the surface coating of prefabricated trestle bridges according to claim 2, characterized in that, The method for calculating the image sharpness in each grid plane includes: ； In the formula, In order to be in The image sharpness of the grid surface at any given time; In this grid image, the first The combined gradient magnitude of each pixel in the X and Y axes. This refers to the number of pixels in the grid image.

8. An automated spraying quality assessment system for the surface coating of prefabricated trestle bridges, characterized in that, include: Mesh surface division unit is used to obtain the BIM model of the prefabricated trestle and to divide the surface of the BIM model into a mesh. The corresponding mesh surface on the surface of the assembled trestle after spraying is mapped out using the mesh divided on the surface of the BIM model. The complex topography mesh surface acquisition unit drives the robotic arm to move according to each mesh surface after the mesh is divided. During the movement, the robotic arm uses the provided vision sensor to collect point cloud data and image data of the mesh surface mapped on the surface of the assembled trestle after spraying. Based on the point cloud elevation value and pixel gray value of each mesh surface, the surface undulation degree of each mesh surface is obtained. The degree of undulation of each grid surface is used to determine the complex topography grid surface among all grid surfaces; The actual acquisition distance determination unit obtains the optimal tilt angle for each complex topography grid surface during image acquisition based on the position of each point cloud in the complex topography grid surface, the average gradient of the point cloud depth, and the gray-level histogram; and obtains the actual acquisition distance of each complex topography grid surface by using the optimal tilt angle for image acquisition, the surface undulation of the complex topography grid surface, and the acquisition distance of the vision sensor. The complex topography mesh surface image determination unit is a robotic arm that adjusts the vision sensor according to the optimal tilt angle and actual acquisition distance of each complex topography mesh surface to complete the image acquisition of each complex topography mesh surface and obtain the mesh surface image of the complex topography mesh surface. The evaluation unit assesses the coating quality of the entire prefabricated trestle surface using images of all grid surfaces on the completed coating surface.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement an automated spraying quality assessment method for the surface coating of an assembled trestle as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements an automated spraying quality assessment method for the surface coating of an assembled trestle as described in any one of claims 1 to 7.

Images