H-shaped Steel Structure Cleaning Method, Device, Equipment and Storage Medium

Abstract

The present invention relates to the field of laser cleaning, and particularly to a cleaning method, device, equipment and storage medium for H-shaped steel structures. The method includes: performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data; modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned; determining the spatial coordinates of the surface of the H-shaped steel structure to be cleaned according to the point cloud data model; performing triangulation according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; according to the three-dimensional structure of the H-shaped steel structure to be cleaned, through a PLC control strategy, planning the cleaning path of the H-shaped steel structure to be cleaned and completing the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned, so as to obtain the three-dimensional structure of the H-shaped steel structure, plan the cleaning path according to the three-dimensional structure, and complete the cleaning according to the cleaning path.

Description

Technical Field

[0001] The present invention relates to the field of laser cleaning, and particularly to a cleaning method, device, equipment and storage medium for H-shaped steel structures. Background Technique

[0002] The H-shaped steel structure is a new type of economic building steel. The cross-sectional shape of the H-shaped steel structure is economically reasonable, with good mechanical properties. During rolling, the elongation of each point on the cross-section is relatively uniform and the internal stress is small. Compared with ordinary I-beams, it has the advantages of a large section modulus, light weight, and metal savings. Therefore, how to accurately and efficiently clean the H-shaped steel structure has become an urgent technical problem to be solved. Currently, in the traditional method of mainly using ultrasonic waves to clean the H-shaped steel, rust and other solid impurities on the surface of the H-shaped steel precipitate at the bottom of the liquid storage tank. To facilitate the placement of the H-shaped steel, the length of the liquid storage tank is relatively long. Therefore, when cleaning the precipitate at the bottom of the liquid storage tank, it is necessary to gradually clean along the length direction of the liquid storage tank, resulting in low cleaning efficiency and increased labor intensity due to the back-and-forth movement of personnel.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a cleaning method, device, equipment and storage medium for H-shaped steel structures, aiming to solve the technical problem in the prior art that it is difficult to achieve high-efficiency cleaning of H-shaped steel structures.

[0005] To achieve the above object, the present invention provides a cleaning method for H-shaped steel structures, and the cleaning method for H-shaped steel structures includes the following steps:

[0006] Performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data;

[0007] Modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned;

[0008] Determining the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model;

[0009] Performing triangulation according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned;

[0010] According to the three-dimensional structure of the H-shaped steel structure to be cleaned, through a PLC control strategy, planning the cleaning path of the H-shaped steel structure to be cleaned and completing the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned.

[0011] Optionally, the H-shaped steel structure to be cleaned is three-dimensionally scanned by a binocular camera to obtain scanned point cloud data, including:

[0012] Respectively obtain the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned;

[0013] Respectively perform convolution operations with the same parameters on the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned to obtain a left feature map and a right feature map;

[0014] Combine the left feature map and the right feature map to obtain a sequence image of two frames, and perform a 3D convolution operation on the sequence image to obtain a disparity feature map, and obtain the point cloud data of the H-shaped steel structure to be cleaned according to the disparity feature map.

[0015] Optionally, the combining the left feature map and the right feature map to obtain a sequence image of two frames, and performing a 3D convolution operation on the sequence image to obtain a disparity feature map includes:

[0016] Perform a 3D convolution operation on the sequence image;

[0017] Flatten the disparity feature map after the 3D convolution operation into a two-dimensional vector, perform a fully connected operation on the two-dimensional vector, and rearrange the fully connected two-dimensional vector to obtain a one-channel three-dimensional feature image;

[0018] Obtain the left-eye view or the right-eye view again, and perform a 3D convolution operation on the obtained left-eye view or right-eye view;

[0019] Perform a channel combination on the one-channel three-dimensional feature image and the left-eye view or the right-eye view received again after the 3D convolution operation to obtain a combined image;

[0020] Perform a deconvolution operation on the combined image after the 3D convolution operation to obtain a disparity feature map.

[0021] Optionally, the modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned includes:

[0022] Establish a coarse-grained model according to the coarse-grained properties of the point cloud data, and construct a fine-grained model corresponding to the coarse-grained model according to the fine-grained properties of the point cloud data;

[0023] Combine the coarse-grained model and the fine-grained model to obtain the point cloud data model.

[0024] Optionally, the determining the spatial coordinates of the surface of the H-shaped steel structure to be cleaned according to the point cloud data model includes:

[0025] According to the point cloud data model, a reference three-dimensional coordinate system is set around the H-shaped steel structure to be cleaned;

[0026] Obtain the three-dimensional coordinate points of several control points on the three-dimensional coordinate system;

[0027] Take the three-dimensional coordinate points of several control points on the three-dimensional coordinate system as reference nodes, and establish the spatial position topological relationship between the points to be measured on the surface of the H-shaped steel structure to be cleaned and several control points on the three-dimensional coordinate system;

[0028] According to the spatial position topological relationship, obtain the spatial coordinates of the points to be measured on the surface of the H-shaped steel structure to be cleaned in the reference three-dimensional coordinate system.

[0029] Optionally, the triangulation is performed according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned, including:

[0030] Extract key frames through binocular disparity according to the spatial coordinates of the surface of the three-dimensional structure of the H-shaped steel structure to be cleaned;

[0031] Group the key frames and use the structure from motion algorithm to extract representative frames from each group of key frames, and perform structure from motion recovery on the representative frames;

[0032] Calibrate each group of key frames according to the structure from motion result, optimize each group of key frames, and obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

[0033] Optionally, the triangulation is performed according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned, including:

[0034] Extract key frames through binocular disparity according to the spatial coordinates of the surface of the three-dimensional structure of the H-shaped steel structure to be cleaned;

[0035] Group the key frames and use the structure from motion algorithm to extract representative frames from each group of key frames, and perform structure from motion recovery on the representative frames;

[0036] Calibrate each group of key frames according to the structure from motion result, optimize each group of key frames, and obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

[0037] In addition, to achieve the above object, the present invention also proposes an H-shaped steel structure cleaning device, and the H-shaped steel structure cleaning device includes:

[0038] A scanning module for performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data;

[0039] A modeling module, configured to model the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned;

[0040] A determination module, configured to determine the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model;

[0041] A triangulation module, configured to perform triangulation according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain a three-dimensional structure of the H-shaped steel structure to be cleaned;

[0042] A cleaning module, configured to plan a cleaning path for the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned through a PLC control strategy and complete the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned.

[0043] In addition, to achieve the above object, the present invention further provides an H-shaped steel structure cleaning device, where the device includes: a memory, a processor, and an H-shaped steel structure cleaning program stored on the memory and running on the processor, and the H-shaped steel structure cleaning program is configured to implement the H-shaped steel structure cleaning method as described above.

[0044] In addition, to achieve the above object, the present invention further provides a storage medium, where an H-shaped steel structure cleaning program is stored on the storage medium, and when the H-shaped steel structure cleaning program is executed by a processor, the H-shaped steel structure cleaning method as described above is implemented.

[0045] The present invention discloses an H-shaped steel structure cleaning method, device, equipment, and storage medium. The method includes: performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain scanned point cloud data; modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned; determining the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model; performing triangulation according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain a three-dimensional structure of the H-shaped steel structure to be cleaned; planning a cleaning path for the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned through a PLC control strategy and completing the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned, so as to determine the three-dimensional structure of the H-shaped steel structure to be cleaned according to the H-shaped steel structure to be cleaned, determine the cleaning path of the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned, and then complete the cleaning through a laser according to the cleaning path of the H-shaped steel structure to be cleaned. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic structural diagram of an H-shaped steel structure cleaning device in a hardware operating environment related to an embodiment of the present invention;

[0047] Figure 2 It is a schematic flowchart of the first embodiment of the cleaning method for H-shaped steel structures of the present invention;

[0048] Figure 3 It is a schematic flowchart of the second embodiment of the cleaning method for H-shaped steel structures of the present invention;

[0049] Figure 4 It is a schematic flowchart of the third embodiment of the cleaning method for H-shaped steel structures of the present invention;

[0050] Figure 5 It is a schematic diagram of the functional modules of the first embodiment of the cleaning device for H-shaped steel structures of the present invention.

[0051] The realization, functional features and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] Refer to Figure 1 , Figure 1 It is a schematic structural diagram of the H-shaped steel structure cleaning equipment for the hardware operating environment involved in the solution of the embodiment of the present invention.

[0054] Such as Figure 1As shown in the figure, the H-shaped steel structure cleaning device may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. For the wired interface of the user interface 1003, it may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed Random Access Memory (RAM) memory, or a stable memory (Non-volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art can understand that Figure 1 the structure shown in does not constitute a limitation on the H-shaped steel structure cleaning device, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0056] As Figure 1 shown, the memory 1005 regarded as a computer storage medium may include an operating system, a network communication module, a user interface module, and an H-shaped steel structure cleaning program.

[0057] In Figure 1 the H-shaped steel structure cleaning device shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the user device; the H-shaped steel structure cleaning device calls the H-shaped steel structure cleaning program stored in the memory 1005 through the processor 1001 and executes the H-shaped steel structure cleaning method provided in the embodiments of the present invention.

[0058] Based on the above hardware structure, an embodiment of the H-shaped steel structure cleaning method of the present invention is proposed.

[0059] Referring to Figure 2 , Figure 2 which is a schematic flowchart of the first embodiment of the H-shaped steel structure cleaning method of the present invention, the first embodiment of the H-shaped steel structure cleaning method of the present invention is proposed.

[0060] In the first embodiment, the H-shaped steel structure cleaning method includes the following steps:

[0061] Step S10: Three-dimensionally scan the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data.

[0062] It should be understood that the execution subject of this embodiment is an H-shaped steel structure cleaning device, which has functions such as data processing, data communication, and program operation.

[0063] In a specific implementation, the binocular camera is assembled, and the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned are respectively obtained and sent to the H-shaped steel structure cleaning device; the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned are respectively subjected to convolution operations with the same parameters to obtain a left feature map and a right feature map; the left feature map and the right feature map are combined to obtain a sequence image of two frames, and the sequence image is subjected to a 3D convolution operation to obtain a disparity feature map, and the point cloud data of the H-shaped steel structure to be cleaned is obtained according to the disparity feature map, so as to obtain the point cloud data of the H-shaped steel structure to be cleaned by acquiring the left-eye view and the right-eye view of the H-shaped steel structure and according to the left-eye view and the right-eye view.

[0064] It should be noted that performing a convolution operation on the left-eye view to obtain a left feature map and performing a convolution operation on the right-eye view to obtain a right feature map can be understood as inputting the received left-eye view and right-eye view into two twin branches. The twin branches mean using exactly the same feature extraction method for the left-eye view and the right-eye view. The operation parameters used for the left-eye view and the right-eye view within the twin branches must be exactly the same, and the weights of the two branches are strictly shared during network training, that is, the convolutions finally learned during network training are exactly the same after the left-eye view and the right-eye view are respectively input into the twin branches.

[0065] Step S20: Model the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned.

[0066] In a specific implementation, a coarse-grained model is established according to the coarse-grained nature of the point cloud data, and a fine-grained model corresponding to the coarse-grained model is constructed according to the fine-grained nature of the point cloud data; the coarse-grained model and the fine-grained model are combined to obtain the point cloud data model.

[0067] It should be understood that for the coarse-grained data, due to the small number of point clouds, fast model establishment and parsing time, fast real-time imaging can be achieved. For the fine-grained data, due to the large number of point clouds, refined modeling can be achieved. Among them, the coarse-grained data and the corresponding fine-grained data can be separated during acquisition by the acquisition module, or can be separated after being processed by the server to obtain the point cloud data model.

[0068] Step S30: Determine the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model.

[0069] In a specific implementation, according to the point cloud data model, a reference three-dimensional coordinate system is set around the H-shaped steel structure to be cleaned; the three-dimensional coordinate points of several control points on the three-dimensional coordinate system are obtained; the three-dimensional coordinate points of several control points on the three-dimensional coordinate system are used as reference nodes to establish the spatial position topological relationship between the measurement points on the surface of the H-shaped steel structure to be cleaned and several control points on the three-dimensional coordinate system; according to the spatial position topological relationship, the spatial coordinates of the measurement points on the surface of the H-shaped steel structure to be cleaned in the reference three-dimensional coordinate system are obtained, so that a reference three-dimensional coordinate system is set around the H-shaped steel structure to be cleaned, and according to the spatial position topological relationship, the spatial coordinates on the coordinate system are obtained.

[0070] It should be understood that the projection point of the measurement point on the plane where the three-dimensional coordinate system is located is denoted as the projection point. The physical space coordinates of the projection point Mp are (xp, yp, zp). Let the Euclidean distances from the measurement point to the XOY plane, YOZ plane, and ZOX plane be the first Euclidean distance value L1, the second Euclidean distance value L2, and the third Euclidean distance value L3 respectively; the first Euclidean distance value L1, the second Euclidean distance value L2, the third Euclidean distance value L3, and the vertical distance h are obtained by direct calculation. Also, since the conversion between physical space coordinates and the three-dimensional coordinate system belongs to an isometric transformation, the distance values are global invariants, and the relative position relationship of the first Euclidean distance value L1, the second Euclidean distance value L2, the third Euclidean distance value L3, and the vertical distance h remains unchanged in the scanner coordinate system. Using this property, the spatial coordinates of the measurement point in the coordinate system can be solved.

[0071] Step S40: Perform triangulation according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

[0072] In a specific implementation, according to the spatial coordinates on the surface of the three-dimensional structure of the H-shaped steel structure to be cleaned, key frames are extracted through binocular parallax; the key frames are grouped and the representative frames are extracted from each group of key frames through the structure from motion algorithm, and the structure from motion is restored for the representative frames; each group of key frames is calibrated according to the structure from motion result, and each group of key frames is optimized to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned, and the three-dimensional structure is obtained by the inverse transformation of the cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned.

[0073] It should be understood that the key frames are divided into the same group according to the sequence order, and the key frames at the same position in each group of key frames are selected in order and divided into a group. Detect the feature points of each image in the group, perform feature point matching on all images in the group to obtain two-dimensional image feature point structure information; triangulate according to the calibrated camera parameters and image feature point matching to obtain the initial scene three-dimensional point structured information, initialize the three-dimensional position of the camera, and filter out abnormal three-dimensional data points through multi-view geometry; use structure from motion to optimize the three-dimensional structure of the to-be-cleaned H-shaped steel structure scene after filtering.

[0074] Step S50: According to the three-dimensional structure of the to-be-cleaned H-shaped steel structure, through the PLC control strategy, plan the cleaning path of the to-be-cleaned H-shaped steel structure and complete the cleaning according to the cleaning path of the to-be-cleaned H-shaped steel structure.

[0075] It should be understood that the internal working mode of the PLC generally adopts the cyclic scanning working mode, and the interrupt working mode is added in some large and medium-sized PLCs. After the user has debugged the user program, write the program into the PLC memory through the programmer, and at the same time connect the on-site input signals and the corresponding controlled actuators to the input end of the input module and the output end of the output module respectively. Then select the running working mode for the PLC, and the subsequent work will be completed by the PLC according to the user program. The overview diagram is the block diagram of the PLC execution process. During the working process of the PLC, six modules are mainly processed.

[0076] In a specific implementation, according to the three-dimensional structure of the to-be-cleaned H-shaped steel structure, plan the cleaning path of the to-be-cleaned H-shaped steel structure by the method of combining artificial intelligence and robots; correct the cleaning path through the PLC control strategy, and according to the corrected cleaning path of the to-be-cleaned H-shaped steel structure, clean the to-be-cleaned H-shaped steel structure by a laser.

[0077] It should be understood that since the accurate cleaning path is obtained through the above method, the laser machine used here has upgraded the traditional 200mm line width laser to a 6000W laser head with a 300mm line width, greatly improving the cleaning efficiency.

[0078] In this embodiment, the H-shaped steel structure to be cleaned is three-dimensionally scanned by a binocular camera to obtain the scanned point cloud data; the point cloud data is modeled to obtain the point cloud data model of the H-shaped steel structure to be cleaned; the spatial coordinates on the surface of the H-shaped steel structure to be cleaned are determined according to the point cloud data model; triangulation is performed according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; according to the three-dimensional structure of the H-shaped steel structure to be cleaned, through the PLC control strategy, the cleaning path of the H-shaped steel structure to be cleaned is planned and the cleaning is completed according to the cleaning path of the H-shaped steel structure to be cleaned, so as to determine the three-dimensional structure of the H-shaped steel structure to be cleaned according to the H-shaped steel structure to be cleaned, and determine the cleaning path of the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned, and then complete the cleaning through a laser according to the cleaning path of the H-shaped steel structure to be cleaned.

[0079] Refer to Figure 3 , Figure 3 which is a schematic flowchart of the second embodiment of the H-shaped steel structure cleaning method of the present invention. Based on the above Figure 2 shown first embodiment, the second embodiment of the H-shaped steel structure cleaning method of the present invention is proposed.

[0080] In the second embodiment, the step S10 includes:

[0081] Step S101: Obtain the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned respectively.

[0082] It should be noted that a binocular camera is used here to obtain the left-eye view and the right-eye view. The binocular camera is installed on the robotic arm and can scan the H-shaped steel structure in all directions as the robotic arm moves, and obtain two different views on the left and right respectively.

[0083] Step S102: Perform convolution operations with the same parameters on the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned respectively to obtain a left feature map and a right feature map.

[0084] It should be noted that the convolution operation step size range is that the step size is greater than 4 and does not include the maximum pooling operation. The maximum pooling operation is not set here to prevent the feature position from drifting, so that the next operation step can learn effective disparity information. In addition, the convolution operation step size range is set large, and the convolution operation steps with a convolution step greater than two are used to gradually reduce the resolution of the received left-eye view and right-eye view, so as to reduce the size of the training data set in the subsequent operation steps and speed up the calculation speed of the network training process.

[0085] Step S103: Combine the left feature map and the right feature map to obtain a sequence image of two frames, perform a 3D convolution operation on the sequence image to obtain a disparity feature map, and obtain the point cloud data of the H-shaped steel structure to be cleaned based on the disparity feature map.

[0086] In a specific implementation, perform a 3D convolution operation on the sequence image; flatten the disparity feature map after the 3D convolution operation into a two-dimensional vector, perform a fully connected operation on the two-dimensional vector, and rearrange the fully connected two-dimensional vector to obtain a one-channel three-dimensional feature image; obtain the left view or the right view again, and perform a 3D convolution operation on the obtained left view or right view; perform a channel combination on the one-channel three-dimensional feature image and the left view or the right view received again after the 3D convolution operation to obtain a combined image; perform a deconvolution operation on the combined image after the 3D convolution operation to obtain a disparity feature map.

[0087] It should be understood that the disparity information of the left view and the right view is learned through the 3D convolution operation. Among them, the left feature map and the right feature map are combined to obtain a sequence image of two frames. The specific method of this operation is: combine the left feature map and the right feature map in the frame number dimension, that is, add one dimension of the frame number dimension to make the left feature map and the right feature map form a sequence image of two frames. The number of times of the above 3D convolution operation is greater than or equal to once and includes a max pooling operation. The purpose of this step is to continue to extract the features related to the disparity information in the disparity feature map through multiple 3D convolution operations. The max pooling operation can reduce the resolution of the disparity feature map, further reduce the size of the training data set in the subsequent operation steps, and speed up the calculation speed of the network training process. After performing multiple 3D convolution operations on the disparity feature map, a disparity feature map after the 3D convolution operation is obtained, and the resolution at the disparity feature map after the 3D convolution operation is the lowest in the entire network.

[0088] In this embodiment, the left view and the right view of the H-shaped steel structure to be cleaned are obtained respectively; the left view and the right view of the H-shaped steel structure to be cleaned are respectively subjected to convolution operations with the same parameters to obtain a left feature map and a right feature map; the left feature map and the right feature map are combined to obtain a sequence image of two frames, and the sequence image is subjected to a 3D convolution operation to obtain a disparity feature map, and point cloud data of the H-shaped steel structure to be cleaned is obtained according to the disparity feature map; the point cloud data is modeled to obtain a point cloud data model of the H-shaped steel structure to be cleaned; the spatial coordinates on the surface of the H-shaped steel structure to be cleaned are determined according to the point cloud data model; triangulation is performed according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; according to the three-dimensional structure of the H-shaped steel structure to be cleaned, through a PLC control strategy, the cleaning path of the H-shaped steel structure to be cleaned is planned and the cleaning is completed according to the cleaning path of the H-shaped steel structure to be cleaned, so as to perform three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain a left feature map and a right feature map through convolution operations with the same parameters, and perform processing to obtain target point cloud data.

[0089] Referring to Figure 4 , Figure 4 is a schematic flowchart of the third embodiment of the H-shaped steel structure cleaning method of the present invention. Based on the above Figure 2 shown in the first embodiment, the third embodiment of the H-shaped steel structure cleaning method of the present invention is proposed.

[0090] In the third embodiment, the step S20 includes:

[0091] Step S201: Establish a coarse-grained model according to the coarse-grained nature of the point cloud data, and construct a fine-grained model corresponding to the coarse-grained model according to the fine-grained nature of the point cloud data.

[0092] In specific implementation, a coarse-grained model is constructed using coarse-grained data, a fine-grained model corresponding to the coarse-grained model is constructed using the fine-grained data, a coarse-grained model is constructed using the coarse-grained data, and a fine-grained model corresponding to the coarse-grained model is constructed using the fine-grained data. By performing modeling on the coarse-grained data and the fine-grained data through parallel different tasks, compared with serial modeling, the time required for modeling can be shortened.

[0093] Step S202: Combine the coarse-grained model and the fine-grained model to obtain the point cloud data model.

[0094] It should be noted that by using the coarse-grained data and the corresponding fine-grained data in the point cloud data to construct a coarse-grained model and a fine-grained model respectively, since the amount of data in the coarse-grained model is small and the construction and parsing speeds are relatively fast, the coarse-grained model information is sent to the terminal after the coarse-grained model is generated first, so that the terminal displays the coarse-grained image to generate the fine-grained model, and the obtained coarse-grained model and fine-grained model are combined to obtain the point cloud data model.

[0095] In this embodiment, the H-shaped steel structure to be cleaned is three-dimensionally scanned by a binocular camera to obtain the scanned point cloud data; a coarse-grained model is established according to the coarse-grained properties of the point cloud data, and a fine-grained model corresponding to the coarse-grained model is constructed according to the fine-grained properties of the point cloud data; the coarse-grained model and the fine-grained model are combined to obtain the point cloud data model; the spatial coordinates on the surface of the H-shaped steel structure to be cleaned are determined according to the point cloud data model; triangulation is performed according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; according to the three-dimensional structure of the H-shaped steel structure to be cleaned, through the PLC control strategy, the cleaning path of the H-shaped steel structure to be cleaned is planned and the cleaning is completed according to the cleaning path of the H-shaped steel structure to be cleaned, so that the point cloud data model is obtained by combining the fine-grained model and the coarse-grained model, and the three-dimensional structure is obtained through the point cloud data model, and the cleaning path is planned.

[0096] In addition, an embodiment of the present invention also proposes a storage medium, on which an H-shaped steel structure cleaning program is stored, and when the H-shaped steel structure cleaning program is executed by a processor, the steps of the H-shaped steel structure cleaning method described above are implemented.

[0097] Since this storage medium can adopt the technical solutions of all the above embodiments, it at least has the beneficial effects brought by the technical solutions of the above embodiments, which will not be elaborated here one by one.

[0098] Refer to Figure 5 , Figure 5 which is a schematic diagram of the functional modules of the first embodiment of the H-shaped steel structure cleaning device of the present invention.

[0099] In the first embodiment of the H-shaped steel structure cleaning device of the present invention, the H-shaped steel structure cleaning device includes:

[0100] A scanning module 10, configured to three-dimensionally scan the H-shaped steel structure to be cleaned by a binocular camera to obtain the scanned point cloud data;

[0101] A modeling module 20, configured to perform modeling on the point cloud data to obtain the point cloud data model of the H-shaped steel structure to be cleaned.

[0102] A determination module 30, configured to determine the spatial coordinates of the surface of the H-shaped steel structure to be cleaned according to the point cloud data model.

[0103] A triangulation module 40, configured to perform triangulation according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned, so as to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

[0104] A cleaning module 50, configured to plan the cleaning path of the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned through a PLC control strategy, and complete the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned.

[0105] In this embodiment, the H-shaped steel structure to be cleaned is three-dimensionally scanned by a binocular camera to obtain the scanned point cloud data; the point cloud data is modeled to obtain the point cloud data model of the H-shaped steel structure to be cleaned; the spatial coordinates of the surface of the H-shaped steel structure to be cleaned are determined according to the point cloud data model; triangulation is performed according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; according to the three-dimensional structure of the H-shaped steel structure to be cleaned, through a PLC control strategy, the cleaning path of the H-shaped steel structure to be cleaned is planned and the cleaning is completed according to the cleaning path of the H-shaped steel structure to be cleaned, so as to determine the three-dimensional structure of the H-shaped steel structure to be cleaned according to the H-shaped steel structure to be cleaned, and determine the cleaning path of the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned, and then complete the cleaning through a laser according to the cleaning path of the H-shaped steel structure to be cleaned.

[0106] In one embodiment, the scanning module 10 is further configured to three-dimensionally scan the H-shaped steel structure to be cleaned by a binocular camera to obtain the scanned point cloud data, including:

[0107] Obtain the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned respectively;

[0108] Perform convolution operations with the same parameters on the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned respectively to obtain a left feature map and a right feature map;

[0109] Combine the left feature map and the right feature map to obtain a sequence image of two frames, and perform a 3D convolution operation on the sequence image to obtain a disparity feature map, and obtain the point cloud data of the H-shaped steel structure to be cleaned according to the disparity feature map.

[0110] In one embodiment, the scanning module 10 is further configured to combine the left feature map and the right feature map to obtain a sequence image of two frames, and perform a 3D convolution operation on the sequence image to obtain a disparity feature map, including:

[0111] Perform a 3D convolution operation on the sequence of images;

[0112] Flatten the disparity feature map after the 3D convolution operation into a two-dimensional vector, perform a fully connected operation on the two-dimensional vector, and rearrange the fully connected two-dimensional vector to obtain a one-channel three-dimensional feature image;

[0113] Obtain the left-eye view or the right-eye view again, and perform a 3D convolution operation on the obtained left-eye view or right-eye view;

[0114] Perform channel combination on the one-channel three-dimensional feature image and the left-eye view or right-eye view received again after the 3D convolution operation to obtain a combined image;

[0115] Perform a deconvolution operation on the combined image after the 3D convolution operation to obtain a disparity feature map.

[0116] In one embodiment, the modeling module 20 is further configured to model the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned, including:

[0117] Establish a coarse-grained model according to the coarse-grained properties of the point cloud data, and construct a fine-grained model corresponding to the coarse-grained model according to the fine-grained properties of the point cloud data;

[0118] Combine the coarse-grained model and the fine-grained model to obtain the point cloud data model.

[0119] In one embodiment, the determining module 30 is further configured to determine the spatial coordinates of the surface of the H-shaped steel structure to be cleaned according to the point cloud data model, including:

[0120] Set a reference three-dimensional coordinate system around the H-shaped steel structure to be cleaned according to the point cloud data model;

[0121] Obtain the three-dimensional coordinate points of several control points on the three-dimensional coordinate system;

[0122] Use the three-dimensional coordinate points of several control points on the three-dimensional coordinate system as reference nodes to establish the spatial position topological relationship between the points to be measured on the surface of the H-shaped steel structure to be cleaned and the several control points on the three-dimensional coordinate system;

[0123] According to the spatial position topological relationship, obtain the spatial coordinates of the points to be measured on the surface of the H-shaped steel structure to be cleaned in the reference three-dimensional coordinate system.

[0124] In one embodiment, the triangulation module 40 is further configured to perform triangulation based on the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned, so as to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned, including:

[0125] Extract key frames through binocular disparity according to the spatial coordinates on the surface of the three-dimensional structure of the H-shaped steel structure to be cleaned;

[0126] Group the key frames and use the structure from motion algorithm to extract representative frames from each group of key frames, and perform motion structure recovery on the representative frames;

[0127] Calibrate each group of key frames according to the structure from motion result, optimize each group of key frames, and obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

[0128] In one embodiment, the cleaning module 50 is further configured to plan the cleaning path of the H-shaped steel structure to be cleaned according to the three-dimensional structure of the H-shaped steel structure to be cleaned through a PLC control strategy, and complete the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned, including:

[0129] Plan the cleaning path of the H-shaped steel structure to be cleaned through a method combining artificial intelligence and a robot according to the three-dimensional structure of the H-shaped steel structure to be cleaned;

[0130] Modify the cleaning path through a PLC control strategy, and clean the H-shaped steel structure to be cleaned through a laser according to the modified cleaning path of the H-shaped steel structure to be cleaned.

[0131] Other embodiments or specific implementation manners of the H-shaped steel structure cleaning device of the present invention may refer to the above method embodiments, and thus at least have all the beneficial effects brought by the technical solutions of the above embodiments, which will not be elaborated here.

[0132] It should be noted that in this article, the terms "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0133] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments. Among the several device unit claims listed, several of these devices may be specifically embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order, and these words may be interpreted as names.

[0134] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a Read Only Memory image (ROM) / Random Access Memory (RAM), magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0135] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A cleaning method for H-shaped steel structures, characterized in that, The method includes the following steps: Performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data; Modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned; Determining the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model; Performing triangulation according to the point cloud data model and the spatial coordinates on the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; According to the three-dimensional structure of the H-shaped steel structure to be cleaned, through a PLC control strategy, planning the cleaning path of the H-shaped steel structure to be cleaned and completing the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned; The performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain the scanned point cloud data includes: Respectively obtaining the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned; Performing convolution operations with the same parameters on the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned respectively to obtain a left feature map and a right feature map; Combining the left feature map and the right feature map to obtain a sequence image of two frames, and performing a 3D convolution operation on the sequence image to obtain a disparity feature map, and obtaining the point cloud data of the H-shaped steel structure to be cleaned according to the disparity feature map; The combining the left feature map and the right feature map to obtain a sequence image of two frames, and performing a 3D convolution operation on the sequence image to obtain a disparity feature map includes: Performing a 3D convolution operation on the sequence image; Flattening the disparity feature map after the 3D convolution operation into a two-dimensional vector, performing a fully connected operation on the two-dimensional vector, and rearranging the fully connected two-dimensional vector to obtain a one-channel three-dimensional feature image; Obtaining the left-eye view or the right-eye view again, and performing a 3D convolution operation on the obtained left-eye view or right-eye view; Performing channel combination on the one-channel three-dimensional feature image and the left-eye view or the right-eye view received again after the 3D convolution operation to obtain a combined image; Performing a deconvolution operation on the combined image after the 3D convolution operation to obtain a disparity feature map.

2. The method according to claim 1, wherein The modeling the point cloud data to obtain a point cloud data model of the H-shaped steel structure to be cleaned includes: Establishing a coarse-grained model according to the coarse-grained properties of the point cloud data, and constructing a fine-grained model corresponding to the coarse-grained model according to the fine-grained properties of the point cloud data; Combining the coarse-grained model and the fine-grained model to obtain the point cloud data model.

3. The method according to claim 1, characterized in that, The determining the spatial coordinates on the surface of the H-shaped steel structure to be cleaned according to the point cloud data model includes: Setting a reference three-dimensional coordinate system around the H-shaped steel structure to be cleaned according to the point cloud data model; Obtaining the three-dimensional coordinate points of several control points on the three-dimensional coordinate system; Taking the three-dimensional coordinate points of several control points on the three-dimensional coordinate system as reference nodes, and establishing the spatial position topological relationship between the measurement points on the surface of the H-shaped steel structure to be cleaned and several control points on the three-dimensional coordinate system in the three-dimensional coordinate system; According to the spatial position topological relationship, obtain the spatial coordinates of the measurement points on the surface of the H-shaped steel structure to be cleaned in the reference three-dimensional coordinate system.

4. The method according to claim 1, characterized in that, Performing triangulation based on the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned, including: Extract key frames through binocular parallax according to the spatial coordinates on the surface of the three-dimensional structure of the H-shaped steel structure to be cleaned; Group the key frames and use the structure from motion algorithm to extract representative frames from each group of key frames and perform motion structure recovery on the representative frames; Calibrate each group of key frames according to the structure from motion results, optimize each group of key frames, and obtain the three-dimensional structure of the H-shaped steel structure to be cleaned.

5. The method according to claim 1, wherein According to the three-dimensional structure of the H-shaped steel structure to be cleaned, through the PLC control strategy, plan the cleaning path of the H-shaped steel structure to be cleaned and complete the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned, including: Plan the cleaning path of the H-shaped steel structure to be cleaned by combining artificial intelligence and robots according to the three-dimensional structure of the H-shaped steel structure to be cleaned; Correct the cleaning path through the PLC control strategy, and clean the H-shaped steel structure to be cleaned by a laser according to the corrected cleaning path of the H-shaped steel structure to be cleaned.

6. A cleaning device for H-shaped steel structures, characterized in that, The H-shaped steel structure cleaning device includes: A scanning module for performing three-dimensional scanning on the H-shaped steel structure to be cleaned through a binocular camera to obtain scanned point cloud data; A modeling module for modeling the point cloud data to obtain the point cloud data model of the H-shaped steel structure to be cleaned; A determination module for determining the spatial coordinates of the surface of the H-shaped steel structure to be cleaned according to the point cloud data model; A triangulation module for performing triangulation according to the point cloud data model and the spatial coordinates of the surface of the H-shaped steel structure to be cleaned to obtain the three-dimensional structure of the H-shaped steel structure to be cleaned; A cleaning module for planning the cleaning path of the H-shaped steel structure to be cleaned through the PLC control strategy and completing the cleaning according to the cleaning path of the H-shaped steel structure to be cleaned; The cleaning module is further configured to respectively obtain the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned; perform convolution operations with the same parameters on the left-eye view and the right-eye view of the H-shaped steel structure to be cleaned to obtain a left feature map and a right feature map; combine the left feature map and the right feature map to obtain a sequence image of two frames, and perform a 3D convolution operation on the sequence image to obtain a disparity feature map, and obtain the point cloud data of the H-shaped steel structure to be cleaned according to the disparity feature map; The cleaning module is further configured to perform 3D convolution operation on the sequence of images; flatten the disparity feature map after the 3D convolution operation into a two-dimensional vector, perform a fully connected operation on the two-dimensional vector, and rearrange the fully connected two-dimensional vector to obtain a one-channel three-dimensional feature image; acquire the left-eye view or the right-eye view again, and perform 3D convolution operation on the acquired left-eye view or right-eye view; perform channel combination on the one-channel three-dimensional feature image and the left-eye view or right-eye view that is received again after the 3D convolution operation to obtain a combined image; perform deconvolution operation on the combined image after the 3D convolution operation to obtain a disparity feature map.

7. A cleaning device for H-shaped steel structures, characterized in that, The H-shaped steel structure cleaning device includes a memory, a processor, and an H-shaped steel structure cleaning program stored on the memory and executable on the processor. When the H-shaped steel structure cleaning program is executed by the processor, it implements the H-shaped steel structure cleaning method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, An H-shaped steel structure cleaning program is stored on the storage medium. When the H-shaped steel structure cleaning program is executed by a processor, it implements the H-shaped steel structure cleaning method according to any one of claims 1 to 5.

Images