Cover surface welding device and control method and control system thereof

Abstract

The invention discloses a capping welding device and a control method and system thereof. The method comprises the steps that the midpoint of a light beam emitted by a laser emitter is controlled to move to the center of a welding seam, welding is conducted, and an auxiliary light source is turned on under the condition that the welding seam is welded to a cover face; the laser module is controlled to collect a surface image of the cover face, the surface image is subjected to cover face segmentation, two cover face weld bead images are obtained, and the number of the cover face weld beads is two; and effective feature points are determined according to the cosmetic welding bead image, coordinate transformation is conducted on the effective feature points, a target coordinate point of the crawling device is obtained, and the crawling device is controlled to move according to the target coordinate point so as to weld the cosmetic. According to the method, the technical problem that the capping feature points are difficult to extract for tracking due to factors such as unobvious laser stripe features, influence of surface pits or other impurities, position change of a molten pool influenced by gravity during welding and the like in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of welding robots, and in particular to a cover welding device and a control method thereof, a computer-readable storage medium and a control system. Background Art

[0002] During the welding process, there are often problems such as the welding gun deviating from the center of the weld, the weld bead offset, and welding defects. Especially in cap welding, the existing weld tracking methods face challenges due to factors such as the complex surface morphology of the weld bead, unclear laser stripe characteristics, and the constant change of the position of the molten pool due to gravity. Cap welding is usually used as the last step after welding is completed. The main purpose is to form a smooth and beautiful covering layer on the weld surface. It is usually performed on the top of the weld. In recent years, the application of sensor technology has proposed a variety of sensor technologies in the field of robotic welding. Among them, laser sensors have become an ideal choice for welding control due to their simple structure, large amount of information, high precision and good stability.

[0003] In one prior art, the weld bead features before the capping are identified and feedback is provided in real time to adjust the capping weld bead path and weld tracking; in another prior art, a profiling plate corresponding to the quarter arc shape and curvature of the tank body is designed to control the walking track of the trolley. However, in the existing capping welding technology, there are problems such as unclear laser stripe features, the influence of surface pits or other impurities, and the position change of the weld bead due to gravity, which leads to a decrease in welding tracking accuracy.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a cover surface welding device and its control method, a computer-readable storage medium and a control system, so as to at least solve the technical problem in the related art that it is difficult to extract the cover surface feature points for tracking due to factors such as unclear laser stripe features, the influence of surface pits or other impurities, and the position change of the molten pool due to gravity during welding.

[0006] According to one aspect of an embodiment of the present invention, a cover surface welding device is provided, which at least includes a laser module for capturing a surface image of the cover surface, wherein the cover surface is the surface of the weld during the welding process, and the laser module includes: a laser emitter for emitting laser to irradiate the surface of the cover surface; an auxiliary light source having the same light projection angle as that of the laser emitter, and being used to assist the laser emitter in irradiating the surface of the cover surface; an image collector, disposed directly above the weld and spaced apart from the weld surface by a first predetermined distance, and being used to capture an image under the illumination of the laser emitter and the auxiliary light source; and a filter, disposed directly below the image collector and spaced apart from the image collector by a second predetermined distance, and being used to select light of a target band, wherein the target band is the same as the band of light emitted by the auxiliary light source, and the first predetermined distance is greater than the second predetermined distance.

[0007] Optionally, the cover welding device also includes: a welding tool for welding the weld; a crawling device, on which the welding tool and the laser module are installed, and the laser module is arranged in front of the welding tool, the front being the forward direction of the crawling device, and the crawling device is used to drive the welding tool and the laser module to crawl to complete the welding of the weld.

[0008] According to another aspect of an embodiment of the present invention, a control method for a cover welding device is also provided, comprising: controlling the midpoint of the light beam emitted by the laser emitter to move to the center of the weld to perform welding, and turning on an auxiliary light source when welding to the cover, wherein the cover is the surface of the weld during the welding process; controlling the laser module to capture a surface image of the cover, performing cover segmentation on the surface image to obtain a cover weld bead image, wherein the cover weld is a welding path of the cover and there are two of them; determining effective feature points according to the cover weld bead image, performing coordinate conversion on the effective feature points to obtain target coordinate points of a crawling device, and controlling the crawling device to move according to the target coordinate points to weld the cover.

[0009] Optionally, determining effective feature points according to the cover weld image includes: determining multiple fusion points of each cover weld in the cover weld image, fitting the multiple fusion points in each cover weld to obtain a fitting curve; determining effective points corresponding to each cover weld according to the fitting curve, and determining the effective feature points based on the effective points.

[0010] Optionally, determining the effective points corresponding to each of the cover welds according to the fitting curve, and determining the effective feature points according to the effective points, includes: deleting the fusion points whose distance from the fitting curve is greater than a preset distance threshold, and determining the remaining fusion points as the effective points corresponding to each of the cover welds; when the number of effective points corresponding to each of the cover welds is greater than a preset number threshold, calculating the midpoints of the effective points to obtain the midpoint of the weld corresponding to each of the cover welds, and obtaining two midpoints of the welds; connecting the two midpoints of the welds to obtain a weld line, calculating the midpoint of the weld line to obtain the effective feature points.

[0011] Optionally, before performing cover segmentation on the surface image, the control method of the cover welding device also includes: marking a cover weld bead area in the surface image to obtain a cover weld bead area image; converting the cover weld bead area image into a mask image, and performing data expansion on the mask image.

[0012] Optionally, the surface image is segmented to obtain a cover weld image, including: analyzing the surface image through a deep learning model to obtain the cover weld image, wherein the deep learning model is trained by multiple groups of data, and each group of the multiple groups of data includes: historical surface images and cover weld image labels.

[0013] Optionally, the effective feature points are subjected to coordinate transformation to obtain target coordinate points of the crawling device, including: acquiring a posture transformation matrix of the laser module and the crawling device; and calculating the target coordinate points corresponding to the crawling device according to the effective feature points and the posture transformation matrix.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the control methods of the cover welding device described above.

[0015] According to another aspect of an embodiment of the present invention, a control system for a cover welding device is also provided, comprising: a cover welding device as described in any one of the above; a controller, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a control method for executing any one of the cover welding devices described in the above.

[0016] In an embodiment of the present invention, the midpoint of the laser emitter's emitted light beam is controlled to move to the center of the weld to perform welding, and the auxiliary light source is turned on when welding to the cover surface, wherein the cover surface is the surface of the weld during the welding process; the laser module is controlled to collect the surface image of the cover surface, and the surface image is segmented to obtain the cover surface weld bead image, wherein the cover surface weld bead is the welding path of the cover surface and there are two; the effective feature points are determined according to the cover surface weld bead image, the effective feature points are converted into coordinates, the target coordinate points of the crawling device are obtained, and the crawling device is controlled to move according to the target coordinate points to weld the cover surface. Through the technical solution provided by the present invention, the purpose of high-definition acquisition of complex surfaces by the laser sensor system, analysis and processing of images using a deep learning model, and at the same time, the effective feature points are extracted and the position of the welding tool is corrected in real time, thereby achieving the technical effect of higher precision in the cover surface tracking during the welding process, improving the accuracy and stability of tracking, and thus solving the technical problem in the related art that it is difficult to extract the cover surface feature points for tracking due to factors such as unclear laser stripe features, surface pits or other impurities, and changes in the position of the molten pool due to gravity during welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a schematic structural diagram of a laser module according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of a cover welding device according to an embodiment of the present invention;

[0020] Figure 3 It is a hardware structure block diagram of a mobile terminal of a control method of a cover welding device according to an embodiment of the present invention;

[0021] Figure 4 is a flow chart of a control method of a cover welding device according to an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of feature points of a cover area according to an embodiment of the present invention;

[0023] Figure 6 is a flowchart of cover image post-processing according to an embodiment of the present invention;

[0024] Figure 7 is a flow chart of a control method of an optional cap welding device according to an embodiment of the present invention;

[0025] Figure 8is a schematic diagram of a control device for a cover welding device according to an embodiment of the present invention.

[0026] The above drawings include the following reference numerals:

[0027] 102. Processor; 104. Memory; 106. Transmission device; 108. Input and output device; 11. Laser module; 12. Welding tool; 13. Crawling device; 14. Laser transmitter; 15. Auxiliary light source; 16. Image collector; 17. Filter. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] As introduced in the background technology, in the related art, it is difficult to extract the cover feature points for tracking due to factors such as unclear laser stripe features, the influence of surface pits or other impurities, and the position change of the molten pool due to gravity during welding. In view of the above defects, in the embodiments of the present invention, a cover welding device and a control method thereof, a computer-readable storage medium and a control system are provided.

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] According to one aspect of an embodiment of the present invention, a cover surface welding device is provided, which at least includes a laser module. The structural schematic diagram of the laser module is as shown in FIG. Figure 1As shown, the laser module may include: a laser emitter 14, an auxiliary light source 15, an image collector 16 and a filter 17, wherein the laser emitter 14 is used to emit laser to irradiate the surface of the cover; the auxiliary light source 15 has the same light projection angle as the laser emitter 14, and is used to assist the laser emitter 14 in irradiating the surface of the cover; the image collector 16 is arranged directly above the weld and is spaced apart from the weld surface by a first predetermined distance, and is used to capture an image under the irradiation of the laser emitter 14 and the auxiliary light source 15; the filter 17 is arranged directly below the image collector 16 and is spaced apart from the image collector 16 by a second predetermined distance, and is used to select light of a target band, the target band is the same as the band of light emitted by the auxiliary light source 15, and the first predetermined distance is greater than the second predetermined distance.

[0033] Figure 2 Schematic diagram of a cover welding device according to an embodiment of the present invention. Figure 2 As shown, the cover surface welding device may include: a laser module 11, a welding tool 12 and a crawling device 13. The cover surface welding device is described in detail below.

[0034] The laser module 11 is used to collect the surface image of the cover surface, where the cover surface is the surface of the weld during the welding process.

[0035] It should be noted that the above-mentioned cover refers to the layer when the weld is welded to the outermost surface, that is, the outermost layer of the weld. A laser sensor is installed on the crawling robot (i.e., the crawling device 13) vertically to the surface of the workpiece. The installation height must ensure that the camera imaging in the laser sensor is clear, and ensure that the sensor is located at a distance in front of the welding tool 12. After the installation is completed, turn on the laser light source and project a straight beam on the groove surface of the unwelded workpiece. The midpoint of the laser scanning beam of the laser sensor is adjusted to be located at the exact center of the groove. When welding to the cover layer, turn on the auxiliary light source 15.

[0036] The laser sensor (i.e., laser module 11) is composed of a laser emitter 14, an industrial camera (image collector 16), a filter 17, and an auxiliary light source 15. The auxiliary light source 15 should be consistent with the wavelength of the filter 17. The auxiliary light source 15 is installed in the laser module 11 at a certain angle, and the difference between α and β does not exceed 10° (the actual installation angle needs to be related to the auxiliary projection area and should be consistent with the projection angle of the laser emitter 14 as much as possible). The projected light source needs to be directly below the camera's field of view.

[0037] The welding tool 12 is used for welding the weld.

[0038] It should be noted that the welding tool 12 may refer to a welding gun and a laser cladding head required for various welding processes.

[0039] The crawling device 13, the welding tool 12 and the laser module 11 are installed on the crawling device 13, the laser module 11 is arranged in front of the welding tool 12, and the front is the forward direction of the crawling device 13. The crawling device 13 is used to drive the welding tool 12 and the laser module 11 to crawl to complete the welding of the weld.

[0040] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 3 FIG. 1 is a hardware structure block diagram of a mobile terminal of a control method for a cover welding device according to an embodiment of the present invention. Figure 3 As shown, the mobile terminal may include one or more ( Figure 3 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 3 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown.

[0041] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the control method of the mobile robot in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] According to an embodiment of the present invention, a method embodiment of a control method for a cover welding device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Figure 4 is a flow chart of a control method of a cover welding device according to an embodiment of the present invention. Figure 4 As shown, the method comprises the following steps:

[0044] Step S201, control the midpoint of the laser emitter to move to the center of the weld to perform welding, and turn on the auxiliary light source when welding to the cover surface, wherein the cover surface is the surface of the weld during the welding process.

[0045] In this embodiment, the midpoint of the laser scanning beam of the laser sensor can be adjusted to be located at the exact center of the groove, which can improve the accuracy of cover surface position detection. During welding, the specific position of the cover surface can be determined based on the position of the laser scanning beam, thereby achieving the purpose of tracking the cover surface. When welding to the cover surface layer, the auxiliary light source is turned on to better capture the image information of the cover surface, thereby further improving the accuracy and stability of cover surface tracking.

[0046] Step S202, controlling the laser module to collect a surface image of the cover surface, and performing cover surface segmentation on the surface image to obtain a cover surface weld bead image, wherein the cover surface weld bead is a welding path of the cover surface and there are two of them.

[0047] In this embodiment, when welding to the cover layer, the crawling robot can be moved to control the laser sensor to collect the original image of the cover groove of the workpiece, and then the cover image is segmented to obtain images of the two welding paths of the cover, so as to more accurately identify the welding path and improve the welding quality and efficiency.

[0048] Step S203, determining effective feature points according to the cover weld image, performing coordinate conversion on the effective feature points to obtain target coordinate points of the crawler device, and controlling the crawler device to move according to the target coordinate points to weld the cover.

[0049] In this embodiment, the feature points required for cover surface tracking can be determined through the fusion lines on both sides of the cover surface, the extracted feature points can be filtered to retain valid feature points, and the crawling robot can be controlled to move according to the target coordinate points to weld the cover surface. Through image recognition and feature point extraction, the welding target point can be determined and automatic control can be realized, thereby achieving precise welding of the cover surface.

[0050] As can be seen from the above, in an embodiment of the present invention, the midpoint of the light beam emitted by the laser emitter is controlled to move to the center of the weld to perform welding, and when welding to the cover surface, the auxiliary light source is turned on, wherein the cover surface is the surface of the weld during the welding process; the laser module is controlled to capture the surface image of the cover surface, and the surface image is segmented to obtain the cover surface weld bead image, wherein the cover surface weld bead is the welding path of the cover surface and there are two of them; the effective feature points are determined according to the cover surface weld bead image, the effective feature points are converted into coordinates, the target coordinate points of the crawling device are obtained, and the crawling device is controlled to move according to the target coordinate points to weld the cover surface, thereby achieving the purpose of high-definition acquisition of complex surfaces by the laser sensor system, analyzing and processing images by using a deep learning model, and at the same time, extracting effective feature points and correcting the position of the welding tool in real time, thereby achieving the technical effect of higher precision in cover surface tracking during welding, and improving the tracking accuracy and stability.

[0051] The above-mentioned technical solution provided by the embodiment of the present invention solves the technical problem in the related art that it is difficult to extract the cover feature points for tracking due to factors such as unclear laser stripe features, the influence of surface pits or other impurities, and the change of position of the molten pool due to gravity during welding.

[0052] According to the above embodiment of the present invention, determining effective feature points based on the cover weld bead image includes: determining multiple fusion points of each cover weld bead in the cover weld bead image, fitting the multiple fusion points in each cover weld bead to obtain a fitting curve; determining effective points corresponding to each cover weld bead according to the fitting curve, and determining effective feature points based on the effective points.

[0053] In this embodiment, feature points can be extracted, and fusion lines on both sides of the cover (i.e., multiple fusion points of each cover weld) can be extracted according to the cover weld image after image processing. Straight lines are fitted to the fusion lines on both sides, which can more accurately analyze the shape and quality of the cover weld, determine the effective points corresponding to each cover weld, and thus determine the effective feature points, so as to screen out welding points with better quality and exclude abnormal points and noise points, so as to better understand the key features and parameters in the welding process and improve the reliability and stability of welding quality.

[0054] In the above embodiment of the present invention, the valid points corresponding to each cover weld bead are determined according to the fitting curve, and the valid feature points are determined according to the valid points, including: deleting the fusion points whose distance from the fitting curve is greater than a preset distance threshold, and determining the remaining fusion points as the valid points corresponding to each cover weld bead; when the number of valid points corresponding to each cover weld bead is greater than the preset number threshold, calculating the midpoint of the valid points to obtain the midpoint of the weld bead corresponding to each cover weld bead, and obtaining two weld bead midpoints; connecting the two weld bead midpoints to obtain a weld bead connecting line, calculating the midpoint of the weld bead connecting line, and obtaining the valid feature points.

[0055] It should be noted that the above-mentioned effective characteristic point is the midpoint of the effective points on both sides of the fusion line of the weld edge.

[0056] Figure 5 is a schematic diagram of feature points of a cover area according to an embodiment of the present invention. Figure 5 As shown, the points on both sides that deviate from the fitting line by a certain threshold, that is, the fusion points whose distance from the fitting curve is greater than the preset distance threshold, are filtered out, and the remaining points are used as the valid points of the fusion lines on both sides. The mean values ​​in the X and Y directions are calculated based on the midpoints on both sides as the valid feature points, and the image valid feature points are scaled to the original size according to the input image size.

[0057] According to the above embodiment of the present invention, before performing cover segmentation on the surface image, the control method of the cover welding device also includes: marking the cover weld bead area in the surface image to obtain a cover weld bead area image; converting the cover weld bead area image into a mask image, and performing data expansion on the mask image.

[0058] In this embodiment, the original cover image can be preprocessed to mark the cover welding area, convert it into a cover area mask image, and perform data expansion processing to improve the image quality, so as to facilitate more accurate identification of the weld area in the subsequent period. By performing data expansion to increase the number and diversity of training samples, the generalization ability and recognition accuracy of the model can be improved.

[0059] Optionally, the above preprocessing may include: face cover area annotation, mask image conversion and image data enhancement.

[0060] Specifically, the above-mentioned cover area annotation includes the fusion lines on both sides of the cover weld edge, and the cover weld area is annotated with labelme to obtain the json file of the cover weld, and the json file is converted into a cover area mask image (the cover area is (0, 0, 255), and the background area is (0, 0, 0)) through the json conversion algorithm, wherein the data enhancement algorithm randomly blocks, translates and flips the original cover image in the horizontal direction, and the data set is the expanded image including the original image and the image after random occlusion, translation and flipping in the horizontal direction, and the corresponding mask image is also subjected to data expansion.

[0061] According to the above embodiment of the present invention, the surface image is segmented to obtain the cover weld image, including: analyzing the surface image through a deep learning model to obtain the cover weld image, wherein the deep learning model is obtained by training multiple groups of data, and each group of the multiple groups of data includes: historical surface images and cover weld image labels.

[0062] In this embodiment, a deep learning model trained by multiple sets of data is used to analyze and segment the preprocessed lid image. By learning the features and patterns in the image through a large amount of training data, the lid image can be accurately analyzed and segmented, and the various parts of the lid can be more accurately identified, and further analyzed and processed, thereby achieving more accurate data extraction and application.

[0063] After the above-mentioned expanded images are reconstructed into 512*512, they are put into the deep learning model for training. The model training steps are as follows: the above-mentioned deep learning model is the deeplabv3+ model. First, the expanded data set is divided into training set, validation set and test set in a ratio of 8:1:1 for model training. The model uses the cross entropy loss function and the dice coefficient as the total loss function, sets the weight ratio of two different loss functions, and saves the weight file of each round of training. After each batch of training is completed, the generalization performance of the batch model is verified using the validation set. The weight ratio of the loss function is determined by minimizing the validation set loss and the total loss. During the training process, the segmentation results can be evaluated based on the accuracy and intersection-over-union ratio. It is necessary to ensure that the accuracy and intersection-over-union ratio of the weld background and the cover foreground are not less than 90%, which is considered to meet the convergence conditions.

[0064] Specifically, it is the mmsegmentation model framework in the SenseTime model framework. The deeplabv3+ segmentation model is used in mmsegmentation. The model is optimized using the stochastic gradient SGD optimizer. The initial learning rate is 0.01, the weight decay is 0.005, the momentum is 0.9, and the polynomial function is used to adjust the learning rate. Among them, lr is the current learning rate, initial_lr is the initial learning rate, iter is the current number of iterations, max_iter is the maximum number of iterations, and p is the exponent of the polynomial. The learning rate is reduced according to the number of training rounds or iterations to achieve learning rate decay during training. In this way, a total of 20,000 (20k) epochs are trained, and the validation set is used for validation every 2,000 times until the model with the optimal weight ratio is obtained after 20k epochs, and the training is completed.

[0065] Figure 6 is a flowchart of post-processing of a cover image according to an embodiment of the present invention. Figure 6As shown in the figure, it is necessary to perform image post - processing on the segmented cover surface image, including grayscale processing, binarization processing, morphological opening operation, and small - area removal. Among them, small - area removal is achieved by finding the number of connected components in the grayscale image, and removing the connected components with an area smaller than the minimum area threshold, so as to reduce the influence of arc splash. Specifically, it includes: cover surface tracking, using a deep - learning segmentation model to segment the cover surface weld bead, image filtering - removing small - area regions, starting from the first row as the starting row n, traversing the columns to determine the two - side fusion points. Judge If n < img.rows; n += n. In the case of satisfaction, save the two - side fusion points. In the case of dissatisfaction, fit the straight line of the two - side fusion points, set the threshold T1 to remove the deviated points, set the threshold T2 for the number of reserved points. When the number of reserved points does not satisfy being greater than T2, switch to the straight - line mode; in the case of satisfaction, calculate the mean value of all points, and save the x - coordinate value as the feature point. After the Y - coordinate is fixed, += 1 for each frame, output the pixel coordinates (x, y), and finally obtain the feature - point welding torch coordinates.

[0066] According to the above - mentioned embodiments of the present invention, coordinate transformation is performed on the effective feature points to obtain the target coordinate points of the crawling device, including: obtaining the pose transformation matrix of the laser module and the crawling device; calculating the target coordinate points corresponding to the crawling device according to the effective feature points and the pose transformation matrix.

[0067] In this embodiment, by obtaining the pose transformation matrix of the laser module and the crawling device to describe their relative positions and pose relationships in space, registration and alignment between them can be realized, so as to achieve accurate data acquisition and processing. Calculating the target coordinate points corresponding to the crawling device according to the effective feature points and the pose transformation matrix can be used to achieve target positioning and tracking. By identifying effective feature points and calculating the target coordinate points, accurate positioning and tracking of the target can be realized.

[0068] Specifically, the laser sensor and the crawling robot have a fixed pose relationship, and the poses of the welding tool and the crawling robot can be obtained in real time. Through coordinate transformation using the rotation matrix, the X, Y, and Z of the effective feature points are obtained. Timing the processing time of the algorithm for a single cover - surface image, the calculation time of the effective feature points is 70 - 80 ms, and the welding tool is corrected in real time according to the calculated effective feature points.

[0069] Figure 7 is a flowchart of an optional control method for a cover - surface welding device according to an embodiment of the present invention, as Figure 7 shown, a laser sensor system is built by combining a laser sensor and an auxiliary light source, and beam debugging is carried out. Pre - processing, deep - learning model training, model prediction of the cover - surface image, and post - processing operations are performed on the collected cover - surface image. By effectively filtering the cover - surface feature points, effective feature points are extracted, and combined with the pose information of the crawling robot, the position of the welding tool is corrected in real time.

[0070] Through the technical solution provided by the above embodiment of the present invention, the laser sensor carries an auxiliary light source consistent with the filter band, and the auxiliary light source is fixed outside the laser sensor and is consistent with the projection direction of the line laser emitter. Since the projection area of ​​the auxiliary light source is an area, it can be tilted at a certain angle to the line laser emitter. When welding to the cover layer, the auxiliary light source is turned on to obtain more weld image information to avoid the problem that the line laser information is too little to distinguish the cover feature information. A deep learning method using a semantic segmentation image algorithm is used, and the deeplabv3+ segmentation algorithm is taken as an example. The data set includes the collected cover image, mask image conversion, data-enhanced cover image, and mask image. The trained model is used to segment and predict the cover area, and the cover feature map is output. The output feature map is post-processed and the effective feature points of the cover position are extracted to remove the uneven edge interference caused by splashes and arc light, and the feature points are scaled to the original image size. The following problems were effectively solved: 1) The deformation of the line laser at the cover position is not obvious, and it is easily affected by noise such as splashes and reflections. The feature points required for the crawling robot to track the cover are difficult to extract, and the laser tracking accuracy is not high; 2) During the welding process, the surface of the cover weld is complex, and the position of the molten pool is constantly changing due to gravity. The line laser module only judges the groove information through the feature points on the laser stripes, and the amount of information collected is small, so the cover features cannot be distinguished.

[0071] As can be seen from the above, in the embodiment of the present invention, a laser sensor system is built by combining a laser sensor and an auxiliary light source, and the beam is debugged to ensure that the laser stripes are projected on the surface of the welding workpiece and are clearly visible, which effectively solves the problem of unclear laser stripe features, realizes high-definition acquisition of complex surfaces in the process of cap welding, and effectively extracts key features in the welding process. A deep learning model is introduced, and the Deeplabv3+ model is used to segment the cap image. The learning rate is adjusted by a polynomial function to improve the convergence and training effect of the model, and the optimal model weight is obtained to identify the weld cap area. At the same time, image post-processing is performed, including morphological operations and small area removal, to cope with changes in welding position and the complexity of the welding surface, thereby improving the tracking accuracy and solving the problem of surface pits or other impurities affecting the position change of the molten pool due to gravity during welding. By effectively filtering out the cap feature points, effective feature points are extracted, and combined with the posture information of the crawling robot, the algorithm is timed for the processing time of a single cap image. The calculation time of the effective feature points is 70-80ms, which meets the real-time correction of the position of the welding tool, so that high accuracy and stability can be maintained during the welding process.

[0072] That is, the above-mentioned technical scheme provided by the embodiment of the present invention comprehensively considers various factors, and solves the special situation of cover welding, in which the laser stripe features are not obvious and there are difficulties in recognition, and the existing weld tracking methods face limited technical difficulties. It realizes higher-precision cover tracking during the welding process.

[0073] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0075] According to an embodiment of the present invention, a control device for a cap welding device for implementing the control method of the cap welding device is also provided. Figure 8 is a schematic diagram of a control device of a cover welding device according to an embodiment of the present invention, Figure 8 As shown, the device comprises: a first processing module 1001, a second processing module 1003 and a third processing module 1005. The control device of the cover welding device is described below.

[0076] The first processing module 1001 is used to control the midpoint of the light beam emitted by the laser transmitter to move to the center of the weld for welding, and to turn on the auxiliary light source when welding to the cover surface, wherein the cover surface is the surface of the weld during the welding process.

[0077] The second processing module 1003 is used to control the laser module to collect the surface image of the cover surface, and perform cover surface segmentation on the surface image to obtain a cover surface weld bead image, wherein the cover surface weld bead is a welding path of the cover surface and there are two of them.

[0078] The third processing module 1005 is used to determine effective feature points according to the cover weld image, perform coordinate conversion on the effective feature points to obtain target coordinate points of the crawler device, and control the crawler device to move according to the target coordinate points to weld the cover.

[0079] It should be noted here that the above-mentioned first processing module 1001, second processing module 1003 and third processing module 1005 correspond to steps S502 to S506 in the above-mentioned embodiments, and the three modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0080] As can be seen from the above, in the scheme recorded in the above-mentioned embodiment of the present invention, the first processing module can be used to control the midpoint of the light beam emitted by the laser emitter to move to the center of the weld to perform welding, and when welding to the cover surface, the auxiliary light source is turned on, wherein the cover surface is the surface of the weld during the welding process; then the second processing module can be used to control the laser module to collect the surface image of the cover surface, and the surface image is segmented to obtain the cover surface weld bead image, wherein the cover surface weld bead is the welding path of the cover surface and there are two of them; finally, the third processing module can be used to determine the effective feature points according to the cover surface weld bead image, and the effective feature points are converted into coordinates to obtain the target coordinate points of the crawling device, and the crawling device is controlled to move according to the target coordinate points to weld the cover surface, thereby achieving the purpose of high-definition acquisition of complex surfaces by the laser sensor system, analyzing and processing images by using a deep learning model, and at the same time, extracting effective feature points and correcting the position of the welding tool in real time, thereby achieving the technical effect of higher precision in cover surface tracking during welding, and improving the tracking accuracy and stability.

[0081] The above-mentioned technical solution provided by the embodiment of the present invention solves the technical problem in the related art that it is difficult to extract the cover feature points for tracking due to factors such as unclear laser stripe features, the influence of surface pits or other impurities, and the change of position of the molten pool due to gravity during welding.

[0082] In an optional embodiment, the third processing module includes: a processing unit, used to determine multiple fusion points of each cover weld in the cover weld image, and fit the multiple fusion points in each cover weld to obtain a fitting curve; a determination unit, used to determine the valid points corresponding to each cover weld according to the fitting curve, and determine the valid feature points based on the valid points.

[0083] In an optional embodiment, the determination unit includes: a first processing subunit, used to delete the fusion points whose distance from the fitting curve is greater than a preset distance threshold, and determine the remaining fusion points as valid points corresponding to each cover weld; a calculation subunit, used to calculate the midpoint of the valid points when the number of valid points corresponding to each cover weld is greater than a preset number threshold, to obtain the midpoint of the weld corresponding to each cover weld, and to obtain two weld midpoints; a second processing subunit, used to connect the two weld midpoints to obtain a weld line, calculate the midpoint of the weld line, and obtain a valid feature point.

[0084] In an optional embodiment, the control device of the cover welding device also includes: a marking module, which is used to mark the cover weld area in the surface image before the surface image is cover segmented to obtain the cover weld area image; a fourth processing module, which is used to convert the cover weld area image into a mask image and perform data expansion on the mask image.

[0085] In an optional embodiment, the second processing module includes: an analysis unit, used to analyze the surface image through a deep learning model to obtain a cover weld image, wherein the deep learning model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical surface images and cover weld image labels.

[0086] In an optional embodiment, the third processing module includes: an acquisition unit for acquiring the posture transformation matrix of the laser module and the crawling device; and a calculation unit for calculating the target coordinate point corresponding to the crawling device based on the effective feature points and the posture transformation matrix.

[0087] According to another aspect of an embodiment of the present invention, a control system for a cover welding device is also provided, comprising: any one of the above-mentioned cover welding devices; a controller, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include a control method for executing any one of the above-mentioned cover welding devices.

[0088] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, any one of the control methods of the cover welding device described above is executed.

[0089] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, and when the computer instructions are executed by a processor, any one of the above control methods for a cover welding device is executed.

[0090] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned control methods for the cover welding device.

[0091] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.

[0092] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: controlling the midpoint of the light beam emitted by the laser emitter to move to the center of the weld for welding, and turning on the auxiliary light source when welding to the cover surface, wherein the cover surface is the surface of the weld during the welding process; controlling the laser module to capture the surface image of the cover surface, performing cover surface segmentation on the surface image, and obtaining a cover surface weld bead image, wherein the cover surface weld bead is the welding path of the cover surface and there are two of them; determining effective feature points according to the cover surface weld bead image, performing coordinate conversion on the effective feature points, obtaining the target coordinate points of the crawling device, and controlling the crawling device to move according to the target coordinate points to weld the cover surface.

[0093] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: determining multiple fusion points of each cover weld in the cover weld image, fitting the multiple fusion points in each cover weld to obtain a fitting curve; determining valid points corresponding to each cover weld according to the fitting curve, and determining valid feature points according to the valid points.

[0094] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: deleting fusion points whose distance from the fitting curve is greater than a preset distance threshold, and determining the remaining fusion points as valid points corresponding to each cover weld; when the number of valid points corresponding to each cover weld is greater than a preset number threshold, calculating the midpoint of the valid points to obtain the midpoint of the weld corresponding to each cover weld, and obtaining two weld midpoints; connecting the two weld midpoints to obtain a weld line, calculating the midpoint of the weld line to obtain a valid feature point.

[0095] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: marking a cover weld area in a surface image to obtain a cover weld area image; converting the cover weld area image into a mask image, and performing data expansion on the mask image.

[0096] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: analyzing the surface image through a deep learning model to obtain a cover weld image, wherein the deep learning model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical surface images and cover weld image labels.

[0097] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining the posture transformation matrix of the laser module and the crawling device; and calculating the target coordinate points corresponding to the crawling device based on the valid feature points and the posture transformation matrix.

[0098] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A cover welding device, characterized in that: At least includes a laser module for collecting a surface image of a cover surface, where the cover surface is the surface of the weld during the welding process, and the laser module includes: A laser emitter, used for emitting laser light to illuminate the surface of the cover; an auxiliary light source, having the same light projection angle as that of the laser emitter, and used to assist the laser emitter in irradiating the surface of the cover; An image collector is arranged directly above the weld and separated from the weld surface by a first predetermined distance, and is used to capture images under the illumination of the laser emitter and the auxiliary light source; The filter is arranged directly below the image collector and is separated from the image collector by a second predetermined distance, and is used to select light of a target band, wherein the target band is the same as the band of light emitted by the auxiliary light source, and the first predetermined distance is greater than the second predetermined distance.

2. The cover welding device according to claim 1, characterized in that: The cover surface welding device also includes: Welding tools, used for welding the weld seam; A crawling device, the welding tool and the laser module are installed on the crawling device, the laser module is arranged in front of the welding tool, the front is the forward direction of the crawling device, and the crawling device is used to drive the welding tool and the laser module to crawl to complete the welding of the weld.

3. A control method for a cover welding device, characterized in that: The cap welding device is the cap welding device according to any one of claims 1 to 2, and the control method comprises: Control the midpoint of the laser emitter to move the light beam to the center of the weld to perform welding, and when welding to the cover surface, turn on the auxiliary light source, wherein the cover surface is the surface of the weld during the welding process; Controlling the laser module to collect a surface image of the cover surface, and performing cover surface segmentation on the surface image to obtain a cover surface weld bead image, wherein the cover surface weld bead is a welding path of the cover surface and there are two of them; The effective feature points are determined according to the cover weld image, the effective feature points are converted into coordinates to obtain the target coordinate points of the crawling device, and the crawling device is controlled to move according to the target coordinate points to weld the cover.

4. The control method of the cover welding device according to claim 3, characterized in that: Determining effective feature points according to the cover weld image includes: Determining a plurality of fusion points of each cap weld in the cap weld image, and fitting the plurality of fusion points in each cap weld to obtain a fitting curve; The effective points corresponding to each of the cover welds are determined according to the fitting curve, and the effective feature points are determined according to the effective points.

5. The control method of the cover welding device according to claim 4, characterized in that: Determining the effective points corresponding to each of the cap welds according to the fitting curve, and determining the effective feature points according to the effective points, including: Deleting the fusion points whose distances from the fitting curve are greater than a preset distance threshold, and determining the remaining fusion points as valid points corresponding to each of the cap welds; When the number of effective points corresponding to each of the cover welds is greater than a preset number threshold, the midpoints of the effective points are calculated to obtain the midpoint of each of the cover welds, thereby obtaining two midpoints of the welds; The midpoints of the two welds are connected to obtain a weld line, and the midpoint of the weld line is calculated to obtain the effective feature point.

6. The control method of the cover welding device according to claim 3, characterized in that: Before performing cover segmentation on the surface image, the method further comprises: Marking a cap weld bead area in the surface image to obtain a cap weld bead area image; The cover weld bead area image is converted into a mask image, and data expansion is performed on the mask image.

7. The control method of the cover welding device according to claim 3, characterized in that: Performing cover surface segmentation on the surface image to obtain a cover surface weld bead image, including: The surface image is analyzed by a deep learning model to obtain the cover weld image, wherein the deep learning model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical surface images and cover weld image labels.

8. The control method of the cover welding device according to claim 3, characterized in that: The effective feature points are converted into coordinates to obtain the target coordinate points of the crawling device, including: Obtaining the position transformation matrix of the laser module and the crawling device; The target coordinate point corresponding to the crawling device is calculated based on the effective feature points and the posture transformation matrix.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the control method of the cover welding device according to any one of claims 3 to 8.

10. A control system for a cap welding device, characterized in that: include: The cover welding device according to any one of claims 1 to 2; A controller, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a control method for executing the cover welding device described in any one of claims 3 to 8.

Images