Method and system for controlling automatic welding equipment for metal products

Abstract

The invention relates to a metal product automatic welding equipment control method and system, and the method comprises the steps: collecting a target image for a to-be-welded metal product, carrying out the defect detection of the to-be-welded metal product through a defect detection model according to the target image, and carrying out the welding control of the automatic welding equipment according to the defect information. Due to the fact that the defect problem of the metal product is considered in control of the automatic welding equipment, the problem that the welding effect of the metal product with the defect is poor can be solved; moreover, the pre-trained defect detection model is trained according to the real image and the simulation image, so that the generalization ability of the model is relatively high, the method can be applied to defect detection of various scenes, and the accuracy of defect detection is improved. Therefore, according to the technical scheme, the automatic welding equipment is controlled to weld the metal product through the defect information with high accuracy, and the welding effect of the metal product is improved.

Description

Technical Field

[0001] The present application relates to the field of automation technology, and in particular to a control method and system for automatic welding equipment for metal products. Background Art

[0002] Automatic welding equipment is a type of equipment that uses automation to perform welding processes. Widely used in industrial production, it can improve welding efficiency and quality while reducing manual intervention. Automatic welding equipment is widely used in industries such as automotive manufacturing, machining, petrochemicals, aerospace, and electronics, enabling the welding of metal products involved in these industries.

[0003] Currently, automatic welding equipment for metal products usually adopts a fixed control strategy, for example: the welding points of the automatic welding equipment are planned in advance, and the automatic welding equipment is controlled to weld according to the welding points.

[0004] This control method can be applied to scenarios where the metal product has no defects, but cannot be applied to scenarios where the metal product has defects. Therefore, for metal products with defects, there is a problem of poor welding effect. Summary of the Invention

[0005] The purpose of this application is to provide a control method and system for automatic welding equipment for metal products. The control method and system for automatic welding equipment for metal products control the automatic welding equipment to weld metal products through defect information with high accuracy, thereby improving the welding effect of metal products.

[0006] In a first aspect, the present application provides a method for controlling automatic welding equipment for metal products, comprising: acquiring a target image captured of a metal product to be welded; performing defect detection on the metal product to be welded based on the target image using a pre-trained defect detection model to obtain defect information of the metal product to be welded, wherein the training data of the pre-trained defect detection model includes real images and simulated images, the real images being images of real metal products with real defects, and the simulated images being images of real metal products with simulated defects; and controlling the automatic welding equipment to weld the metal product to be welded based at least on the defect information.

[0007] Optionally, the control method of the automatic welding equipment for metal products also includes: acquiring a first real image obtained by capturing an image of a real metal product with real defects; acquiring a second real image obtained by capturing an image of a real metal product without real defects; determining simulated defect information for characterizing the generation requirements of simulated defects; generating a simulated image of a real metal product with simulated defects based on the first real image, the second real image and the simulated defect information through a pre-trained image generation model; and training a defect detection model to be trained based on the first real image and the simulated image to obtain a pre-trained defect detection model.

[0008] Optionally, the determination of the simulated defect information for characterizing the generation requirements of the simulated defect includes: obtaining a description text corresponding to the real defect; obtaining a preset description text corresponding to the simulated defect; generating a description text corresponding to the simulated defect based on the description text corresponding to the real defect and the preset description text; obtaining a shape mask corresponding to the simulated defect; obtaining a labeling box mask corresponding to the simulated defect; and determining the description text corresponding to the simulated defect, the shape mask and the labeling box mask as the simulated defect information.

[0009] Optionally, the simulated defect information includes: a labeling box mask corresponding to the simulated defect, and generating a simulated image of a real metal product with simulated defects based on the first real image, the second real image and the simulated defect information through a pre-trained image generation model, including: masking the real defects in the first real image according to the labeling box mask to obtain a first masked image; masking the area to be generated with simulated defects in the second real image according to the labeling box mask to obtain a second masked image; determining a target masked image based on the first masked image and the second masked image; and generating a simulated image of a real metal product with simulated defects based on the target masked image and the simulated defect information through a pre-trained image generation model.

[0010] Optionally, the simulated defect information also includes: a description text and a shape mask corresponding to the simulated defect, and the pre-trained image generation model includes: an image encoder, a shape encoder, a text encoder, an autoregressive model and an image decoder. The pre-trained image generation model generates a simulated image of a real metal product with simulated defects according to the target cover image and the simulated defect information, including: generating an initial image vector through the image encoder; encoding the target cover image through the image encoder to obtain a cover image condition vector; encoding the description text through the text encoder to obtain a description text condition vector; encoding the shape mask through the shape encoder to obtain a shape condition vector; generating a target image vector through the autoregressive model according to the cover image condition vector, the description text condition vector, the shape condition vector and the initial image vector; and decoding the target image vector through the image decoder to obtain a simulated image of a real metal product with simulated defects.

[0011] Optionally, the control method of the automatic welding equipment for metal products also includes: obtaining an image sample, wherein the image sample is an image of a real metal product sample with a real defect sample; determining real defect sample information based on the image sample, wherein the real defect sample information includes a marked box mask sample corresponding to the real defect sample; according to the marked box mask sample, covering the real defect sample in the image sample to obtain a covered image sample; and training the image generation model to be trained based on the real defect sample information, the covered image sample and the image sample to obtain a pre-trained image generation model.

[0012] Optionally, the defect information includes: defect location, defect size and defect type, and controlling the automatic welding equipment to weld the metal product to be welded at least based on the defect information includes: obtaining preset control parameters for controlling the automatic welding equipment to weld the metal to be welded; determining whether the metal product to be welded is a qualified metal product based on the defect size and the defect type; if the metal product to be welded is a qualified metal product, determining target control parameters based on the defect location, the defect size, the defect type and the preset control parameters; if the metal product to be welded is an unqualified metal product, determining target control parameters based on the defect location, the defect size and the defect type; and controlling the automatic welding equipment to weld the metal product to be welded based on the target control parameters.

[0013] Optionally, the automatic welding equipment is a laser welding machine, and the preset control parameters include the multiple laser welding points arranged in time sequence, the dwell times corresponding to the multiple laser welding points, and the laser parameters corresponding to the multiple laser welding points. The target control parameters are determined according to the defect position, the defect size, the defect type, and the preset control parameters, including: determining whether there is a target laser welding point matching the defect position among the multiple laser welding points; if there is a target laser welding point matching the defect position among the multiple laser welding points, adjusting the dwell time and laser parameters corresponding to the target laser welding point according to the defect size and the defect type to obtain the target control parameters; if there is no target laser welding point matching the defect position among the multiple laser welding points, generating a new laser welding point according to the defect position, and determining the dwell time and laser parameters corresponding to the new laser welding point according to the defect size and the defect type; adjusting the preset control parameters according to the new laser welding point and the dwell time and laser parameters corresponding to the new laser welding point to obtain the target control parameters.

[0014] Optionally, determining the target control parameters based on the defect position, the defect size and the defect type includes: generating a target laser welding point based on the defect position; determining the dwell time and laser parameters corresponding to the target laser welding point based on the defect size and the defect type; and determining the target control parameters based on the target laser welding point and the dwell time and laser parameters corresponding to the target laser welding point.

[0015] In a second aspect, the present application provides a control system for automatic welding equipment for metal products, comprising: an image acquisition device for acquiring a target image of the metal product to be welded; an automatic welding device for welding the metal product to be welded; and a control device for executing the control method for automatic welding equipment for metal products as described in the first aspect of the present application.

[0016] Through the above technical solution, a target image is captured of the metal product to be welded. A defect detection model is then used to detect defects in the metal product based on the target image. The automatic welding equipment is then controlled based on this defect information. Because the control of the automatic welding equipment takes into account the defects of the metal product, the problem of poor welding results for defective metal products can be resolved. Furthermore, because the pre-trained defect detection model is trained using both real and simulated images, it has strong generalization capabilities and can be applied to defect detection in various scenarios, improving defect detection accuracy. Consequently, this technical solution uses highly accurate defect information to control the automatic welding equipment to weld metal products, improving the welding results of the metal products.

[0017] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the present application but do not constitute a limitation of the present application. In the accompanying drawings:

[0019] Figure 1 The figure is a block diagram of a control system for automatic metal welding equipment according to an exemplary embodiment.

[0020] Figure 2 The figure is a flow chart showing a method for controlling automatic welding equipment for metal products according to an exemplary embodiment.

[0021] Figure 3A The figure is an example diagram showing a labeling box mask according to an exemplary embodiment.

[0022] Figure 3B FIG. 4 is a diagram showing an example of a shape mask according to an exemplary embodiment.

[0023] Figure 3C FIG. 4 is an example diagram showing a cover image according to an exemplary embodiment.

[0024] Figure 4 It is a block diagram of an image generation model according to an exemplary embodiment.

[0025] Figure 5 The figure is a block diagram of a control device for automatic welding equipment for metal products according to an exemplary embodiment.

[0026] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] The following describes the specific embodiments of the present application in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present application and are not intended to limit the present application.

[0028] Automatic welding equipment is a type of equipment that uses automation to perform welding processes. Widely used in industrial production, it can improve welding efficiency and quality while reducing manual intervention. Automatic welding equipment is widely used in industries such as automotive manufacturing, machining, petrochemicals, aerospace, and electronics, enabling the welding of metal products involved in these industries.

[0029] Currently, automatic welding equipment for metal products usually adopts a fixed control strategy, for example: the welding points of the automatic welding equipment are planned in advance, and the automatic welding equipment is controlled to weld according to the welding points.

[0030] This control method can be applied to scenarios where the metal product has no defects, but cannot be applied to scenarios where the metal product has defects. Therefore, for metal products with defects, there is a problem of poor welding effect.

[0031] Based on this, an embodiment of the present application provides a technical solution for collecting target images of metal products to be welded, using a defect detection model to perform defect detection on the metal products to be welded based on the target images, and then controlling the welding of automatic welding equipment based on the defect information.

[0032] Since the control of automatic welding equipment takes into account the defects of metal products, it can solve the problem of poor welding effect on defective metal products. In addition, since the pre-trained defect detection model is trained based on real images and simulated images, the model has strong generalization ability and can be applied to defect detection in various scenarios, thereby improving the accuracy of defect detection.

[0033] Therefore, this technical solution controls the automatic welding equipment to weld metal products through defect information with high accuracy, thereby improving the welding effect of the metal products.

[0034] In the embodiment of the present application, the automatic welding equipment may be a laser welding equipment, or a welding equipment of the same type as the laser welding equipment.

[0035] Figure 1 FIG. 1 is a block diagram of a control system for an automatic metal welding device according to an exemplary embodiment. Figure 1 As shown, the system includes: image acquisition equipment, automatic welding equipment and control equipment.

[0036] The image acquisition device can be set at a corresponding position of the welding process table to capture images of the metal products to be welded on the welding process table, thereby obtaining the target image. The image acquisition device can be an industrial camera, an infrared sensor camera, etc.

[0037] Automatic welding equipment can be set at the corresponding position of the welding process table to weld the metal products to be welded on the welding process table.

[0038] Laser welding equipment, for example, utilizes a high-energy-density laser beam to weld metal materials, offering advantages such as high welding speed, high weld quality, a minimal heat-affected zone, and non-contact processing. Its operating principle is as follows: Laser welding equipment generates a high-energy-density laser beam through a laser and focuses it onto the metal surface. This laser energy absorbs the metal and rapidly heats and melts it, forming a molten metal pool. This pool then rapidly cools and solidifies, forming the weld. Depending on the laser power density, laser welding can be categorized as heat conduction welding or laser deep penetration welding. Heat conduction welding is suitable for thin plate welding, while laser deep penetration welding is suitable for thicker plates.

[0039] The control device is connected to the image acquisition device and the automatic welding device respectively, for example, through the Internet of Things communication connection, and the control device can be a host computer.

[0040] The control device can obtain images from the image acquisition device and optimize the control of the automatic welding equipment according to the images acquired by the image acquisition device.

[0041] Figure 2 The flowchart of a method for controlling automatic welding equipment for metal products according to an exemplary embodiment is shown. The method includes the following steps:

[0042] Step S21: acquiring a target image of the metal product to be welded.

[0043] In step S22, defect detection is performed on the metal product to be welded based on the target image using a pre-trained defect detection model to obtain defect information of the metal product to be welded. The training data of the pre-trained defect detection model includes real images and simulated images. The real images are images of real metal products with real defects, and the simulated images are images of real metal products with simulated defects.

[0044] Step S23: Controlling the automatic welding equipment to weld the metal product at least according to the defect information.

[0045] The metal product to be welded may be a metal product that currently needs to be welded. In an automated industrial process, when the metal product reaches a designated welding position, the image acquisition device may capture an image of the metal product to obtain a target image.

[0046] The metal product to be welded may or may not have defects. If defects exist, they will affect the welding strategy of the automatic welding equipment. If defects do not exist, they will not affect the welding strategy of the automatic welding equipment. Therefore, defect detection can be performed based on the target image.

[0047] In step S22, the target image is input into the pre-trained defect detection model, and defect information output by the defect detection model can be obtained.

[0048] In some embodiments, the pre-trained defect detection model may be a deep convolutional network model, which can obtain corresponding defect information, such as defect type, defect size, and defect shape, by performing defect recognition on an image.

[0049] In some embodiments, the training data of the pre-trained defect detection model may include: real images and simulated images, where the real images are images of real metal products with real defects, and the simulated images are images of real metal products with simulated defects.

[0050] Understandably, capturing some real defects is more difficult. For example, defects located in unusual locations on metal products, defects with unusual shapes, and smaller defects are less common, making it more difficult to capture real images. However, to improve the detection accuracy of the model, images corresponding to these defects are also required. Therefore, using simulated images of simulated defects can compensate for the difficulty in capturing images of these special defects.

[0051] Furthermore, combining real images and simulated images to train the defect detection model can improve the robustness of the defect detection model and improve the detection accuracy.

[0052] As an optional implementation, the training process of the pre-trained defect model includes: obtaining a first real image obtained by capturing an image of a real metal product with real defects; obtaining a second real image obtained by capturing an image of a real metal product without real defects; determining simulated defect information for characterizing generation requirements for simulated defects; generating a simulated image of a real metal product with simulated defects based on the first real image, the second real image and the simulated defect information through a pre-trained image generation model; and training a defect detection model to be trained based on the first real image and the simulated image to obtain a pre-trained defect detection model.

[0053] In this embodiment, defect information labels can be set for real defects in the first real image, and defect information labels can be set for simulated defects in the simulated image based on the simulated defect information. Then, based on the first real image and the simulated image with corresponding labels set, the defect detection model to be trained is trained to obtain a pre-trained defect detection model.

[0054] In some embodiments, the simulated image can be generated by a pre-trained image generation model based on the first real image, the second real image, and simulated defect information for characterizing generation requirements of the simulated defect.

[0055] As an optional implementation, determining the simulated defect information for characterizing the generation requirements of the simulated defect includes: obtaining a description text corresponding to the real defect; obtaining a preset description text corresponding to the simulated defect; generating a description text corresponding to the simulated defect based on the description text corresponding to the real defect and the preset description text; obtaining a shape mask corresponding to the simulated defect; obtaining a labeling box mask corresponding to the simulated defect; and determining the description text, shape mask and labeling box mask corresponding to the simulated defect as the simulated defect information.

[0056] In some embodiments, the description text corresponding to the real defect may be used to describe the type of the real defect, such as cracks, pores, or unfused metal materials.

[0057] In some embodiments, the description text corresponding to the real defect can be determined by the user, or can be determined by detecting the real image through a large language model; or can be determined by a multimodal model, etc., which is not limited here.

[0058] In some embodiments, the preset description text corresponding to the simulated defect may be a description text uploaded by a user and related to the required simulated defect, which may also be used to describe the type of simulated defect to generate requirements.

[0059] In some embodiments, a description text corresponding to a simulated defect may be generated based on a description text corresponding to a real defect and a preset description text.

[0060] For example, in the preset description text, the text that exists in the description text corresponding to the real defect can be deleted, expanded, adjusted, etc. to obtain the description text corresponding to the simulated defect. The description text corresponding to the simulated defect is mainly based on the preset description text, but the duplicate description text of the description text corresponding to the real defect needs to be removed so that the description text corresponding to the simulated defect can compensate for the shortcomings of the real defect.

[0061] In some embodiments, the shape mask corresponding to the simulated defect may be a pre-configured shape mask or a real shape mask extracted based on other images.

[0062] In some embodiments, the annotation box mask corresponding to the simulated defect may be generated based on the shape mask corresponding to the simulated defect, for example, the bounding box mask of the shape mask is used as the annotation box mask.

[0063] Furthermore, the description text, shape mask, and annotation box mask corresponding to the simulated defect may be determined as the simulated defect information.

[0064] In some embodiments, a pre-trained image generation model is used to generate a simulated image of a real metal product with simulated defects based on a first real image, a second real image, and simulated defect information. The method may include: masking the real defects in the first real image based on a marked frame mask to obtain a first masked image; masking the area in the second real image where simulated defects are to be generated based on the marked frame mask to obtain a second masked image; determining a target masked image based on the first masked image and the second masked image; and generating a simulated image of a real metal product with simulated defects based on the target masked image and simulated defect information through a pre-trained image generation model.

[0065] Figure 3A is an example diagram showing a labeling box mask according to an exemplary embodiment. Figure 3A As shown, the annotation box mask can be used to mark the defect area in the image.

[0066] Figure 3B is an example diagram of a shape mask according to an exemplary embodiment. Figure 3B As shown, a shape mask can be used to indicate the shape of defects in an image.

[0067] Figure 3C is an example diagram of a cover image according to an exemplary embodiment. Figure 3C As shown, in the mask image, the area where the simulated defect needs to be generated is masked, while the other areas are retained.

[0068] Then, for the first real image, the area where the real defect is located can be masked to obtain a first masked image. And for the second real image, the area where the simulated defect is to be generated can be masked to obtain a second masked image.

[0069] Furthermore, the image similarity between the first cover image and the second cover image can be determined, and the cover image with an image similarity lower than a preset similarity is determined as the target cover image. Therefore, the number of target cover images is greater than or equal to one.

[0070] Furthermore, the simulated defect information and the target mask image are input into a pre-trained image generation model to obtain a simulated image output by the pre-trained image generation model, which includes the simulated defect.

[0071] Figure 4 is a block diagram of an image generation model according to an exemplary embodiment. Figure 4 As shown, the pre-trained image generation model includes: image encoder, shape encoder, text encoder, autoregressive model and image decoder.

[0072] The image encoder and image decoder can also be implemented through an integrated network structure, such as an adversarial network model. The image encoder can be used to encode the image, and the image decoder can be used to decode the image.

[0073] In addition, the shape encoder can be used to encode shape masks; the text encoder can be used to encode text; and the autoregressive model can generate images through autoregressive prediction.

[0074] Therefore, as an optional implementation, a pre-trained image generation model is used to generate a simulated image of a real metal product with simulated defects based on the target cover image and the simulated defect information, including: generating an initial image vector through an image encoder; encoding the target cover image through the image encoder to obtain a cover image conditional vector; encoding the description text through a text encoder to obtain a description text conditional vector; encoding the shape mask through a shape encoder to obtain a shape conditional vector; generating a target image vector through an autoregressive model based on the cover image conditional vector, the description text conditional vector, the shape conditional vector and the initial image vector; and decoding the target image vector through an image decoder to obtain a simulated image of a real metal product with simulated defects.

[0075] In some embodiments, the image encoder may generate an initial image vector by random initialization using a normal distribution.

[0076] In some embodiments, the image encoder may encode the target cover image into an image vector of the same size as the initial image vector to obtain a cover image condition vector, which may be used as a control condition for subsequent image generation.

[0077] In some embodiments, the text encoder may encode the description text into a vector of the same type as the initial image vector to obtain a description text condition vector, which may be used as a control condition for subsequent image generation.

[0078] In some embodiments, the shape encoder may encode the shape mask into an image vector of the same size as the original image vector, resulting in a shape condition vector.

[0079] In some embodiments, the autoregressive model performs autoregressive prediction based on the mask image condition vector, the description text condition vector, the shape condition vector and the initial image vector to generate a target image vector.

[0080] In some embodiments, the autoregressive model may also be replaced by other models, such as a variational autoencoder, an adversarial network, etc.

[0081] In some embodiments, the image decoder decodes the target image vector to obtain a simulated image of a real metal product with simulated defects.

[0082] In some embodiments, the training process of the pre-trained image generation model may include: obtaining an image sample, which is an image of a real metal product sample with a real defect sample; determining the real defect sample information based on the image sample, the real defect sample information includes the labeled box mask sample corresponding to the real defect sample; according to the labeled box mask sample, the real defect sample in the image sample is covered to obtain a covered image sample; according to the real defect sample information, the covered image sample and the image sample, the image generation model to be trained is trained to obtain a pre-trained image generation model.

[0083] In some embodiments, image samples can be obtained in a variety of ways, such as user uploading, big data acquisition, etc.

[0084] In some embodiments, shape mask samples, annotation box mask samples, and text description samples of real defect samples can be extracted based on image samples to obtain real defect sample information. The extraction methods for these samples can refer to the methods for determining various information in the aforementioned embodiments and will not be described in detail here.

[0085] In some embodiments, the image encoder, shape encoder, text encoder, and image decoder in the image generation model to be trained may be pre-trained encoders or decoders.

[0086] For example, for an image encoder, its image encoding capability can be pre-trained; for a shape encoder, its shape mask encoding capability can be pre-trained; for a text encoder, its text encoding capability can be pre-trained; and for an image decoder, its decoding capability can be pre-trained. For specific training methods for encoders and decoders, reference can be made to mature technologies in the field.

[0087] In some embodiments, an image encoder, a shape encoder, or a text encoder may be used to encode the corresponding information. The encoded information and masked image samples are then used as training samples, and the image samples are used as labels to train the autoregressive model, thereby enabling the autoregressive model to generate images. For example, the autoregressive model may be used to make predictions based on the training samples to obtain prediction results, determine the loss between the prediction results and the image sample labels, and train the autoregressive model based on the loss.

[0088] Furthermore, the pre-trained image generation model can generate simulated images of real metal products with simulated defects based on the input mask image and simulated defect information.

[0089] Furthermore, in combination with the introduction to the training of the defect detection model in the aforementioned embodiment, the pre-trained defect detection model can output defect information based on the input target image.

[0090] As an optional implementation manner, the defect information includes: defect location, defect size and defect type.

[0091] The defect position may be the position of the defect relative to the metal product, the defect size may be represented by a size parameter, and the defect type may be represented by a defect type identifier.

[0092] In step S23, based on at least the defect information, the automatic welding equipment is controlled to weld the metal product.

[0093] In some embodiments, in addition to defect information, welding control can also be performed in conjunction with other information that may affect welding. For example, positioning information can be combined to determine whether there is positioning deviation and, if so, to adjust the control scheme. The embodiments of this application primarily describe how to perform welding control based on defect information, and are not limited to performing welding control based solely on defect information.

[0094] As an optional embodiment, step S23 includes: obtaining preset control parameters for controlling the automatic welding equipment to weld the metal to be welded; determining whether the metal product to be welded is a qualified metal product based on the defect size and defect type; if the metal product to be welded is a qualified metal product, determining the target control parameters based on the defect position, defect size, defect type and preset control parameters; if the metal product to be welded is an unqualified metal product, determining the target control parameters based on the defect position, defect size and defect type; and controlling the automatic welding equipment to weld the metal product to be welded based on the target control parameters.

[0095] In this embodiment, the automatic welding equipment is configured with preset control parameters. The defect size and defect type can be used to first determine whether the metal product to be welded is qualified. Different control parameter updating methods are used according to the judgment results.

[0096] In some embodiments, when the defect size is not within a preset defect size range and / or the defect type does not belong to a preset defect type, the metal product to be welded is determined to be an unqualified metal product; otherwise, the metal product to be welded is determined to be a qualified metal product.

[0097] The preset defect size range and the preset defect type can be configured according to the defects that can be tolerated for metal products in different scenarios.

[0098] In some embodiments, when the metal product to be welded is qualified, it indicates that welding can continue based on preset control parameters, but the welding of defective parts needs to be considered.

[0099] In the case that the metal product to be welded is unqualified, it means that welding cannot be continued based on the preset control parameters, and the defects can be processed first.

[0100] In some embodiments, the automatic welding equipment is a laser welding machine, and the preset control parameters include multiple laser welding points arranged in a time sequence, the dwell times corresponding to the multiple laser welding points, and the laser parameters corresponding to the multiple laser welding points.

[0101] In some embodiments, the laser welding points may be represented by welding point coordinates, which may be three-dimensional coordinates.

[0102] In some embodiments, the dwell time and laser parameters may correspond to different welding effects, wherein the laser parameters may include: the emission power of the laser beam, the irradiation speed of the laser beam, etc.

[0103] Furthermore, determining the target control parameters based on the defect position, defect size, defect type and preset control parameters may include: determining whether there is a target laser welding point that matches the defect position among multiple laser welding points; in the case that there is a target laser welding point that matches the defect position among multiple laser welding points, adjusting the dwell time and laser parameters corresponding to the target laser welding point according to the defect size and defect type to obtain the target control parameters; in the case that there is no target laser welding point that matches the defect position among multiple laser welding points, generating a new laser welding point according to the defect position, and determining the dwell time and laser parameters corresponding to the new laser welding point according to the defect size and defect type, and adjusting the preset control parameters according to the new laser welding point and the dwell time and laser parameters corresponding to the new laser welding point to obtain the target control parameters.

[0104] In some embodiments, the defect position detected by the model can be compared to the position in the image, and by converting the camera coordinate system and the world coordinate system, the defect position in the same coordinate system as the laser welding point can be obtained.

[0105] In some embodiments, the target laser welding point that matches the defect position may be a laser welding point with the same coordinates as or close to (eg, very close to) the defect position.

[0106] In some embodiments, the dwell time and laser parameters corresponding to the target laser welding point are adjusted according to the defect size and defect type, which may include: increasing the dwell time according to the defect size; and adjusting the emission power and speed of the laser beam according to the defect type.

[0107] In some embodiments, laser parameters and dwell times tailored to metal products with varying defect sizes and types can be pre-tested, including offline or simulation testing. This testing generates a priori data, which can then be used to determine appropriate adjustments. For example, if the a priori data indicates that the current transmit power is lower than the required transmit power, the transmit power can be increased.

[0108] In some embodiments, the dwell time and laser parameters corresponding to the newly added laser welding points may also be determined in combination with prior data.

[0109] In some embodiments, target control parameters are determined based on the defect location, defect size, and defect type, including: generating a target laser welding point based on the defect location; determining the dwell time and laser parameters corresponding to the target laser welding point based on the defect size and defect type; and determining the target control parameters based on the target laser welding point and the dwell time and laser parameters corresponding to the target laser welding point.

[0110] In this embodiment, the defect position needs to be welded, so a target laser welding point can be generated based on the defect position, and the dwell time and laser parameters corresponding to the target laser welding point can be further determined to obtain target control parameters.

[0111] Among them, the dwell time and laser parameters corresponding to the target laser welding point are determined based on the defect size and defect type. They can also be determined based on prior data, which will not be repeated here.

[0112] Furthermore, no matter what kind of defects exist in the metal product, the corresponding control of the automatic welding equipment can be achieved to improve the welding effect of the metal product.

[0113] Figure 5 is a block diagram of a control device 500 for automatic welding equipment for metal products according to an exemplary embodiment. Figure 5 As shown, the device includes:

[0114] The acquisition module 501 is used to acquire a target image of the metal product to be welded.

[0115] The detection module 502 is configured to perform defect detection on the metal product to be welded based on the target image using a pre-trained defect detection model to obtain defect information of the metal product to be welded, wherein the training data of the pre-trained defect detection model includes real images and simulated images, wherein the real images are images of real metal products with real defects, and the simulated images are images of real metal products with simulated defects.

[0116] The control module 503 is configured to control the automatic welding equipment to weld the metal product to be welded based at least on the defect information.

[0117] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0118] Figure 6 FIG. 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. Figure 6 As shown, the electronic device 600 may include: a processor 601 , a memory 602 , and may further include one or more of a multimedia component 603 , an input / output (I / O) interface 604 , and a communication component 605 .

[0119] The processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned method for controlling automatic welding equipment for metal products. The memory 602 is used to store various types of data to support the operation of the electronic device 600. Such data may include, for example, instructions for any application or method operating on the electronic device 600, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 602 or sent through the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the processor 601 and other interface modules. The above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 605 may include: a Wi-Fi module, a Bluetooth module, an NFC module.

[0120] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-mentioned method for controlling the automatic welding equipment for metal products.

[0121] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned method for controlling automatic metal welding equipment. For example, the computer-readable storage medium may be the aforementioned memory 602 including the program instructions. The program instructions may be executed by the processor 601 of the electronic device 600 to implement the aforementioned method for controlling automatic metal welding equipment.

[0122] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a processor. When the computer program is executed by the processor, the steps of the above-mentioned method for controlling automatic welding equipment for metal products are implemented.

[0123] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, various simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the scope of protection of the present application.

[0124] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner unless there is any contradiction. In order to avoid unnecessary repetition, this application will not further describe various possible combinations.

[0125] In addition, the various implementation methods of the present application may be arbitrarily combined, and as long as they do not violate the concept of the present application, they should also be regarded as the contents disclosed in the present application.

Claims

1. A method for controlling automatic welding equipment for metal products, characterized in that: include: Acquire a target image of the metal product to be welded; performing defect detection on the metal product to be welded based on the target image using a pre-trained defect detection model to obtain defect information of the metal product to be welded, wherein training data for the pre-trained defect detection model includes real images and simulated images, wherein the real images are images of real metal products with real defects, and the simulated images are images of real metal products with simulated defects; Automatic welding equipment is controlled to weld the metal product to be welded based at least on the defect information.

2. The method for controlling metal automatic welding equipment according to claim 1, characterized in that: The method for controlling the metal product automatic welding equipment further includes: Acquiring a first real image obtained by capturing an image of a real metal product having real defects; acquiring a second real image obtained by capturing an image of a real metal product without any real defects; determining simulation defect information for characterizing generation requirements of the simulation defect; generating, using a pre-trained image generation model, a simulated image of a real metal product having a simulated defect based on the first real image, the second real image, and the simulated defect information; The defect detection model to be trained is trained according to the first real image and the simulated image to obtain a pre-trained defect detection model.

3. The control method of metal automatic welding equipment according to claim 2, characterized in that: The determining of the simulated defect information used to characterize the generation requirement of the simulated defect includes: Obtaining a description text corresponding to the real defect; Obtaining a preset description text corresponding to the simulated defect; Generate a description text corresponding to the simulated defect according to the description text corresponding to the real defect and the preset description text; Obtaining a shape mask corresponding to the simulated defect; Obtaining a labeling box mask corresponding to the simulated defect; The description text, the shape mask, and the annotation box mask corresponding to the simulated defect are determined as the simulated defect information.

4. The control method of metal automatic welding equipment according to claim 2, characterized in that: The simulated defect information includes: a labeling box mask corresponding to the simulated defect; and the pre-trained image generation model generates a simulated image of a real metal product having the simulated defect based on the first real image, the second real image, and the simulated defect information, including: masking the real defects in the first real image according to the annotation box mask to obtain a first masked image; According to the annotation frame mask, the area where the simulated defect is to be generated in the second real image is masked to obtain a second masked image; determining a target mask image according to the first mask image and the second mask image; A simulated image of a real metal product with simulated defects is generated based on the target mask image and the simulated defect information through a pre-trained image generation model.

5. The control method of metal automatic welding equipment according to claim 4, characterized in that: The simulated defect information further includes: description text and a shape mask corresponding to the simulated defect; the pre-trained image generation model includes: an image encoder, a shape encoder, a text encoder, an autoregressive model, and an image decoder; and the pre-trained image generation model generates a simulated image of a real metal product having a simulated defect based on the target mask image and the simulated defect information, including: Generate an initial image vector by the image encoder; Encoding the target cover image by the image encoder to obtain a cover image condition vector; Encoding the description text by the text encoder to obtain a description text condition vector; encoding the shape mask by the shape encoder to obtain a shape condition vector; Generate a target image vector according to the mask image condition vector, the description text condition vector, the shape condition vector and the initial image vector through the autoregressive model; The target image vector is decoded by the image decoder to obtain a simulated image of a real metal product with simulated defects.

6. The method for controlling metal automatic welding equipment according to claim 4, characterized in that: The method for controlling the metal product automatic welding equipment further includes: Acquire an image sample, where the image sample is an image of a real metal product sample having a real defect sample; Determining true defect sample information based on the image sample, where the true defect sample information includes a marked box mask sample corresponding to the true defect sample; According to the marked frame mask sample, a real defect sample in the image sample is covered to obtain a covered image sample; The image generation model to be trained is trained according to the real defect sample information, the masked image sample and the image sample to obtain a pre-trained image generation model.

7. The method for controlling metal product automatic welding equipment according to claim 1, characterized in that: The defect information includes: defect location, defect size, and defect type. Controlling the automatic welding equipment to weld the metal product to be welded based at least on the defect information includes: Acquiring preset control parameters for controlling the automatic welding equipment to weld the metal to be welded; Determining whether the metal product to be welded is a qualified metal product according to the defect size and the defect type; If the metal product to be welded is a qualified metal product, determining target control parameters according to the defect position, the defect size, the defect type and the preset control parameters; In a case where the metal product to be welded is an unqualified metal product, determining target control parameters according to the defect position, the defect size, and the defect type; According to the target control parameters, the automatic welding equipment is controlled to weld the metal product to be welded.

8. The method for controlling metal automatic welding equipment according to claim 7, characterized in that: The automatic welding equipment is a laser welding machine, the preset control parameters include a plurality of laser welding points arranged in a time sequence, the dwell times corresponding to the plurality of laser welding points, and the laser parameters corresponding to the plurality of laser welding points, and determining the target control parameters according to the defect location, the defect size, the defect type, and the preset control parameters includes: determining whether there is a target laser welding point matching the defect position among the plurality of laser welding points; When there is a target laser welding point matching the defect position among the multiple laser welding points, adjusting the dwell time and laser parameters corresponding to the target laser welding point according to the defect size and the defect type to obtain target control parameters; In the case that there is no target laser welding point matching the defect position among the multiple laser welding points, a new laser welding point is generated according to the defect position, and the dwell time and laser parameters corresponding to the new laser welding point are determined according to the defect size and the defect type. According to the new laser welding point and the dwell time and laser parameters corresponding to the new laser welding point, the preset control parameters are adjusted to obtain the target control parameters.

9. The method for controlling metal automatic welding equipment according to claim 7, characterized in that: The determining of target control parameters according to the defect location, the defect size, and the defect type includes: generating a target laser welding point according to the defect position; Determine the dwell time and laser parameters corresponding to the target laser welding point according to the defect size and the defect type; Target control parameters are determined according to the target laser welding point, the dwell time corresponding to the target laser welding point, and laser parameters.

10. A control system for automatic welding equipment for metal products, characterized in that: include: Image acquisition equipment, used to capture target images of metal products to be welded; Automatic welding equipment for welding metal products to be welded; A control device for executing the method for controlling automatic welding equipment for metal products as claimed in any one of claims 1 to 9.

Images