Welding defect detection method, device, storage medium and program product

Abstract

The present invention discloses a welding defect detection method, equipment, storage medium and program product, which relates to the field of welding technology, including: using a defect position detection model to detect the defect position of an original welding image, and obtaining a target welding defect image containing a defect position detection frame; cutting the target welding defect image according to the defect position detection frame, and obtaining a welding defect area image in the defect position detection frame; segmenting the welding defect area image, and obtaining each welding defect area sub-image; splicing the target welding defect image, the welding defect area image and each welding defect area sub-image, and obtaining an image after image enhancement; using a pre-trained model to detect the defect type of the image after image enhancement, and obtaining the target welding defect type. The welding defect detection method provided by the present invention improves the welding defect detection capability, avoids interference from environmental factors, saves manpower, and reduces costs.

Description

Technical Field

[0001] The present invention relates to the field of welding technology, and in particular to a welding defect detection method, equipment, computer-readable storage medium and computer program product. Background Art

[0002] In the field of welding production, defect detection is a crucial link, because even a tiny defect may cause the performance of the welded part to deteriorate or even fail. Commonly used defect detection methods in the production process of welded parts include visual inspection, laser wave detection, and the use of traditional computer vision detection models to detect defect photos.

[0003] However, each of the above welding defect detection methods has its own shortcomings. First, visual inspection is to judge the defect type of welded parts through manual observation. It is limited by the experience and subjective judgment of the inspectors and has limited ability to identify defects. Secondly, laser inspection is to scan the weld surface with a laser and use optical methods to detect surface and near-surface defects of welded parts. The equipment is expensive, the operation is complicated, the surface condition is required, surface pretreatment is required, and it is greatly affected by ambient light and reflection. Finally, the traditional computer vision detection model is used to detect defect photos. In the traditional computer vision detection model, the model's ability is limited to drawing the detection frame, and it is impossible to perform further automatic analysis of the detection results. Manual analysis is still required, which is costly.

[0004] In summary, how to effectively solve the problems of limited defect recognition ability of welding defect detection, great influence of environmental factors on defect detection results, and high labor costs are issues that technical personnel in this field urgently need to solve. Summary of the invention

[0005] An object of the present invention is to provide a welding defect detection method, which improves the welding defect detection capability, avoids interference from environmental factors, saves manpower, and reduces costs; another object of the present invention is to provide a welding defect detection device, a computer-readable storage medium, and a computer program product.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A welding defect detection method, comprising:

[0008] The defect position detection model is used to detect the defect position of the original welding image, and a target welding defect image containing a defect position detection frame is obtained;

[0009] Cropping the target welding defect image according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame;

[0010] Segmenting the welding defect area image to obtain sub-images of each welding defect area;

[0011] splicing the target welding defect image, the welding defect area image and each welding defect area sub-image to obtain an image enhanced image;

[0012] The pre-trained model is used to detect the defect types of the image after image enhancement to obtain the target welding defect types.

[0013] In a specific embodiment of the present invention, segmenting the welding defect area image includes:

[0014] Obtain a list of image resolutions and preset segmentation ratios compatible with the visual encoder in a pre-trained model; wherein the pre-trained model is a model for defect type detection;

[0015] Obtaining the image width and image height of the welding defect area image;

[0016] When it is determined that image segmentation is required according to the image resolution, the image width, the image height and the segmentation ratio list, the welding defect area image is segmented.

[0017] In a specific implementation of the present invention, when it is determined that image segmentation is required according to the image resolution, the image width, the image height and the segmentation ratio list, segmenting the welding defect area image includes:

[0018] Calculate the aspect ratio according to the image width and the image height;

[0019] Select the smaller one from the image width and the image height;

[0020] Calculate the ratio of the smaller one to the corresponding edge resolution in the picture resolution;

[0021] Searching for a target segmentation ratio from the segmentation ratio list according to the aspect ratio and the ratio value;

[0022] The welding defect area image is segmented according to the target segmentation ratio.

[0023] In a specific implementation of the present invention, searching the target segmentation ratio from the segmentation ratio list according to the aspect ratio and the ratio value includes:

[0024] Rounding up the ratio value to obtain an integer ratio value;

[0025] Searching the initial segmentation ratio with the smallest difference from the aspect ratio from the segmentation ratio list;

[0026] The target segmentation ratio is obtained by multiplying the integer ratio value by the initial segmentation ratio.

[0027] In a specific implementation of the present invention, the target welding defect image, the welding defect area image and each welding defect area sub-image are spliced, including:

[0028] Adjusting the target welding defect image to the size of the image resolution to obtain an adjusted welding defect image;

[0029] Adjusting the welding defect area image to the size of the image resolution to obtain an adjusted welding defect area image;

[0030] Adjusting each welding defect area sub-image to the size of the image resolution to obtain each adjusted welding defect area sub-image;

[0031] The adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image are spliced.

[0032] In a specific implementation of the present invention, the adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image are spliced ​​to obtain the image enhanced image, including:

[0033] splicing the pixel values ​​of the adjusted welding defect image, the pixel values ​​of the adjusted welding defect region image, and the pixel values ​​of each adjusted welding defect region sub-image to obtain a target tensor;

[0034] Accordingly, the pre-trained model is used to detect defect types on the image after image enhancement, including:

[0035] The pre-trained model is used to detect defect types on the target tensor.

[0036] In a specific embodiment of the present invention, using the pre-trained model to detect defect types on the target tensor includes:

[0037] Using the visual encoder to perform a convolution operation on the target tensor to obtain a visual feature;

[0038] Projecting the visual features using a fully connected layer in the pre-trained model to obtain an image token;

[0039] Get the preset prompt word token;

[0040] Concatenate the image token and the prompt word token to obtain an input token;

[0041] A pre-trained language model in the pre-trained model is used to perform defect type detection according to the input token.

[0042] In a specific implementation of the present invention, obtaining a preset prompt word token includes:

[0043] The preset natural language prompt word is encoded using the marker in the pre-trained model to obtain the prompt word token.

[0044] In a specific embodiment of the present invention, the pre-trained language model in the pre-trained model is used to perform defect type detection according to the input token to obtain the target welding defect type, including:

[0045] Inputting the input token into the pre-trained language model to obtain an output token of the pre-trained language model;

[0046] The output token is decoded using the marker in the pre-trained model to obtain the target welding defect type in natural language.

[0047] In a specific implementation of the present invention, obtaining a preset prompt word token includes:

[0048] Get a hint token containing the defect type, cause, and improvement measures.

[0049] Accordingly, the pre-trained language model in the pre-trained model is used to detect the defect type according to the input token to obtain the target welding defect type, including:

[0050] The pre-trained language model is used to determine the target welding defect type, defect cause and defect improvement measures according to the input token.

[0051] In a specific embodiment of the present invention, obtaining a prompt word token including defect type, cause and improvement measure suggestion includes:

[0052] Obtain a prompt word token including the defect type, the cause, the improvement measure suggestion and the image path;

[0053] Accordingly, the pre-trained language model is used to determine the target welding defect type, defect cause and defect improvement measures according to the input token, including:

[0054] The pre-trained language model is used to determine the target welding defect type, the defect cause, the defect improvement measures and image path information according to the input token.

[0055] In a specific implementation of the present invention, a training process of the defect position detection model is also included, and the training process of the defect position detection model includes:

[0056] Obtaining samples of each welding defect image including a welding defect detection frame;

[0057] The initial detection model is trained using each welding defect image sample to obtain the defect position detection model.

[0058] A welding defect detection device, comprising:

[0059] Memory for storing computer programs;

[0060] A processor is used to implement the steps of the welding defect detection method as described above when executing the computer program.

[0061] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the welding defect detection method described above are implemented.

[0062] A computer program product comprises a computer program, which implements the steps of the welding defect detection method as described above when executed by a processor.

[0063] The welding defect detection method provided by the present invention uses a defect position detection model to detect the defect position of an original welding image to obtain a target welding defect image including a defect position detection frame; crops the target welding defect image according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame; segments the welding defect area image to obtain each welding defect area sub-image; splices the target welding defect image, the welding defect area image and each welding defect area sub-image to obtain an image after image enhancement; and uses a pre-trained model to detect the defect type of the image after image enhancement to obtain a target welding defect type.

[0064] It can be seen from the above technical scheme that the defect position detection model is obtained by pre-training, and the defect position detection model is used to detect the defect position of the original welding picture of the welded part, so as to obtain the target welding defect picture containing the defect position detection frame. The image cropping image enhancement method is introduced. By cropping the target welding defect picture, the welding defect area picture of the key area in the defect position detection frame is obtained, and the welding defect area picture is divided into multiple welding defect area sub-pictures, and the target welding defect picture, the welding defect area picture and each welding defect area sub-picture are spliced ​​to obtain the image enhanced picture. Image enhancement is performed by using the image information of two different scales, the global and key areas, to improve the recognition effect of welding defects. The pre-trained deep learning model is used to automatically judge the defect type of the image after image enhancement. Compared with the traditional welding defect detection method, it greatly improves the welding defect detection capability, avoids the interference of environmental factors, saves manpower and reduces costs.

[0065] Correspondingly, the present invention also provides a welding defect detection device, equipment and computer-readable storage medium corresponding to the above-mentioned welding defect detection method, which have the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A flowchart of a welding defect detection method according to an embodiment of the present invention;

[0068] Figure 2 This is a diagram of an overall network architecture for welding defect detection in an embodiment of the present invention;

[0069] Figure 3 A processing flow chart of a welding defect detection data stream in an embodiment of the present invention;

[0070] Figure 4 This is a structural block diagram of a welding defect detection device in an embodiment of the present invention;

[0071] Figure 5 This is a structural block diagram of a welding defect detection device in an embodiment of the present invention;

[0072] Figure 6 A schematic diagram of the specific structure of a welding defect detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] Welding defects are common problems in the welding process that may affect the quality and performance of the welded joint. The following are the main types of welding defects and their causes:

[0074] Hot Cracks: rapid cooling, excessive welding stress, and high content of impurities such as sulfur and phosphorus in the material; Cold Cracks: hydrogen embrittlement (introduction of hydrogen during welding), rapid cooling after welding, residual stress in the weld, and welding at low temperature.

[0075] Gas Porosity: Gas does not escape completely during welding, the humidity in the welding area is high, and there is oil or rust on the surface of the welding wire or electrode; Shrinkage Porosity: The weld metal shrinks in volume during cooling and solidification, resulting in the formation of pores.

[0076] Incomplete Penetration: Insufficient welding current, too fast welding speed, unreasonable weld design, or too large or too small weld assembly gap.

[0077] Lack of Fusion: Insufficient welding heat, too fast welding speed, incorrect electrode angle, oxides or other contaminants on the weld surface.

[0078] Weld Bead or Excessive Weld Metal: The welding current is too large, the welding speed is too slow, the welder operates improperly, and the welding rod is improperly selected.

[0079] Undercut: The welding current is too large, the welding speed is uneven, the welding angle is incorrect, and the electrode swings improperly during welding.

[0080] Overlap: welding speed is too slow, welding current is too large, welding rod is improperly selected, and welding rod swings improperly during welding.

[0081] Slag Inclusion: Incomplete slag removal during welding, insufficient welding heat, unreasonable weld design, and poor quality of welding rods or flux.

[0082] Weld Distortion: Uneven heat input during welding, improper welding sequence, insufficient weldment constraint, and improper welding stress release.

[0083] Lamellar Tearing: The welding stress is perpendicular to the lamellar direction of the steel plate, the interlaminar strength of the steel plate is low, and the stress is concentrated during the welding process.

[0084] Brittleness of Weld Metal: Improper selection of welding materials, too long high temperature retention time during welding, too fast cooling speed after welding, and impurities in the weld.

[0085] Cooling Cracks: The cooling speed after welding is too fast, the welding residual stress is too large, and the weld metal is highly brittle.

[0086] Weld leakage: There are holes or lack of fusion in the weld, unreasonable design of the weld joint, and insufficient weld protection during welding.

[0087] Inclusions: Slag or oxides are not completely removed during welding, welding materials are impure, and the welding environment is polluted.

[0088] Surface Irregularities: Unstable current and voltage, uneven welding speed, and improper selection of welding materials during welding.

[0089] Various detection methods are applied in the production process of welded parts to ensure the quality of the final product. At present, the commonly used defect detection methods in the production process of welded parts include: visual inspection, eddy current inspection, ultrasonic inspection, magnetic particle inspection, X-ray inspection, laser scanning inspection, infrared thermal imaging inspection, etc. Although these detection methods have been developed for many years, they still have some shortcomings. In the field of welded parts defect detection, traditional detection methods have their own advantages and disadvantages, are limited by factors such as size, shape, and material properties, and often require a lot of manual operation and professional knowledge. The disadvantages of each of these detection methods still need to be overcome in practical applications to achieve more efficient and comprehensive quality control.

[0090] In recent years, with the development of machine learning and deep learning, artificial intelligence technology has brought new hope to traditional weld defect detection. Using deep learning algorithms, a large amount of welding defect data can be trained, so that the system can automatically learn and identify different types of defects, thereby improving the accuracy and efficiency of defect detection and saving a lot of manpower and equipment costs. For example, image recognition algorithms based on deep learning can be used for the automation of visual inspections, detecting defects by analyzing images of the surface of welds, thereby reducing the burden of manual operations. In addition, deep learning can also be combined with other detection methods, such as ultrasonic or magnetic particle testing, to improve the comprehensiveness and reliability of detection.

[0091] Visual inspection is to judge the defect type of welded parts through manual observation. It is limited by the experience and subjective judgment of the inspectors and has limited ability to identify defects. Laser inspection is to scan the weld surface with laser and use optical methods to detect surface and near-surface defects. The equipment is expensive, the operation is complicated, the surface condition is high, and surface pretreatment is required. It is greatly affected by ambient light and reflection. Traditional computer vision inspection models are used to detect and classify defect photos. In traditional computer vision inspection models, the model's ability is limited to drawing inspection boxes, and it is impossible to conduct further automatic analysis of the inspection results. Manual analysis is still required.

[0092] To this end, the welding defect detection method provided in the present invention improves the welding defect detection capability, avoids interference from environmental factors, saves manpower, and reduces costs.

[0093] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0094] See also Figure 1 , Figure 1 This is a flow chart of an implementation of a welding defect detection method in an embodiment of the present invention. The method may include the following steps:

[0095] S101: Perform defect position detection on the original welding image using a defect position detection model to obtain a target welding defect image including a defect position detection frame.

[0096] The defect position detection model, i.e., the target detection model (DM), is obtained by pre-training. The target detection model can adopt detection models such as the You Only Look Once X (YOLOX) model and the Data-Independent Neural Objectives v2 (DINOv2) model. The original welding image is obtained, and the defect position detection model is used to detect the defect position of the original welding image to obtain a target welding defect image containing a defect position detection frame.

[0097] S102: cropping the target welding defect image according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame.

[0098] After detecting the target welding defect image containing the defect position detection frame, the target welding defect image is cropped according to the defect position detection frame to obtain the welding defect area image in the defect position detection frame. In this way, the key area image with welding defects is cropped out, and other irrelevant areas are cropped out.

[0099] S103: Segment the welding defect area image to obtain sub-images of each welding defect area.

[0100] After the welding defect area image is obtained by cropping, the welding defect area image is segmented to obtain each welding defect area sub-image, thereby segmenting the larger overall welding defect area image into multiple smaller welding defect area sub-images.

[0101] S104: splicing the target welding defect image, the welding defect area image and each welding defect area sub-image to obtain an image after image enhancement.

[0102] After the welding defect area sub-images are obtained by segmentation, the target welding defect image, the welding defect area image and the welding defect area sub-images are concatenated to obtain the image after image enhancement. Image enhancement is achieved by concatenating the target welding defect image, the welding defect area image and the welding defect area sub-images of different scales.

[0103] S105: Using the pre-trained model, the defect type detection is performed on the image after image enhancement to obtain the target welding defect type.

[0104] A model for detecting welding defect types is pre-trained, such as a multimodal large language model (MLLM). After obtaining the image after image enhancement, the pre-trained model is used to detect the defect types in the image after image enhancement to obtain the target welding defect type, that is, the answer to the vision-questioning-answering (VQA) task, thereby realizing automatic detection of welding defect types and saving a lot of manpower and equipment costs.

[0105] It can be seen from the above technical scheme that the defect position detection model is obtained by pre-training, and the defect position detection model is used to detect the defect position of the original welding picture of the welded part, so as to obtain the target welding defect picture containing the defect position detection frame. The image cropping image enhancement method is introduced. By cropping the target welding defect picture, the welding defect area picture of the key area in the defect position detection frame is obtained, and the welding defect area picture is divided into multiple welding defect area sub-pictures, and the target welding defect picture, the welding defect area picture and each welding defect area sub-picture are spliced ​​to obtain the image enhanced picture. Image enhancement is performed by using the image information of two different scales, the global and key areas, to improve the recognition effect of welding defects. The pre-trained deep learning model is used to automatically judge the defect type of the image after image enhancement. Compared with the traditional welding defect detection method, it greatly improves the welding defect detection capability, avoids the interference of environmental factors, saves manpower and reduces costs.

[0106] It should be noted that, based on the above embodiment, the embodiment of the present invention also provides corresponding improved solutions. In the subsequent embodiments, the same steps or corresponding steps as those in the above embodiment can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be repeated one by one in the following improved embodiments.

[0107] See also Figure 2 , Figure 2 The figure is a diagram of the overall network architecture of welding defect detection in an embodiment of the present invention. In stage 1 (Stage-I), the defect position is detected by the defect position detection model. In stage 2 (Stage-II), the defects detected in stage 1 are cropped out of the picture and image enhancement is performed. The pre-trained model is then used to determine the type of welding defects, and the cause analysis of the defects and suggestions for improvement measures are given. Compared with traditional welding defect detection methods, it greatly reduces economic costs, expands the scope of application, saves manpower, and can further give suggestions for improvement in process production, realizing the full process optimization of welding defect detection, cause analysis, and process improvement.

[0108] Figure 2 The solid line in the middle represents the forward propagation process, and the dotted line represents the backward propagation process, that is, the loss calculation. Both forward propagation and backward propagation are required during training, while only forward propagation is required during inference.

[0109] In a specific embodiment of the present invention, segmenting the welding defect area image may include the following steps:

[0110] Step 1: Obtain a list of image resolutions and preset segmentation ratios compatible with the visual encoder in the pre-trained model; wherein the pre-trained model is a model used for defect type detection;

[0111] Step 2: Obtain the image width and image height of the welding defect area image;

[0112] Step 3: When it is determined that image segmentation is required based on the image resolution, image width, image height and segmentation ratio list, the image of the welding defect area is segmented.

[0113] For the convenience of description, the above three steps can be combined for explanation.

[0114] A segmentation ratio list is preset, for example, the segmentation ratio list can be set to [(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]. The image resolution and preset segmentation ratio list compatible with the visual encoder in the pre-trained model for defect type detection are obtained, the image width and image height of the welding defect area image are obtained, and whether the image segmentation is required is determined according to the image resolution, image width, image height and segmentation ratio list. When it is determined that the image segmentation is required according to the image resolution, image width, image height and segmentation ratio list, the welding defect area image is segmented. The segmentation method of the welding defect area image is determined by combining the image resolution, image width, image height and segmentation ratio list, thereby improving the accuracy of image segmentation.

[0115] In a specific implementation of the present invention, when it is determined that image segmentation is required according to the image resolution, image width, image height and segmentation ratio list, segmenting the welding defect area image may include the following steps:

[0116] Step 1: Calculate the aspect ratio based on the image width and image height;

[0117] Step 2: Select the smaller one from the image width and image height;

[0118] Step 3: Calculate the ratio of the smaller one to the corresponding edge resolution in the image resolution;

[0119] Step 4: Find the target segmentation ratio from the segmentation ratio list according to the aspect ratio and ratio value;

[0120] Step 5: Segment the welding defect area image according to the target segmentation ratio.

[0121] For the convenience of description, the above five steps can be combined for explanation.

[0122] After obtaining the image width and image height of the welding defect area image, the aspect ratio is calculated according to the image width and image height. The smaller one is selected from the image width and image height, and the ratio value of the smaller one to the corresponding edge resolution in the image resolution is calculated. The target segmentation ratio is found from the segmentation ratio list according to the aspect ratio and the ratio value, and the welding defect area image is segmented according to the target segmentation ratio. For example, when the aspect ratio is closest to 1:1, if the smaller one of the image width and image height is the image width, and the ratio between the image width and the resolution corresponding to the width edge in the image resolution is greater than 1 and less than 2, then the target segmentation ratio is 2:2, and the welding defect area image is segmented into 4 blocks according to the ratio of 2:2. By finding the most suitable target segmentation ratio from the segmentation ratio list according to the aspect ratio and the ratio value, and segmenting the welding defect area image according to the target segmentation ratio, the image segmentation accuracy is further improved.

[0123] In a specific implementation of the present invention, searching for a target segmentation ratio from a segmentation ratio list according to the aspect ratio and the ratio value may include the following steps:

[0124] Step 1: round up the ratio value to get an integer ratio value;

[0125] Step 2: Find the initial segmentation ratio with the smallest difference from the aspect ratio from the segmentation ratio list;

[0126] Step 3: Multiply the integer ratio value by the initial segmentation ratio to obtain the target segmentation ratio.

[0127] For the convenience of description, the above three steps can be combined for explanation.

[0128] The slicing ratios in the slicing ratio list may have the same ratio value for the number of slicing shares for width and height, but different actual numbers of slicing shares, such as (1, 1), (2, 2), (3, 3). Round up the ratio value to get an integer ratio value, find the initial slicing ratio with the smallest difference from the aspect ratio in the slicing ratio list, and multiply the integer ratio value with the initial slicing ratio, that is, use the integer ratio value to multiply the numerator and denominator in the initial slicing ratio to get the target slicing ratio. For example, when the integer ratio value is 2 and the initial slicing ratio is (1, 1), the calculated target slicing ratio is (2, 2). By rounding up the ratio value and multiplying the integer ratio value with the initial slicing ratio to get the target slicing ratio, the integrity of the image content contained in each weld defect area sub-image after slicing is ensured.

[0129] In a specific embodiment of the present invention, splicing the target welding defect image, the welding defect area image and each welding defect area sub-image may include the following steps:

[0130] Step 1: Adjust the target welding defect image to the size of the image resolution to obtain the adjusted welding defect image;

[0131] Step 2: Adjust the welding defect area image to the size of the image resolution to obtain the adjusted welding defect area image;

[0132] Step 3: adjusting each welding defect area sub-image to the size of the image resolution to obtain each adjusted welding defect area sub-image;

[0133] Step 4: splicing the adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image.

[0134] For the convenience of description, the above four steps can be combined for explanation.

[0135] The visual encoder in the pre-trained model is only compatible with specific image resolutions. After obtaining the target welding defect image, welding defect area image, and each welding defect area sub-image, the target welding defect image is adjusted to the size of the image resolution to obtain the adjusted welding defect image, the welding defect area image is adjusted to the size of the image resolution to obtain the adjusted welding defect area image, each welding defect area sub-image is adjusted to the size of the image resolution to obtain each adjusted welding defect area sub-image, and the adjusted welding defect image, the adjusted welding defect area image, and each adjusted welding defect area sub-image are spliced ​​to obtain an image enhanced image. By adjusting the image resolutions of the target welding defect image, the welding defect area image, and each welding defect area sub-image to the size of the image resolution compatible with the visual encoder, fast and accurate detection of defect types is achieved.

[0136] In a specific embodiment of the present invention, splicing the adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image to obtain an image enhanced image may include the following steps:

[0137] The pixel values ​​of the adjusted welding defect image, the pixel values ​​of the adjusted welding defect region image, and the pixel values ​​of each adjusted welding defect region sub-image are concatenated to obtain a target tensor;

[0138] Accordingly, using the pre-trained model to detect defect types in the enhanced image may include the following steps:

[0139] Use the pre-trained model to detect defect types on the target tensor.

[0140] After adjusting the resolution of the target welding defect image, welding defect area image and each welding defect area sub-image to obtain the adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image, the pixel value of the adjusted welding defect image, the pixel value of the adjusted welding defect area image and the pixel value of each adjusted welding defect area sub-image are spliced ​​to obtain the target tensor, and the pre-trained model is used to detect the defect type of the target tensor. By splicing the pixel value of the adjusted welding defect image, the pixel value of the adjusted welding defect area image and the pixel value of each adjusted welding defect area sub-image, the defect type detection is performed on the spliced ​​target tensor, which improves the detection efficiency and detection accuracy of defect type detection.

[0141] In a specific embodiment of the present invention, using a pre-trained model to detect defect types on a target tensor may include the following steps:

[0142] Step 1: Use the visual encoder to perform convolution operation on the target tensor to obtain visual features;

[0143] Step 2: Use the fully connected layer in the pre-trained model to project the visual features to obtain the image token;

[0144] Step 3: Get the preset prompt word token;

[0145] Step 4: Concatenate the image token and the prompt word token to obtain the input token;

[0146] Step 5: Use the pre-trained language model in the pre-trained model to detect defect types based on the input token.

[0147] For the convenience of description, the above five steps can be combined for explanation.

[0148] After the pixel values ​​of the adjusted welding defect image, the pixel values ​​of the adjusted welding defect area image, and the pixel values ​​of each adjusted welding defect area sub-image are spliced ​​to obtain the target tensor, the target tensor is convolved using the visual encoder to obtain visual features, and the visual features are projected using the fully connected layer in the pre-trained model to obtain image tokens. A prompt word token is pre-set, and the preset prompt word token (Text Token) is obtained. The image token and the prompt word token are spliced ​​to obtain the input token (Input token). The pre-trained language model in the pre-trained model is used to detect the defect type according to the input token. By setting the prompt word token, the defect type is detected according to the input token obtained by splicing the image token and the prompt word token, which further improves the accuracy of defect type detection.

[0149] The pre-trained language model may be a large language model (LLM).

[0150] In a specific implementation of the present invention, obtaining a preset prompt word token may include the following steps:

[0151] The marker in the pre-trained model is used to encode the preset natural language prompt words to obtain prompt word tokens.

[0152] See also Figure 3 , Figure 3 The present invention is a processing flow chart of a welding defect detection data stream in an embodiment of the present invention. The pre-trained model also includes a tokenizer that matches the large language model. Natural language prompt words are pre-set, and the preset natural language prompt words are encoded using the tokenizer in the pre-trained model to obtain prompt word tokens. By pre-setting a tokenizer for encoding and decoding prompt words, a fast and accurate conversion between prompt words in vector form and natural language prompt words recognizable to users during model data transmission is achieved.

[0153] In a specific embodiment of the present invention, using a pre-trained language model in a pre-trained model to detect defect types according to input tokens to obtain target welding defect types may include the following steps:

[0154] Step 1: Input the input token into the pre-trained language model to obtain the output token of the pre-trained language model;

[0155] Step 2: Use the marker in the pre-trained model to decode the output token to obtain the target welding defect type in natural language.

[0156] For the convenience of description, the above two steps can be combined for explanation.

[0157] The input token is input into the pre-trained language model to obtain the output token of the pre-trained language model. The output token is decoded by the marker in the pre-trained model to obtain the target welding defect type in natural language. By using the marker to decode the output token in vector form, the target welding defect type in natural language is obtained, which greatly improves the detection efficiency and accuracy of welding defect type detection.

[0158] In a specific implementation of the present invention, obtaining a preset prompt word token may include the following steps:

[0159] Get a hint token containing the defect type, cause, and improvement measures.

[0160] Accordingly, using the pre-trained language model in the pre-trained model to detect defect types according to the input token to obtain the target welding defect type may include the following steps:

[0161] The pre-trained language model is used to determine the target welding defect type, defect cause and defect improvement measures based on the input token.

[0162] Obtain prompt word tokens containing defect types, causes, and improvement measures, and use the pre-trained language model to determine the target welding defect types, causes, and improvement measures based on the input tokens. By pre-setting prompt word tokens containing defect types, causes, and improvement measures, the pre-trained language model can be used to determine the target welding defect types, causes, and improvement measures based on the input tokens, achieving full process optimization of welding defect detection, cause analysis, and process improvement.

[0163] In a specific embodiment of the present invention, obtaining a prompt word token including defect type, cause and improvement measure suggestion may include the following steps:

[0164] Get the prompt token containing the defect type, cause, improvement suggestion and image path;

[0165] Accordingly, using the pre-trained language model to determine the target welding defect type, defect cause and defect improvement measures according to the input token may include the following steps:

[0166] The pre-trained language model is used to determine the target welding defect type, defect cause, defect improvement measures and image path information based on the input token.

[0167] Obtain prompt word tokens containing defect types, causes, improvement measures and image paths, and use the pre-trained language model to determine the target welding defect types, defect causes, defect improvement measures and image path information based on the input tokens. By setting prompt word tokens containing defect types, causes, improvement measures and image paths, comprehensive acquisition of target welding defect types, defect causes, defect improvement measures and image path information is achieved, making it convenient to determine the image source of the original welding image based on the image path information.

[0168] In a specific embodiment of the present invention, the method may further include a training process of a defect position detection model, and the training process of the defect position detection model may include the following steps:

[0169] Step 1: Obtain each welding defect image sample containing a welding defect detection frame;

[0170] Step 2: Use each welding defect image sample to train the initial detection model to obtain a defect location detection model.

[0171] For the convenience of description, the above two steps can be combined for explanation.

[0172] The welding defect detection method provided by the embodiment of the present invention may also include a training process of a defect position detection model. The training process of the defect position detection model may include obtaining samples of each welding defect image containing a welding defect detection frame, and using each welding defect image sample to train an initial detection model to obtain a defect position detection model. By pre-collecting a large number of samples of each welding defect image containing a welding defect detection frame, and using each welding defect image sample to train the initial detection model, the accuracy and efficiency of the defect position detection model obtained by training in detecting defect positions in welding images are improved.

[0173] During training, each data sample includes a picture and two labels. The picture is a picture of a welding defect. The first label is the bounding box of the welding defect, which is used to train the defect location detection model. The second label is the text answer of the defect type, cause analysis, and improvement measures, which is used to fine-tune the multimodal large model.

[0174] During training, a sample contains an image and a dictionary with strings as key-value pairs. The format of the dictionary is as follows:

[0175] {"query":" / n1. Determine the type of welding defect in the figure. 2. Analyze the cause of this defect. 3. Give suggestions for improvement measures. ","answer":"<defect type><cause analysis><improvement measures>","image_path":"<image path>"}.

[0176] In the dictionary, <> is a placeholder. Indicates that the image token is inserted at this position when the tokens are spliced, while other placeholders are used to insert different strings according to the actual situation. For example, suppose the image path is data / image / image1.jpg and the defect type in the image is wrinkling.

[0177] The following is a sample example:

[0178] {"query":" / n1. Determine the type of welding defect in the figure. 2. Analyze the cause of this defect. 3. Give suggestions for improvement measures. ","answer":"<Unfused><Insufficient welding heat input><Increase welding heat input to ensure that the molten pool temperature is high enough to promote full fusion of weld metal and base material>","image_path":"<Image path>"}.

[0179] like Figure 3 As shown in Figure 1, in stage 1, the data flow processing flow of the defect location detection model is relatively simple, which is to input a welding defect image and output an image with a detection frame, so we will not go into details. In the image cropping processor (Crop Processor) and stage 2, the data flow processing is more complicated. Figure 3 In the figure, the image cropping processor, the marker, the fully connected layer, the visual encoder, and the pre-trained language model represent model components. The image cropping processor and the marker represent model components whose weights are locked and do not participate in the gradient backpropagation to update parameters. The fully connected layer, the visual encoder, and the pre-trained language model represent model components whose weights are opened and whose parameters are updated during backpropagation. The target welding defect image, the adjusted welding defect image, the adjusted welding defect area image, each adjusted welding defect area sub-image, the prompt word, the prompt word token, the image token, the input token, the output token, the loss, and the truth token represent data flows. The target welding defect image, the adjusted welding defect image, the adjusted welding defect area image, each adjusted welding defect area sub-image, the prompt word, the prompt word token, the image token, the input token, the output token, and the solid arrows represent the data flows shared by forward and backward propagation. The target welding defect image, the adjusted welding defect image, the adjusted welding defect area image, each adjusted welding defect area sub-image, the prompt word, the prompt word token, the image token, the input token, the output token, and the dotted arrows represent the data flows unique to backpropagation, i.e., the loss calculation. The functions of each important module will be introduced below.

[0180] In the image cropping processor, the following operations are performed on the target welding defect image with the detection frame output by the defect location detection model:

[0181] (1) Based on the target welding defect image output by the defect location detection model, the image is cropped to crop the defective part out of the image. At the same time, the target welding defect image output by the defect location detection model is reshaped (resized) to the resolution supported by the visual encoder to obtain the adjusted welding defect image.

[0182] (2) The cropped welding defect area image is further segmented, and the cropped welding defect area image is reshaped to a resolution supported by the visual encoder to obtain an adjusted welding defect area image.

[0183] (3) Each of the segmented welding defect area sub-images is reshaped to a resolution supported by the visual encoder to obtain each adjusted welding defect area sub-image.

[0184] (4) The adjusted welding defect picture, the adjusted welding defect area picture, and each adjusted welding defect area sub-picture are displayed. Figure 3 The pixel values ​​of images with the same information scale and resolution are spliced ​​together, and the spliced ​​target tensor has the image information of three information scales: global, key area, and key area local, thus enhancing the image information. The pixel values ​​of the spliced ​​image are used as the input to the visual encoder.

[0185] The visual encoder is used to extract the visual features of the image. It can use pre-trained models such as the Eva-Clip model, the Vision Transformer (ViT), etc. In traditional computer vision models, since the dimensions of processing layers such as convolutional layers are fixed, a visual encoder can only receive images of a fixed resolution.

[0186] The following example illustrates the processing of the image cropping processor. One of the key parameters is:

[0187] The segmentation ratio list is [(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)], which is used to determine the number of segments when further segmenting the cropped welding defect area image.

[0188] Assuming that the image resolution compatible with the current visual encoder is 448×448, and assuming that the size of the original welding image is 1496×976, the size of the defect box after detection is 214×528.

[0189] The processing process of the image cropping processor is as follows:

[0190] (1) Based on the target welding defect image output by the defect location detection model, the image is cropped and the defect part of 214×528 is cropped out from the image separately. At the same time, the target welding defect image of 1496×976 output by the target welding defect image is reshaped to 448×448 to obtain the adjusted welding defect image.

[0191] (2) The cropped 214×528 welding defect area image is further segmented. The aspect ratio 214 / 528≈0.4 is compared with the segmentation ratio in the segmentation ratio list to find the closest ratio, that is, 1 / 3≈0.33. Therefore, the most appropriate segmentation method is to cut it into three parts according to the long side (that is, the height). At the same time, the cropped 214×528 welding defect area image is reshaped to 448×448 to obtain the adjusted welding defect area image.

[0192] (3) The three images are reshaped to 448×448 to obtain the adjusted welding defect area sub-images.

[0193] (4) The adjusted welding defect picture, the adjusted welding defect area picture, and each adjusted welding defect area sub-picture are displayed. Figure 3 The pixel values ​​of the images with different information scales and resolutions of 448×448 are spliced ​​together, and the spliced ​​target tensor has the image information of three information scales: global, key area, and key area local, thus enhancing the image information. The pixel values ​​of the spliced ​​images are used as the input to the visual encoder, which then performs convolution operations on the pixel values ​​of the spliced ​​images to extract the image features.

[0194] The fully connected layer (Multilayer Perceptron, MLP) is used to project the visual features extracted by the visual encoder to obtain image tokens, so as to align with the token dimensions that the pre-trained language model can accept. The weights of the fully connected layer are turned on and participate in back-propagation to update the parameters.

[0195] The tagger that comes with the pre-trained language model is used to process the input and output of the text. The input text is converted into a word vector by the tagger, and after being spliced ​​with the image token, the input token is obtained. The input token is input into the pre-trained language model, and then the output token is obtained. During training, the output token and the pre-set true value token are directly calculated for loss, and the decision on whether to continue iterative training is determined based on the loss calculation result. By performing iterative training control based on the loss calculation result of the output token and the true value token, the accuracy of welding defect detection of the pre-trained model is guaranteed and overfitting is avoided. During inference, the output token is sent to the tagger for decoding, and finally a natural language answer is obtained. The weights of the pre-trained language model are also turned on, that is, the full parameter fine-tuning method is used during training.

[0196] See also Figure 4 , Figure 4 This is a structural block diagram of a welding defect detection device in an embodiment of the present invention, and the device may include:

[0197] A defect position detection module 41 is used to perform defect position detection on the original welding image using a defect position detection model to obtain a target welding defect image containing a defect position detection frame;

[0198] A cropping module 42 is used to crop the target welding defect image according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame;

[0199] The image segmentation module 43 is used to segment the welding defect area image to obtain each welding defect area sub-image;

[0200] An image enhancement module 44 is used to splice the target welding defect image, the welding defect area image and each welding defect area sub-image to obtain an image enhanced image;

[0201] The welding defect type acquisition module 45 is used to use the pre-trained model to detect the defect type of the image after image enhancement to obtain the target welding defect type.

[0202] It can be seen from the above technical scheme that the defect position detection model is obtained by pre-training, and the defect position detection model is used to detect the defect position of the original welding picture of the welded part, so as to obtain the target welding defect picture containing the defect position detection frame. The image cropping image enhancement method is introduced. By cropping the target welding defect picture, the welding defect area picture of the key area in the defect position detection frame is obtained, and the welding defect area picture is divided into multiple welding defect area sub-pictures, and the target welding defect picture, the welding defect area picture and each welding defect area sub-picture are spliced ​​to obtain the image enhanced picture. Image enhancement is performed by using the image information of two different scales, the global and key areas, to improve the recognition effect of welding defects. The pre-trained deep learning model is used to automatically judge the defect type of the image after image enhancement. Compared with the traditional welding defect detection method, it greatly improves the welding defect detection capability, avoids the interference of environmental factors, saves manpower and reduces costs.

[0203] In a specific embodiment of the present invention, the image segmentation module includes:

[0204] The resolution and list acquisition submodule is used to obtain the image resolution and preset segmentation ratio list compatible with the visual encoder in the pre-trained model; wherein the pre-trained model is a model used for defect type detection;

[0205] The image width and height acquisition submodule is used to obtain the image width and image height of the welding defect area image;

[0206] The image segmentation submodule is used to segment the welding defect area image when it is determined that image segmentation is required based on the image resolution, image width, image height and segmentation ratio list.

[0207] In a specific embodiment of the present invention, the image segmentation submodule includes:

[0208] A picture aspect ratio calculation unit, used to calculate the aspect ratio according to the picture width and the picture height;

[0209] A selection subunit is used to select the smaller one from the image width and the image height;

[0210] A resolution ratio value calculation unit, used to calculate the ratio value of the smaller one to the corresponding edge resolution in the picture resolution;

[0211] A segmentation ratio search unit, used for searching a target segmentation ratio from a segmentation ratio list according to the aspect ratio and the ratio value;

[0212] The image segmentation unit is used to segment the welding defect area image according to the target segmentation ratio.

[0213] In a specific implementation of the present invention, the slicing ratio search unit includes:

[0214] The integer ratio value obtaining subunit is used to round up the ratio value to obtain an integer ratio value;

[0215] An initial segmentation ratio search subunit is used to search the initial segmentation ratio with the smallest difference from the aspect ratio from the segmentation ratio list;

[0216] The target segmentation ratio obtaining subunit is used to perform a product operation on the integer ratio value and the initial segmentation ratio to obtain the target segmentation ratio.

[0217] In a specific embodiment of the present invention, the image enhancement module 44 includes:

[0218] A welding defect image adjustment submodule is used to adjust the target welding defect image to the size of the image resolution to obtain an adjusted welding defect image;

[0219] A welding defect area image adjustment submodule is used to adjust the welding defect area image to the size of the image resolution to obtain an adjusted welding defect area image;

[0220] A welding defect area sub-image adjustment sub-module is used to adjust each welding defect area sub-image to the size of the image resolution to obtain each adjusted welding defect area sub-image;

[0221] The image stitching submodule is used to stitch the adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image.

[0222] In a specific embodiment of the present invention, the image stitching submodule specifically stitches the pixel values ​​of the adjusted welding defect image, the pixel values ​​of the adjusted welding defect area image, and the pixel values ​​of each adjusted welding defect area sub-image to obtain a unit of the target tensor;

[0223] The welding defect type acquisition module is specifically a module that uses a pre-trained model to detect the defect type of the target tensor.

[0224] In a specific embodiment of the present invention, the welding defect type obtaining module includes:

[0225] The visual feature acquisition submodule is used to perform convolution operations on the target tensor using the visual encoder to obtain visual features;

[0226] The image token acquisition submodule is used to project the visual features using the fully connected layer in the pre-trained model to obtain the image token;

[0227] A prompt word token acquisition submodule is used to acquire a preset prompt word token;

[0228] An input token acquisition submodule is used to concatenate image tokens and prompt word tokens to obtain input tokens;

[0229] The defect type detection submodule is used to detect the defect type according to the input token using the pre-trained language model in the pre-trained model.

[0230] In a specific embodiment of the present invention, the prompt word token acquisition submodule is specifically a module for encoding preset natural language prompt words using a marker in a pre-trained model to obtain prompt word tokens.

[0231] In a specific embodiment of the present invention, the defect type detection submodule includes:

[0232] An output token obtaining unit, used for inputting an input token into a pre-trained language model to obtain an output token of the pre-trained language model;

[0233] The defect type acquisition module is used to decode the output token using the marker in the pre-trained model to obtain the target welding defect type in natural language.

[0234] In a specific embodiment of the present invention, the prompt word token acquisition submodule is specifically a module for acquiring prompt word tokens including defect type, cause and improvement measure suggestion;

[0235] The defect type detection submodule is specifically a module that uses a pre-trained language model to determine the target welding defect type, defect cause and defect improvement measures based on the input token.

[0236] In a specific embodiment of the present invention, the prompt word token acquisition submodule is specifically a module for acquiring a prompt word token including defect type, cause, improvement measure suggestion and image path;

[0237] The defect type detection submodule is specifically a module that uses a pre-trained language model to determine the target welding defect type, defect cause, defect improvement measures and image path information according to the input token.

[0238] In a specific embodiment of the present invention, the device may further include:

[0239] A welding defect image sample acquisition module is used to acquire each welding defect image sample containing a welding defect detection frame;

[0240] The defect position detection model acquisition module is used to train the initial detection model using each welding defect image sample to obtain the defect position detection model.

[0241] Corresponding to the above method embodiment, see Figure 5 , Figure 5 This is a schematic diagram of a welding defect detection device provided by the present invention, and the device may include:

[0242] A memory 332, for storing computer programs;

[0243] The processor 322 is used to implement the steps of the welding defect detection method of the above method embodiment when executing the computer program.

[0244] For details, please refer to Figure 6 , Figure 6 A specific structural diagram of a welding defect detection device provided in this embodiment, which may have relatively large differences due to different configurations or performances, may include a processor (central processing units, CPU) 322 (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 can be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the welding defect detection device 301.

[0245] The welding defect detection device 301 may also include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .

[0246] The steps in the welding defect detection method described above can be implemented by the structure of the welding defect detection device.

[0247] Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:

[0248] The defect position detection model is used to detect the defect position of the original welding image to obtain a target welding defect image containing a defect position detection frame; the target welding defect image is cropped according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame; the welding defect area image is segmented to obtain each welding defect area sub-image; the target welding defect image, the welding defect area image and each welding defect area sub-image are spliced ​​to obtain an image after image enhancement; the pre-trained model is used to detect the defect type of the image after image enhancement to obtain the target welding defect type.

[0249] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0250] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be elaborated here.

[0251] Corresponding to the above method embodiment, the present invention also provides a computer program product, including a computer program, which implements the steps of the above welding defect detection method when executed by a processor.

[0252] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device, computer-readable storage medium, and computer program product disclosed in the embodiment, since they correspond to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0253] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A welding defect detection method, characterized in that: include: The defect position detection model is used to detect the defect position of the original welding image, and a target welding defect image containing a defect position detection frame is obtained; Cropping the target welding defect image according to the defect position detection frame to obtain a welding defect area image in the defect position detection frame; Segmenting the welding defect area image to obtain sub-images of each welding defect area; splicing the target welding defect image, the welding defect area image and each welding defect area sub-image to obtain an image enhanced image; Using the pre-trained model to detect the defect type of the image after image enhancement, to obtain the target welding defect type; The target welding defect image, the welding defect area image and each welding defect area sub-image are spliced, including: Get the image resolution compatible with the visual encoder in the pre-trained model; Adjusting the target welding defect image to the size of the image resolution to obtain an adjusted welding defect image; Adjusting the welding defect area image to the size of the image resolution to obtain an adjusted welding defect area image; Adjusting each welding defect area sub-image to the size of the image resolution to obtain each adjusted welding defect area sub-image; The adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image are spliced.

2. The welding defect detection method according to claim 1, characterized in that: Segmenting the welding defect area image includes: Obtaining a preset segmentation ratio list; wherein the pre-trained model is a model for defect type detection; Obtaining the image width and image height of the welding defect area image; When it is determined that image segmentation is required according to the image resolution, the image width, the image height and the segmentation ratio list, the welding defect area image is segmented.

3. The welding defect detection method according to claim 2, characterized in that: When it is determined that image segmentation is required according to the image resolution, the image width, the image height and the segmentation ratio list, segmenting the welding defect area image includes: Calculate the aspect ratio according to the image width and the image height; Select the smaller one from the image width and the image height; Calculate the ratio of the smaller one to the corresponding edge resolution in the picture resolution; Searching for a target segmentation ratio from the segmentation ratio list according to the aspect ratio and the ratio value; The welding defect area image is segmented according to the target segmentation ratio.

4. The welding defect detection method according to claim 3, characterized in that: Searching for a target segmentation ratio from the segmentation ratio list according to the aspect ratio and the ratio value includes: Rounding up the ratio value to obtain an integer ratio value; Searching the initial segmentation ratio with the smallest difference from the aspect ratio from the segmentation ratio list; The target segmentation ratio is obtained by multiplying the integer ratio value by the initial segmentation ratio.

5. The welding defect detection method according to claim 1, characterized in that: The adjusted welding defect image, the adjusted welding defect area image and each adjusted welding defect area sub-image are spliced ​​to obtain the image enhanced image, including: splicing the pixel values ​​of the adjusted welding defect image, the pixel values ​​of the adjusted welding defect region image, and the pixel values ​​of each adjusted welding defect region sub-image to obtain a target tensor; Accordingly, the pre-trained model is used to detect defect types on the image after image enhancement, including: The pre-trained model is used to detect defect types on the target tensor.

6. The welding defect detection method according to claim 5, characterized in that: Using the pre-trained model to detect defect types on the target tensor includes: Using the visual encoder to perform a convolution operation on the target tensor to obtain a visual feature; Projecting the visual features using a fully connected layer in the pre-trained model to obtain an image token; Get the preset prompt word token; Concatenate the image token and the prompt word token to obtain an input token; A pre-trained language model in the pre-trained model is used to perform defect type detection according to the input token.

7. The welding defect detection method according to claim 6, characterized in that: Get the preset prompt tokens, including: The preset natural language prompt word is encoded using the marker in the pre-trained model to obtain the prompt word token.

8. The welding defect detection method according to claim 6, characterized in that: Using the pre-trained language model in the pre-trained model to detect defect types according to the input token to obtain target welding defect types, including: Inputting the input token into the pre-trained language model to obtain an output token of the pre-trained language model; The output token is decoded using the marker in the pre-trained model to obtain the target welding defect type in natural language.

9. The welding defect detection method according to claim 6, characterized in that: Get the preset prompt tokens, including: Get a hint token containing the defect type, cause, and improvement measures. Accordingly, the pre-trained language model in the pre-trained model is used to detect the defect type according to the input token to obtain the target welding defect type, including: The pre-trained language model is used to determine the target welding defect type, defect cause and defect improvement measures according to the input token.

10. The welding defect detection method according to claim 9, characterized in that: Get a hint token containing the defect type, cause, and improvement suggestions, including: Obtain a prompt word token including the defect type, the cause, the improvement measure suggestion and the image path; Accordingly, the pre-trained language model is used to determine the target welding defect type, defect cause and defect improvement measures according to the input token, including: The pre-trained language model is used to determine the target welding defect type, the defect cause, the defect improvement measures and image path information according to the input token.

11. The welding defect detection method according to claim 1, characterized in that: The method also includes a training process of the defect position detection model, wherein the training process of the defect position detection model includes: Obtaining samples of each welding defect image including a welding defect detection frame; The initial detection model is trained using each welding defect image sample to obtain the defect position detection model.

12. A welding defect detection device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the welding defect detection method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the welding defect detection method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the welding defect detection method according to any one of claims 1 to 11 are implemented.

Images