Welding seam positioning method and device, storage medium and electronic equipment

Abstract

The invention relates to the technical field of welding seam tracking, in particular to a welding seam positioning method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring regional information of a welding seam; determining a binary image corresponding to the welding seam according to the regional information; and according to the binarized image, determining the position information of the welding seam feature points of the welding seam so as to position the welding seam. According to the method, firstly, the welding seam is coarsely positioned by acquiring the regional information of the welding seam, and then the position information of the welding seam feature points of the welding seam is determined by the binarization image corresponding to the welding seam, so that the welding seam is accurately positioned. The problems that a pure visual recognition mode is prone to being affected by complex backgrounds, ambient light interference, object surface reflection and other factors, and misrecognition or recognition failure is caused are solved. The accuracy and the stability of welding seam tracking are improved, and a guarantee is further provided for normal operation of industrial production.

Description

Technical Field

[0001] The present disclosure relates to the technical field of weld seam tracking, and particularly relates to a weld seam positioning method, device, storage medium and electronic device. Background Art

[0002] Welding technology plays a crucial role in industrial production. Currently, automated welding has become the mainstream trend in the development of this field. However, in actual operation, factors such as manufacturing, assembly, fixture design, and welding deformation inevitably introduce errors, which requires the introduction of a weld seam tracking system to accurately compensate for these errors. With its powerful recognition ability, the weld seam tracking system can accurately capture the position of the weld seam, calculate the deviation value, and transmit these values to the motion system, enabling the welding actuator to accurately align with the weld seam, thereby ensuring excellent welding quality.

[0003] Currently, the widely used weld seam tracking system mainly relies on the deflection phenomenon of the line laser on the weld seam surface. By using an industrial camera to capture the deflected line laser image, extracting the coordinates of the deflection position, and calculating the actual position of the weld seam based on this. However, when facing the butt welding of thin plate parts with complex structures, especially when using the long focal length laser autogenous welding technology, higher requirements are imposed on the installation height of the weld seam tracking system. At the same time, as the installation height increases, conventional weld seam tracking methods often struggle to identify those butt welds with small gaps or even "zero gaps", resulting in poor accuracy and stability of weld seam tracking.

[0004] Therefore, how to improve the accuracy and stability of weld seam tracking has become a key technical problem to be solved currently. Summary of the Invention

[0005] The main purpose of the present disclosure is to provide a weld seam positioning method, device, storage medium and electronic device, aiming to solve the technical problems of poor accuracy and low stability of weld seam tracking in the prior art.

[0006] To achieve the above object, the present disclosure proposes a weld seam positioning method, including:

[0007] Obtain the regional information of the weld seam;

[0008] Determine the binary image corresponding to the weld seam according to the regional information;

[0009] Determine the position information of the weld feature points of the weld seam according to the binary image to position the weld seam.

[0010] Optionally, the determining the binary image corresponding to the weld seam according to the regional information includes:

[0011] Extract the regional image of the area where the weld seam is located according to the said regional information;

[0012] Preprocess the said regional image to obtain a preprocessed regional image;

[0013] Perform threshold segmentation on the preprocessed regional image to obtain the binary image.

[0014] Optionally, the preprocessing of the said regional image to obtain a preprocessed regional image includes:

[0015] Perform grayscale processing on the said regional image to obtain a grayscale image corresponding to the regional image;

[0016] Based on the Hough transform, perform filtering processing on the grayscale image to obtain the preprocessed regional image.

[0017] Optionally, the determination of the position information of the weld seam feature points according to the said binary image includes:

[0018] Perform linear fitting on the said binary image to obtain the center line equation and the contour line corresponding to the weld seam;

[0019] Perform intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld seam feature points, and the feature point coordinates are used to represent the position of the weld seam in the image coordinate system;

[0020] Determine the position information of the weld seam feature points according to the feature point coordinates.

[0021] Optionally, after performing intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld seam feature points, it further includes:

[0022] Determine the first coordinates corresponding to the start point and the end point of the weld seam according to the feature point coordinates;

[0023] Obtain the second coordinates corresponding to the start point and the end point of the weld seam, and the second coordinates are obtained by translating the workpiece corresponding to the weld seam according to a preset translation method;

[0024] Determine the relationship formula between the position deviation of the workpiece and the position coordinates obtained in the image coordinate system according to the first coordinates and the second coordinates;

[0025] Determine the deviation value of the weld seam feature points in the world coordinate system according to the relationship formula, so as to determine the position information of the weld seam feature points according to the deviation value.

[0026] Optionally, the obtaining of the regional information of the weld seam includes:

[0027] Obtain the image to be recognized;

[0028] Input the image to be recognized into the trained weld recognition model to obtain the weld recognition result output by the weld recognition model, where the weld recognition result is used to characterize the position information of the weld in the weld image data; wherein, the weld recognition model is trained based on the weld image data;

[0029] Obtain the weld area according to the weld recognition result.

[0030] Optionally, the training process of the weld recognition model includes:

[0031] Collect weld image data;

[0032] Preprocess the weld image data to obtain a weld data set, where the weld data set includes weld image data marked with the position of the weld;

[0033] Based on the YOLO object detection algorithm, train the initial recognition model according to the weld data set to obtain the weld recognition model.

[0034] Optionally, the preprocessing of the weld image data includes:

[0035] Obtain a distortion model, where the distortion model includes radial distortion and tangential distortion;

[0036] Perform distortion correction on the weld image data through the distortion model to obtain the weld image data after distortion correction.

[0037] In addition, to achieve the above object, the present disclosure also provides a weld positioning device, where the weld positioning device includes:

[0038] An acquisition module for acquiring the regional information of the weld;

[0039] A determination module for determining the binary image corresponding to the weld according to the regional information;

[0040] A positioning module for determining the position information of the weld feature points of the weld according to the binary image to position the weld.

[0041] In addition, to achieve the above object, the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the above method.

[0042] In addition, to achieve the above object, the present disclosure also provides an electronic device, which includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the above method.

[0043] In addition, to achieve the above object, the present disclosure also provides a computer program product, which implements the above method when being run by a processor.

[0044] The beneficial effects that the present disclosure can achieve.

[0045] The technical solution proposed in the embodiment of the present disclosure is to obtain the region information of the weld seam, determine the binary image corresponding to the weld seam according to the region information, and determine the position information of the weld feature points of the weld seam according to the binary image to position the weld seam. Through this technical solution, the weld seam is roughly positioned by obtaining the region information of the weld seam first, and then the position information of the weld feature points of the weld seam is determined by the binary image corresponding to the weld seam to accurately position the weld seam. In this way, the problem that the pure vision recognition method is easily affected by factors such as complex background, environmental light interference, and object surface reflection, resulting in misrecognition or non-recognition, is solved. The accuracy and stability of weld seam tracking are improved, and further guarantee is provided for the normal progress of industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0047] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment related to the solution of the embodiment of the present disclosure;

[0048] Figure 2 It is a schematic flowchart of a weld seam positioning method related to the solution of the embodiment of the present disclosure;

[0049] Figure 3 It is an image of the position information of a weld seam related to the solution of the embodiment of the present disclosure;

[0050] Figure 4 It is a binary image of a weld seam related to the solution of the embodiment of the present disclosure;

[0051] Figure 5 It is an image after intersection calculation related to the solution of the embodiment of the present disclosure;

[0052] Figure 6The block diagram of a weld positioning device is involved in the solution of the embodiment of the present disclosure.

[0053] The realization of the purpose, functional features and advantages of the present disclosure will be further described with reference to the accompanying drawings in combination with the embodiments. Specific embodiments

[0054] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without making creative efforts belong to the scope of protection of the present disclosure.

[0055] Refer to Figure 1 , Figure 1 which is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present disclosure.

[0056] Generally, the device includes: at least one processor 301, a memory 302, and a weld positioning program stored on the memory 302 and executable on the processor 301. The weld positioning program is configured to implement the steps of the weld positioning method as described above.

[0057] The processor 301 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 301 may be implemented in at least one of the following hardware forms: DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. The processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process operations related to the weld positioning method, so that the weld positioning method model can be autonomously trained and learned to improve efficiency and accuracy.

[0058] The memory 302 may include one or more storage media, which may be non-transitory. The memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory storage media in the memory 302 is used to store at least one instruction for being executed by the processor 301 to implement the weld seam positioning method provided in the method embodiments of the present disclosure.

[0059] In some embodiments, the terminal may further optionally include: a communication interface 303 and at least one peripheral device. The processor 301, the memory 302, and the communication interface 303 may be connected through a bus or signal lines. Each peripheral device may be connected to the communication interface 303 through a bus, signal lines, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 304, a display screen 305, and a power supply 306.

[0060] The communication interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 may be implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0061] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 304 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 304 may communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: a metropolitan area network, each generation of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 304 may further include a circuit related to NFC (Near Field Communication), and the present disclosure does not limit this.

[0062] The display screen 305 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 305 is a touch display screen, the display screen 305 also has the ability to collect touch signals on or above the surface of the display screen 305. The touch signals can be input as control signals to the processor 301 for processing. At this time, the display screen 305 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 305 can be one, the front panel of the electronic device; in other embodiments, the display screen 305 can be at least two, respectively arranged on different surfaces of the electronic device or in a folding design; in still other embodiments, the display screen 305 can be a flexible display screen, arranged on the curved surface or folding surface of the electronic device. Even, the display screen 305 can also be set to an irregular non-rectangular shape, that is, a special-shaped screen. The display screen 305 can be prepared from materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0063] The power supply 306 is used to supply power to each component in the electronic device. The power supply 306 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When the power supply 306 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology. Those skilled in the art can understand that Figure 1 the structure shown in does not constitute a limitation on the device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0064] In addition, an embodiment of the present disclosure also proposes a storage medium, on which a weld positioning program is stored. When the weld positioning program is executed by a processor, the steps of the weld positioning method described above are implemented. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of using the same method will not be elaborated either. For the technical details not disclosed in the storage medium embodiment involved in the present disclosure, please refer to the description of the method embodiment of the present disclosure. By way of example, the program instructions can be deployed to be executed on one device, or on multiple devices located at one location, or on multiple devices distributed at multiple locations and interconnected through a communication network.

[0065] Those of ordinary skill in the art can understand that all or part of the processes in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The above program can be stored in a storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the above storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0066] In related technologies, conventional weld tracking methods are difficult to identify butt welds with small gaps or even "zero gaps". To address this problem, a pure vision weld tracking system has emerged. The pure vision weld tracking system directly captures the "black line" at the "zero gap" weld through an industrial camera and determines the position of the weld through image processing. Although this method can identify "zero gap" butt welds at a higher installation height, compared with the traditional line laser method, the pure vision system is more susceptible to factors such as complex backgrounds, ambient light interference, and object surface reflections, which may lead to misidentification or failure to identify.

[0067] In view of this, the present disclosure proposes a weld positioning method, device, storage medium, and electronic device to solve the technical problems in related technologies.

[0068] Refer to Figure 2 , Figure 2 which is a schematic flowchart of a weld positioning method according to an embodiment of the present disclosure, including the following steps:

[0069] Step S11: Obtain the region information of the weld.

[0070] Step S12: Determine the binary image corresponding to the weld according to the region information.

[0071] Step S13: Determine the position information of the weld feature points of the weld according to the binary image to position the weld.

[0072] Through the above technical solution, the region information of the weld is obtained. According to the region information, the binary image corresponding to the weld is determined. According to the binary image, the position information of the weld feature points of the weld is determined to position the weld. Through this technical solution, the weld is roughly positioned by first obtaining the region information of the weld, and then the position information of the weld feature points of the weld is determined by the binary image corresponding to the weld to accurately position the weld. In this way, the problem that the pure vision recognition method is easily affected by factors such as complex backgrounds, ambient light interference, and object surface reflections, resulting in misidentification or failure to identify, is solved. The accuracy and stability of weld tracking are improved, and further guarantee is provided for the normal progress of industrial production.

[0073] In a possible way, obtaining the region information of the weld seam includes:

[0074] Obtain the image to be recognized;

[0075] Input the image to be recognized into the trained weld seam recognition model, and obtain the weld seam recognition result output by the weld seam recognition model. The weld seam recognition result is used to characterize the position information of the weld seam in the weld seam image data. Among them, the weld seam recognition model is trained based on the weld seam image data;

[0076] According to the weld seam recognition result, obtain the region information of the weld seam.

[0077] It should be understood that the weld seam recognition model is trained based on the weld seam image data. The weld seam image data is the real image of the weld seam itself. Based on the deep learning of the weld seam image and through the image processing algorithm to accurately locate the weld seam, it can solve the problem that the traditional pure vision recognition method is easily affected by factors such as complex background, environmental light interference, and object surface reflection, resulting in misrecognition or failure to recognize.

[0078] Exemplarily, Figure 3 The solution of this embodiment of the present disclosure relates to an image of the position information of a weld seam. As Figure 3 shown, in, the position information of the weld seam recognized by the weld seam recognition model can be marked with a square box. The position information of the weld seam can be represented by (x, y, h, w). Among them, x is the abscissa of the upper left vertex of the area where the weld seam is located in the image coordinate system, y is the ordinate of the upper left vertex of the area where the weld seam is located in the image coordinate system, h is the height of the area where the weld seam is located in the image coordinate system, and w is the width of the area where the weld seam is located in the image coordinate system. Of course, other methods can also be used to represent the position information of the weld seam, and this embodiment of the present disclosure does not limit this.

[0079] Therefore, after obtaining the weld seam recognition result including the position information (x, y, h, w) of the weld seam, the weld seam area corresponding to the weld seam can be determined according to the position information (x, y, h, w).

[0080] In a possible way, the training process of the weld seam recognition model includes:

[0081] Collect weld seam image data;

[0082] Preprocess the weld seam image data to obtain a weld seam data set, and the weld seam data set includes weld seam image data marked with the position of the weld seam;

[0083] Based on the YOLO object detection algorithm, train the initial recognition model according to the weld seam data set to obtain the weld seam recognition model.

[0084] Exemplarily, collecting weld seam image data can be to use an industrial camera to capture multiple weld seam images under different backgrounds and different lighting conditions, or to collect a large number of weld-related images from data sources such as public datasets, web crawlers, and other self-owned devices. Among them, the weld seam images can include individual weld seam images, workpiece images containing weld seams, and workpiece images without weld seams. Preprocessing the weld seam image data can include operations such as resizing the image, normalizing, etc. to increase the generalization ability of the model. Among them, data augmentation methods can include local mosaics, image quality changes, spatial changes, and scaling, etc. Image quality change methods can include saturation changes, grayscale changes, and exposure changes, and spatial changes can include rotations at angles such as 30°, 60°, 90°, etc.

[0085] Then, the positions of the weld seams contained in each weld seam image in the weld seam image data can be labeled manually or using a labeling tool to create a training set, a validation set, and a test set. Among them, the labeling method can be labeling through bounding boxes, key points, or pixel-level segmentation. The training set, validation set, and test set can be divided in a ratio of 8:1:1. The training set is used to provide data support for the training and learning of the weld seam recognition model, the validation set is used for fine-tuning the parameters of the weld seam recognition model, and the test set is used to evaluate the performance of the weld seam recognition model. The embodiments of the present disclosure do not make specific limitations on the selection of the above various technical methods.

[0086] Exemplarily, target detection algorithms such as the YOLO series, R-CNN series, SSD, RetinaNet, etc. can be used to train, validate, and test the initial recognition model according to the weld seam dataset to obtain a weld seam recognition model. The embodiments of the present disclosure do not limit the specific manner of training the initial recognition model.

[0087] In a possible way, determining the binary image corresponding to the weld seam according to the region information includes:

[0088] Extracting the region image of the region where the weld seam is located according to the region information;

[0089] Preprocessing the region image to obtain a preprocessed region image;

[0090] Performing threshold segmentation on the preprocessed region image to obtain a binary image.

[0091] It should be understood that performing threshold segmentation on the preprocessed region image can be to set a threshold for the preprocessed region image, thereby performing threshold segmentation to obtain a binary image. Among them, the threshold can be an adaptive threshold, and the adaptive threshold can be determined by the following method:

[0092] Taking an n×n pixel neighborhood centered on each pixel point, calculating its Gaussian mean value as the threshold for segmentation respectively, and the adaptive threshold calculation formula is:

[0093]

[0094] where \(th(u, v)\) is the threshold with \((u, v)\) as the center point, \(n\) is the neighborhood size, which can be taken as 15 in this disclosure, and \(S\) uv represents a neighborhood of size \(n\times n\) pixels with the center point at \((a, b)\), \(\sigma\) is the standard deviation of the Gaussian function, which can be 1 in this disclosure, and \(I(x, y)\) is the gray value of the point with coordinates \((x, y)\).

[0095] Exemplarily, according to the above adaptive threshold calculation formula, first, the preprocessed regional image can be divided into \(11\times11\) neighborhood blocks, and weights are assigned to each pixel point in the neighborhood. The farther the distance from the neighborhood center point, the smaller the weight, and the closer, the larger the weight. After calculating the weight matrix, then multiply the gray value of each pixel point by its respective weight, and then add up the multiplied gray values of each, and determine the added value as the adaptive threshold.

[0096] Then, threshold segmentation can be to compare the gray value of each pixel point with the adaptive threshold. Pixel points with gray values greater than or equal to the adaptive threshold are set to 0, and pixel points with gray values less than the adaptive threshold are set to 255. In this way, a binary image of the weld as shown in Figure 4 can be obtained.

[0097] In a possible way, preprocessing the regional image to obtain the preprocessed regional image includes:

[0098] Performing grayscale processing on the regional image to obtain the grayscale image corresponding to the regional image;

[0099] Based on the Hough transform, performing filtering processing on the grayscale image to obtain the preprocessed regional image.

[0100] Exemplarily, performing grayscale processing on the regional image can take the maximum value among the RGB components in the color regional image as the gray value to convert the color regional image into a grayscale image and remove redundant information in the regional image.

[0101] Exemplarily, since both the weld and the workpiece edge to be extracted are straight lines, therefore, according to this feature, through the Hough transform method, points not on the straight line can be removed, and the pixel area covered by the straight line fitting can be retained to achieve the purpose of reducing noise.

[0102] Specifically, a polar coordinate system can be established in the original space, and the straight line in the polar coordinate system can be expressed as:

[0103] \(r = x\cos\theta + y\sin\theta\)

[0104] Among them, r is the distance between the straight line and the origin of coordinates, and the parameter θ is the angle between the perpendicular line of the straight line and the x-axis. In the Hough coordinate system established in the Hough space, the horizontal axis of the coordinate is r, and the vertical axis of the coordinate is θ. That is, the point (r, θ) in the Hough space corresponds to the straight line r = xcosθ + ysinθ in the polar coordinate space. Therefore, the straight line in the polar coordinate system can be evaluated by the number of lines intersecting at a point in the Hough coordinate system. That is to say, in the Hough coordinate system, the more lines passing through a point, the more points the straight line mapped in the polar coordinate system is composed of. Therefore, to find the straight line passing through the most points in the original space, only need to find the point where as many lines as possible converge in the Hough space. In this way, based on the Hough transform, the grayscale image can be filtered to obtain the preprocessed region image.

[0105] In a possible way, according to the binary image, determine the position information of the weld feature points of the weld, including:

[0106] Perform linear fitting on the binary image to obtain the center line equation and contour line corresponding to the weld;

[0107] Calculate the intersection points according to the center line equation and the contour line to obtain the characteristic point coordinates of the weld feature points. The characteristic point coordinates are used to represent the position of the weld in the image coordinate system;

[0108] Determine the position information of the weld feature points according to the characteristic point coordinates.

[0109] Exemplarily, the straight line information obtained by linear fitting of the weld image can be expressed as

[0110] , where is the coordinate of the first endpoint of the first straight line, is the coordinate of the second endpoint of the first straight line, is the coordinate of the first endpoint of the nth straight line, is the coordinate of the second endpoint of the nth straight line.

[0111] To determine the center line equation and the contour line corresponding to the weld, first, the slope and intercept of the straight line information in the binary image can be calculated according to the following calculation formulas:

[0112]

[0113] Among them, k i is the slope of the ith straight line, and b i is the intercept of the ith straight line.

[0114] Then, slope thresholds and intercept thresholds can be set to extract the groups of lines whose slope differences and intercept differences in the line information are respectively less than their respective thresholds. If the number of line groups is 3, it indicates that the extraction is valid. If the number of line groups is not equal to 3, it indicates that the extraction is invalid, and the image needs to be reshot. When the extraction is valid, the median value of the slopes of each line group can be selected as the center line for determination. The center line obtained from the two line groups with similar slopes is the contour center line, and the center line obtained from the other line group is the weld center line.

[0115] Exemplarily, performing the intersection point calculation can be to calculate the intersection point coordinates of two contour center lines and the weld center line respectively, that is, the characteristic point coordinates of the weld feature points in the image coordinate system can be obtained. Figure 5 The solution of the embodiment of the present disclosure relates to an image after intersection point calculation. As Figure 5 shown, after performing the intersection point calculation on the binary image, the intersection points can be marked on the original image.

[0116] In a possible manner, after calculating the intersection points of the weld feature points according to the center line equation and the contour line to obtain the characteristic point coordinates, it further includes:

[0117] Determine the first coordinates corresponding to the start point and the end point of the weld according to the characteristic point coordinates;

[0118] Obtain the second coordinates corresponding to the start point and the end point of the weld, where the second coordinates are obtained by translating the workpiece corresponding to the weld according to a preset translation method;

[0119] Determine the relationship between the position deviation of the workpiece and the position coordinates obtained in the image coordinate system according to the first coordinates and the second coordinates;

[0120] Determine the deviation value of the weld feature points in the world coordinate system according to the relationship, so as to determine the position information of the weld feature points according to the deviation value.

[0121] It should be understood that the world coordinate system is usually the absolute coordinate system of the system, which can provide a unified and global reference framework. Converting the position in the image coordinate system into the deviation value in the world coordinate system can more intuitively display the position, direction and size of the object in the real world, so as to be more conveniently applied to various actual scenarios. In addition, since the image coordinate system is in pixels and the world coordinate system is in actual physical units (such as millimeters, meters, etc.). Therefore, through coordinate conversion, the pixel-level accuracy can be improved to the physical-level accuracy, thereby improving the measurement and positioning accuracy of the entire system.

[0122] On the other hand, in many applications, it is usually necessary to fuse data from different sensors to obtain more comprehensive information. By converting the position in the image coordinate system into the deviation value in the world coordinate system, it is more convenient to fuse the image data with other sensor data, thereby improving the performance and reliability of the entire system.

[0123] Therefore, after calculating the intersection points based on the center line equation and the contour line to obtain the characteristic point coordinates of the weld feature points, the position of the weld feature points in the image coordinate system can be converted into the deviation value in the world coordinate system, so as to determine the position information of the weld feature points according to the deviation value.

[0124] Exemplarily, the workpiece corresponding to the weld can be placed at the center of the field of view of the industrial camera. According to the characteristic point coordinates, the first coordinates (u0, v0) of the starting point and the ending point of the weld in the image coordinate system are determined. Then, the workpiece is translated according to a preset translation method. For example, the workpiece can be translated 1 mm horizontally and vertically respectively, and this is repeated 5 times. And the second coordinates corresponding to the starting point and the ending point of the weld are recorded, that is, for each translation, the coordinate values (u0, v m ) measured by translating the pixel value m times horizontally in the image coordinate system, and the coordinate values (u n , v0) measured by translating n times vertically in the image coordinate system.

[0125] The relational expression between the workpiece position deviation and the coordinates measured in the image coordinate system is:

[0126]

[0127] where, e u and e v are respectively the horizontal and vertical deviation values of the weld feature points in the world coordinate system; (u, v) are the coordinates of the weld feature points in the image coordinate system.

[0128] In this way, the deviation values e u , e v of the weld feature points in the world coordinate system can be determined, so as to determine the position information of the weld feature points according to the deviation values.

[0129] In a possible way, preprocess the weld image data, including:

[0130] Obtain the distortion model, and the distortion model includes radial distortion and tangential distortion;

[0131] Through the distortion model, correct the distortion of the weld image data to obtain the distortion-corrected weld image data.

[0132] It should be understood that radial distortion is the error in the radial position of the image coordinates, which is caused by the lens shape defect. Distortion correction of the acquired weld image data can improve the detection accuracy.

[0133] Among them, the distortion includes radial distortion and tangential distortion. The mathematical model for radial distortion can be described by the first few terms of the Taylor series expansion around the center point. Usually, the first three terms, namely k1, k2, and k3 terms, are used for description. The adjustment formula for radial distortion is:

[0134]

[0135] where x 0r , y 0r are the pixel coordinates after radial distortion; x, y are the ideal pixel coordinates; k1, k2, k3 are the radial distortion parameters; r is the distance from the correction point to the center point in the image coordinate system.

[0136] Tangential distortion is generally caused by the installation position error between the lens and the photosensitive element, resulting in the non - parallelism between the lens and the imaging plane. The adjustment formula for tangential distortion is:

[0137]

[0138] where x 0t , y 0t are the pixel coordinates after tangential distortion; x, y are the ideal pixel coordinates; p1, p2 are the tangential distortion parameters; r is the distance from the correction point to the center point in the image coordinate system.

[0139] According to the distortion model including radial distortion and tangential distortion, the relationship between the distorted image and the ideal image can be obtained. Therefore, the ideal image can be solved through the actually captured distorted image to complete the distortion correction. The adjustment formula for the distortion model including radial distortion and tangential distortion is:

[0140]

[0141] In this way, the weld image data can be distortion - corrected according to the adjustment formula to obtain the weld image data after distortion correction.

[0142] Referring to Figure 6 , Figure 6 which is the structural block diagram of a weld positioning device according to an embodiment of the present disclosure. Based on the same inventive concept as the foregoing embodiment, the device includes:

[0143] An acquisition module 10, configured to acquire the region information of the weld;

[0144] A determination module 20, configured to determine the binary image corresponding to the weld according to the region information;

[0145] A positioning module 30, configured to determine the position information of the weld feature points of the weld seam according to the binary image, so as to position the weld seam.

[0146] Optionally, the determining module 20 is configured to:

[0147] Extract the region image of the region where the weld seam is located according to the region information;

[0148] Perform preprocessing on the region image to obtain a preprocessed region image;

[0149] Perform threshold segmentation on the preprocessed region image to obtain the binary image.

[0150] Optionally, the determining module 20 is configured to:

[0151] Perform graying processing on the region image to obtain a gray image corresponding to the region image;

[0152] Based on the Hough transform, perform filtering processing on the gray image to obtain the preprocessed region image.

[0153] Optionally, the positioning module 30 is configured to:

[0154] Perform line fitting on the binary image to obtain the center line equation and the contour line corresponding to the weld seam;

[0155] Perform intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld feature points, and the feature point coordinates are used to characterize the position of the weld seam in the image coordinate system;

[0156] Determine the position information of the weld feature points according to the feature point coordinates.

[0157] Optionally, the positioning module 30 is further configured to:

[0158] After performing intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld feature points, determine the first coordinates corresponding to the start point and the end point of the weld seam according to the feature point coordinates;

[0159] Obtain the second coordinates corresponding to the start point and the end point of the weld seam, where the second coordinates are obtained by translating the workpiece corresponding to the weld seam according to a preset translation method;

[0160] Determine the relationship between the position deviation of the workpiece and the position coordinates obtained in the image coordinate system according to the first coordinates and the second coordinates;

[0161] According to the relationship, determine the deviation value of the weld feature point in the world coordinate system, so as to determine the position information of the weld feature point according to the deviation value.

[0162] Optionally, the obtaining module 10 is configured to:

[0163] Obtain the image to be recognized;

[0164] Input the image to be recognized into the trained weld recognition model, and obtain the weld recognition result output by the weld recognition model, where the weld recognition result is used to characterize the position information of the weld in the weld image data; wherein, the weld recognition model is trained based on the weld image data;

[0165] Obtain the weld area according to the weld recognition result.

[0166] Optionally, the training process of the weld recognition model includes:

[0167] Collect weld image data;

[0168] Preprocess the weld image data to obtain a weld data set, where the weld data set includes weld image data marked with the position of the weld;

[0169] Based on the YOLO object detection algorithm, train the initial recognition model according to the weld data set to obtain the weld recognition model.

[0170] Optionally, the preprocessing of the weld image data includes:

[0171] Obtain a distortion model, where the distortion model includes radial distortion and tangential distortion;

[0172] Perform distortion correction on the weld image data through the distortion model to obtain the weld image data after distortion correction.

[0173] It should be noted that since the steps executed by the device in this embodiment are the same as those in the foregoing method embodiment, the specific implementation manner and the achievable technical effects can refer to the foregoing embodiment, and will not be elaborated here.

[0174] In addition, in one embodiment, the embodiment of the present disclosure further provides an electronic device, where the device includes a processor, a memory, and a computer program stored in the memory, and the computer program implements the steps of the method in the foregoing embodiment when being run by the processor.

[0175] In addition, in one embodiment, an embodiment of the present disclosure further provides a computer storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method in the foregoing embodiment are implemented.

[0176] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the foregoing memories. The computer may be various computing devices including intelligent terminals and servers.

[0177] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0178] As an example, the executable instructions may or may not correspond to a file in the file system, and may be stored as part of a file that stores other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program being discussed, or stored in multiple cooperating files (such as files that store one or more modules, subroutines, or code portions).

[0179] As an example, the executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0180] It should be noted that in this document, the term "including", "may include", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article, or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or system including that element.

[0181] The serial numbers of the foregoing embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0182] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory / random access memory, magnetic disk, optical disk), and includes several instructions for causing a multimedia terminal device (which can be a mobile phone, computer, television receiver, or network device, etc.) to execute the methods described in various embodiments of the present disclosure.

[0183] The above are only optional embodiments of the present disclosure, and do not limit the patent scope of the present disclosure accordingly. Any equivalent structural transformation made by using the content of the specification and drawings of the present disclosure under the inventive concept of the present disclosure, or any direct / indirect application in other related technical fields is included in the patent protection scope of the present disclosure.

Claims

1. A weld seam positioning method, characterized in that Including: Obtain the regional information of the weld seam; Determine the binary image corresponding to the weld seam according to the regional information; Determine the position information of the weld feature points of the weld seam according to the binary image, so as to locate the weld seam.

2. The method according to claim 1, characterized in that, The determining the binary image corresponding to the weld seam according to the regional information includes: Extract the regional image of the area where the weld seam is located according to the regional information; Perform preprocessing on the regional image to obtain a preprocessed regional image; Perform threshold segmentation on the preprocessed regional image to obtain the binary image.

3. The method according to claim 2, wherein The performing preprocessing on the regional image to obtain a preprocessed regional image includes: Perform grayscale processing on the regional image to obtain the grayscale image corresponding to the regional image; Based on the Hough transform, perform filtering processing on the grayscale image to obtain the preprocessed regional image.

4. The method according to claim 1, characterized in that The determining the position information of the weld feature points of the weld seam according to the binary image includes: Perform linear fitting on the binary image to obtain the center line equation and the contour line corresponding to the weld seam; Perform intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld feature points, and the feature point coordinates are used to represent the position of the weld seam in the image coordinate system; Determine the position information of the weld feature points according to the feature point coordinates.

5. The method according to claim 4, characterized in that, After performing intersection point calculation according to the center line equation and the contour line to obtain the feature point coordinates of the weld feature points, it further includes: Determine the first coordinates corresponding to the starting point and the ending point of the weld seam according to the feature point coordinates; Obtain the second coordinates corresponding to the starting point and the ending point of the weld seam, and the second coordinates are obtained by translating the workpiece corresponding to the weld seam according to a preset translation method; Determine the relationship formula between the position deviation of the workpiece and the position coordinates obtained in the image coordinate system according to the first coordinates and the second coordinates; Determine the deviation value of the weld feature points in the world coordinate system according to the relationship formula, so as to determine the position information of the weld feature points according to the deviation value.

6. The method according to claim 1, characterized in that, The obtaining the regional information of the weld seam includes: Obtain the image to be recognized; Input the image to be recognized into the trained weld seam recognition model, and obtain the weld seam recognition result output by the weld seam recognition model, and the weld seam recognition result is used to represent the position information of the weld seam in the weld seam image data; wherein, the weld seam recognition model is trained based on the weld seam image data; Obtain the weld seam area according to the weld seam recognition result.

7. The method according to claim 6, wherein The training process of the weld seam recognition model includes: Collect weld seam image data; Perform preprocessing on the weld seam image data to obtain a weld seam data set, and the weld seam data set includes weld seam image data marked with the position of the weld seam; Based on the YOLO object detection algorithm, perform model training on the initial recognition model according to the weld seam data set to obtain the weld seam recognition model.

8. The method according to claim 7, characterized in that, The performing preprocessing on the weld seam image data includes: Obtain a distortion model, and the distortion model includes radial distortion and tangential distortion; Through the distortion model, the weld image data is corrected for distortion to obtain the distortion-corrected weld image data.

9. A weld positioning device, characterized in that, It includes: An acquisition module for acquiring the regional information of the weld; A determination module for determining the binary image corresponding to the weld according to the regional information; A positioning module for determining the position information of the weld feature points of the weld according to the binary image to position the weld.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and the processor executes the computer program to implement the method according to any one of claims 1-8.

11. An electronic device, characterized in that, The electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1-8.

Images