A friction stir welding control method, system and machine tool based on visual sensing

Abstract

The present application provides a friction stir welding control method, system and machine tool based on visual sensing. The control method includes: collecting a first image containing a workpiece to be welded on a welding platform through a first camera; obtaining a first relative coordinate of a second marking point relative to a first marking point based on the first image, wherein the second marking point is arranged on the workpiece to be welded and the first marking point is arranged on the welding platform; controlling a second camera to move to a position where the workpiece to be welded can be completely photographed based on the first relative coordinate; collecting a second image through the second camera and obtaining weld seam information of the workpiece to be welded based on the second image; planning a virtual welding path relative to the second marking point based on the weld seam information; planning an actual welding path based on the virtual welding path and the first relative coordinate; and controlling a stirring pin to perform friction stir welding on the workpiece to be welded along the actual welding path. The present application particularly solves the technical problem of low efficiency when continuously producing different types of workpieces to be welded.

Description

Technical Field

[0001] The present application relates to the technical field of friction stir welding, and particularly relates to a friction stir welding control method, system and machine tool based on visual sensing. Background Art

[0002] Friction Stir Welding (FSW) is a solid-phase joining technology. Compared with conventional welding methods, friction stir welding utilizes a high-speed rotating stirring head inserted into the weld seam of the workpiece to be welded, generating heat by friction with the workpiece to be welded, thereby raising the temperature of the material of the workpiece to be welded at the weld seam to soften it and achieve the purpose of welding. Due to the advantages of low cost, small welding deformation, high quality and high welding efficiency, friction stir welding is widely applied in the 3C field.

[0003] In the prior art, it is necessary to extract the weld seam features of the workpiece to be processed in advance for friction stir welding, then manually set the welding path, and then import the three-dimensional model of the workpiece to be welded and the set welding path into the friction stir welding machine tool, resulting in low processing efficiency. Moreover, there are a variety of workpiece models to be welded by a welding machine tool. For different models of workpieces to be welded, it is necessary to manually set the welding path and import the model, resulting in low processing efficiency; and before welding, it is also necessary to place different workpieces to be welded in the corresponding established positions, resulting in extremely low processing efficiency. Summary of the Invention

[0004] The present application provides a friction stir welding control method, system and machine tool based on visual sensing, aiming to solve the technical problem of low efficiency of friction stir welding in the prior art, especially to solve the technical problem of low efficiency in continuous production of different types of workpieces to be welded.

[0005] On the one hand, the present application provides a friction stir welding control method based on visual sensing for controlling the movement of a friction stir welding head, and a stirring pin is provided on the friction stir welding head; the control method includes: collecting a first image by a first camera;

[0006] Based on the first image, obtaining a first relative coordinate of a second marking point relative to a first marking point, and obtaining size information of the workpiece to be welded; wherein, the second marking point is arranged on the workpiece to be welded, and the first marking point is arranged on the welding platform;

[0007] Based on the first relative coordinate and the size information, controlling the second camera to move to a position where the workpiece to be welded can be completely photographed;

[0008] Controlling the second camera to collect a second image, and obtaining weld seam information of the workpiece to be welded based on the second image;

[0009] Plan a virtual welding path relative to the second marking point based on the weld information;

[0010] Plan an actual welding path based on the virtual welding path and the first relative coordinates;

[0011] Control the stirring pin to extend into the weld of the workpiece to be welded, and perform friction stir welding on the workpiece to be welded along the actual welding path.

[0012] Optionally, determine at least three second marking points according to the shape of the weld of the workpiece to be welded. At least three of the second marking points are arranged around the weld and define at least one virtual triangle. The step of obtaining the first relative coordinates of the second marking point relative to the first marking point includes: based on the first image, obtain three first relative coordinates of at least three of the three marking points relative to the first marking point, and determine the geometric center coordinates of one of the at least one virtual triangle relative to the first marking point based on the three first relative coordinates. The step of controlling the second camera to move to a position where the workpiece to be welded can be completely photographed includes: controlling the second camera to move above the geometric center coordinates, and performing zooming according to the dimension information to collect the second image, and the collected second image completely includes at least two of the weld and the second marking points.

[0013] Optionally, the step of planning a virtual welding path relative to the second marking point based on the weld information includes: randomly selecting one of at least three of the second marking points as the main reference point; randomly selecting one of at least three of the second marking points as the secondary reference point, and the main reference point and the secondary reference point are respectively located on both sides of the weld; identify the weld through image recognition technology to obtain the weld track; associate the weld track with the main reference point to obtain the first virtual welding path; associate the weld track with the secondary reference point to obtain the second virtual welding path; based on the relative position of the secondary reference point and the main reference point, transform the second virtual welding path into the third virtual welding path; judge the similarity between the third virtual welding path and the first virtual welding path. If the similarity is greater than the preset parameter, use the first virtual welding path as the virtual welding path; if the similarity is less than the preset parameter, at least adjust one of the resolution, brightness, and / or focal length of the second camera and / or control the second camera to move closer to or away from the welding workpiece in the direction perpendicular to the welding platform to be able to re-obtain the second image, and repeat the above steps until the similarity is greater than or equal to the preset parameter, and use the finally determined first virtual welding path as the virtual welding path.

[0014] Optionally, the step of identifying the weld seam through the image technology includes image preprocessing, weld seam identification, and weld seam segmentation.

[0015] Optionally, the step of planning the actual welding path based on the virtual welding path and the first relative coordinates includes: based on the first relative coordinates of the main reference point and the first marking point, transforming the first virtual welding path that meets the conditions into an actual welding path with reference to the first marking point.

[0016] Optionally, the step of obtaining the first relative coordinates of the second marking point relative to the first marking point and the size information of the workpiece to be welded based on the first image specifically includes: determining whether the workpiece to be welded is on the welding platform based on the first image and a preset image; if so, preprocessing the first image to improve the resolution of the first image; based on the first image with improved resolution, identifying the workpiece to be welded and the second marking point to obtain the first relative coordinates of the second marking point relative to the first marking point and the size information of the workpiece to be welded.

[0017] Optionally, the step of controlling the stirring pin to extend into the weld seam of the workpiece to be welded and performing friction stir welding on the workpiece to be welded along the actual welding path includes: identifying the label of the workpiece to be welded based on the size information; obtaining the rotation speed and moving speed of the stirring pin based on the label; randomly selecting one end of the actual welding path as the starting point, controlling the stirring pin to move directly above the starting point according to the relative position between the starting point and the first marking point, controlling the stirring pin to extend into the weld seam of the workpiece to be welded, and driving the stirring pin to perform friction stir welding on the workpiece to be welded along the actual welding path at the rotation speed and the moving speed.

[0018] This application also proposes a machine tool, including:

[0019] A frame;

[0020] A welding platform, which is connected to the frame and is used to place the workpiece to be welded; a first marking point is provided on the welding platform;

[0021] A first camera, which is fixed to the frame and is used to collect a first image containing the workpiece to be welded on the welding platform; wherein, a second marking point is provided on the workpiece to be welded;

[0022] A second camera, which is movably arranged relative to the frame;

[0023] A friction stir welding head, the friction stir welding head being movably arranged on the frame, and a stirring pin being provided on the friction stir welding head; and

[0024] A processor, the processor being electrically connected to the first camera, the second camera and the friction stir welding head respectively; the processor is configured to: adopt the friction stir welding control method based on visual sensing as described above.

[0025] Optionally, a rotatable shoulder is provided on the friction stir welding head, and the stirring pin is connected to the shoulder; when the stirring pin extends to the weld of the workpiece to be welded, the shoulder abuts against the surface of the workpiece to be welded; wherein, a groove is provided on the shoulder, and an opening of the groove is opened on the surface; the groove extends along the radial direction of the shoulder from the connection part of the shoulder and the stirring pin to the outer peripheral wall of the shoulder, and the depth of the groove gradually increases in its extending direction.

[0026] The present application also provides a friction stir welding control system based on visual sensing, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and are configured to be the friction stir welding control method based on visual sensing as described above.

[0027] In the technical solution of the embodiment of the present application, through the first image captured by the first camera, the first relative coordinate of the second marking point relative to the first marking point is obtained. Based on the first relative coordinate, the second camera is controlled to move to a position where the workpiece to be welded can be completely photographed, so as to be able to collect the complete weld information of the workpiece to be welded. Based on the weld information, a virtual welding path relative to the second marking point is planned. Based on the first marking point and the virtual welding path, an actual welding path is obtained, so as to control the stirring pin to extend into the weld of the workpiece to be welded and perform friction stir welding on the workpiece to be welded along the actual welding path. In the technical solution of the embodiment of the present application, regardless of the shape, size and placement position of the workpiece to be welded, the weld information can be accurately extracted, and an actual welding path relative to the welding platform (the first marking point) is established through the recognition of the weld information, and then the stirring pin is controlled to perform friction stir welding, without the operator deliberately placing the workpiece to be welded at a preset position, nor modeling the welding gap, which improves the welding efficiency and is conducive to continuous production. Description of the Drawings

[0028] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0029] Figure 1 It is a schematic flowchart of an embodiment of the friction stir welding control method based on visual sensing provided by the embodiments of the present application;

[0030] Figure 2 It is a schematic flowchart of another embodiment of the friction stir welding control method based on visual sensing provided by the embodiments of the present application;

[0031] Figure 3 It is a schematic flowchart of a specific embodiment of step S500 of the friction stir welding control method based on visual sensing provided in the embodiments of the present application;

[0032] Figure 4 It is a schematic flowchart of step S600 of the friction stir welding control method based on visual sensing in the embodiments of the present application;

[0033] Figure 5 It is a schematic flowchart of step S200 of the friction stir welding control method based on visual sensing in the embodiments of the present application;

[0034] Figure 6 It is a schematic flowchart of step S700 of the friction stir welding control method based on visual sensing in the embodiments of the present application;

[0035] Figure 7 It is a schematic plan view of placing a workpiece to be welded on the welding platform in the embodiments of the present application;

[0036] Figure 8 It is another schematic plan view of placing a workpiece to be welded on the welding platform in the embodiments of the present application;

[0037] Figure 9 It is a schematic plan view of placing another workpiece to be welded on the welding platform in the embodiments of the present application;

[0038] Figure 10 It is a schematic structural diagram of the machine tool proposed in the embodiments of the present application;

[0039] Figure 11 It is a schematic diagram of the shoulder and the stirring pin proposed in the embodiments of the present application;

[0040] Figure 12 It is another schematic diagram of the shoulder and the stirring pin proposed in the embodiments of the present application. Detailed implementation manners

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present invention.

[0042] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0043] In the present application, the term "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described as "exemplary" in the present application is not necessarily to be construed as more preferred or more advantageous than other embodiments. The following description is given to enable any person skilled in the art to make and use the present invention. In the following description, details are set forth for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be practiced without these specific details. In other instances, well-known structures and processes are not described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the present application.

[0044] The embodiments of the present application provide a friction stir welding control method, system and machine tool based on visual sensing, which will be described in detail below respectively.

[0045] The visual sensing friction stir welding control method proposed in the embodiments of the present application is used to control the movement of the friction stir welding head of a friction stir welding machine tool. A stirring pin is provided on the friction stir welding head. Specifically, the control method can insert the stirring pin into the weld seam of the workpiece to be welded and perform friction stir welding on the workpiece to be welded along the actual welding path, so that friction is generated between the stirring pin and the workpiece to be welded, causing the material around the weld seam of the workpiece to be welded to soften and fill into the weld seam gap to form the workpiece to be welded into one body.

[0046] In the prior art, to identify the weld seam by visual sensing technology and weld the workpiece to be welded into one body, the commonly used method is as follows: extract the weld seam features of the workpiece to be welded in advance, then manually set the welding path, and then import the three-dimensional model of the workpiece to be welded and the set welding path into the friction stir welding machine tool; in this method, the workpiece to be welded needs to be placed at a preset position to align with the pre-set coordinate system. However, due to the small weld seam gap of friction stir welding, the deviation of the placement position of the workpiece to be welded needs to be strictly controlled. However, the types and sizes of the workpieces to be welded targeted by the friction stir welding machine tool are different. Therefore, when placing workpieces to be welded of different types and sizes, a large amount of time is required to position the workpiece to be welded, resulting in low processing efficiency and being not conducive to continuous processing.

[0047] For this reason, as Figure 1 shown, it is a schematic flowchart of an embodiment of the visual sensing-based friction stir welding control method in the embodiments of the present application. The control method includes:

[0048] S100, acquiring a first image by a first camera 40;

[0049] The first camera 40 is fixed on the machine tool. Generally speaking, the first camera 40 is a wide-angle camera, and its viewing angle can at least cover the welding platform 10 to be able to acquire the first image of the workpiece to be welded 20 placed on the welding platform 10.

[0050] S200, based on the first image, obtaining the first relative coordinate of a second marking point P2 relative to a first marking point P1. Wherein, the second marking point P2 is arranged on the workpiece to be welded 20, and the first marking point P1 is arranged on the welding platform 10.

[0051] Identify the first image and identify the first marking point P1 and the second marking point P2. For the same machine tool, the welding platform 10 is fixed, so the first marking point P1 is also fixed, and thus the coordinates of the first marking point P1 can be pre-stored in the system. According to the relative positions of the first marking point P1 and the second marking point P2 and the coordinate values of the first marking point P1, obtain the first relative coordinates of the second marking point P2 relative to the first marking point P1. For example, if the system calibrates the coordinates of the first marking point P1 as (x1, y1, z1) and the coordinates of the second marking point P2 as (x2, y2, z2), then the first relative coordinates of the second marking point P2 relative to the first marking point P1 are (x2 - x1, y2 - y1, z2 - z1).

[0052] S300, based on the first relative coordinates, control the second camera 50 to move to a position where the workpiece to be welded 20 can be completely photographed.

[0053] In order to effectively improve the processing efficiency, the workpiece to be welded 20 is placed randomly on the welding platform 10 without being placed at a preset position. For this reason, in the control method of the present application, the second camera 50 is movably arranged on the machine tool to be able to clearly photograph the workpiece to be welded 20, so as to be able to extract the weld information of the workpiece to be welded for planning the welding path. The initial position of the second camera 50 is preset, so its initial position is determined relative to the first marking point P1. For this reason, based on the first relative coordinates, control the second camera 50 to move to a position where the workpiece to be welded 20 can be completely photographed. The second camera 50 can be driven by a motion mechanism, such as a hydraulic mechanism, an electric mechanism or a rack and pinion mechanism, etc.

[0054] Moreover, the weld gap of the workpiece to be welded 20 is small and the recognition difficulty is large. The accuracy of the weld information obtained from the first image collected by the first camera 40 is not enough. Therefore, in the present application, the second camera 50 that can move is used to collect a second image with higher resolution for extracting the weld information. That is: the first image is a global image, and the second image is a local image.

[0055] S400, control the second camera 50 to collect a second image, and based on the second image, obtain the weld information of the workpiece to be welded 20;

[0056] When the second camera 50 moves to a position where the workpiece to be welded 20 can be photographed, collect a second image through the second camera 50. By recognizing the second image, obtain the weld information of the workpiece to be welded 20. The weld information includes the width of the weld gap and the direction of the weld gap.

[0057] S500, based on the weld information, plan a virtual welding path relative to the second marking point P2;

[0058] Taking the second marking point P2 as the origin, a second coordinate system is established. Under the second coordinate system, a trajectory equation of the weld gap is established based on the width and orientation of the weld gap. This trajectory equation is the virtual welding path. S600, based on the virtual welding path and the first relative coordinates, plan the actual welding path;

[0059] Taking the first marking point P1 as the origin, under the first coordinate system, based on the first relative coordinates, convert the trajectory equation under the second coordinate system to the first coordinate system, and then construct another trajectory equation, which is the actual welding path.

[0060] S700, control the stirring pin 80 to extend into the weld of the workpiece to be welded 20, and perform friction stir welding on the workpiece to be welded 20 along the actual welding path.

[0061] In the machine tool, the initial position of the stirring pin 80 is a definite value, and its initial position relative to the first marking point P1 is also definite. Therefore, when the actual welding path is determined, control the stirring pin 80 to extend into the weld gap of the workpiece to be welded 20, and control the stirring pin 80 to move along the actual welding path to perform friction stir welding on the workpiece to be welded 20 until the weld gap of this strip is welded.

[0062] In the technical solution of the embodiment of the present application, through the first image captured by the first camera 40, obtain the first relative coordinates of the second marking point P2 relative to the first marking point P1. Based on the first relative coordinates, control the second camera 50 to move to a position where the workpiece to be welded 20 can be completely photographed, so as to be able to collect the complete weld information of the workpiece to be welded 20. Based on the weld information, plan the virtual welding path relative to the second marking point P2. Based on the first marking point P1 and the virtual welding path, obtain the actual welding path, so as to control the stirring pin 80 to extend into the weld of the workpiece to be welded 20, and perform friction stir welding on the workpiece to be welded 20 along the actual welding path. In the technical solution of the embodiment of the present application, regardless of the shape, size and placement position of the workpiece to be welded 20, the weld information can be accurately extracted, and the actual welding path relative to the welding platform 10 is established through the recognition of the weld information, and then the stirring pin 80 is controlled to perform welding. There is no need for the operator to deliberately place the workpiece to be welded 20 at a preset position, nor is it necessary to model the welding gap, which improves the welding efficiency and is conducive to continuous production.

[0063] In the prior art, modeling the weld through image recognition has become a common method. The weld belongs to a long and narrow structure, and often has very high requirements for resolution and viewing angle. However, in the prior art, a fixed camera is used to photograph the workpiece to be welded 20 to obtain an image for planning the welding path; however, in the face of workpieces of different sizes (such as Figure 7 and9 as shown), with different shapes (such as Figure 7 and 9 as shown), different types, and different placement positions (such as Figure 7 and 8 as shown), when welding parts 20, since the shooting angle of the fixed camera is fixed, the image quality obtained finally for path planning is difficult to be consistent, resulting in a large system error and unable to guarantee the welding quality. Therefore, in order to reduce the system error and improve the welding quality, that is, in order to obtain a second image with similar quality (the accuracy of the weld information obtained from the second image is reliable), the embodiment of the present application controls the shooting position of the second camera 50 through the first image (global image) captured by the fixed camera, so that the second camera 50 moves to a reliable position to shoot the welding part 20 to obtain a second image (local image), and then the system error of the welding path obtained can be reduced when facing welding parts 20 with different sizes, shapes, types, and placement positions.

[0064] In the actual production process, the shape of the weld may be linear, curved, or a hybrid type (linear + curved), etc. Therefore, in the technical solution of the present application, at least three second marking points P2 are provided on the welding part 20 to facilitate the second camera 50 to move to directly above the weld gap or a position close to directly above the weld gap. At least three second marking points P2 are arranged around the weld, defining at least one virtual triangle. Generally speaking, the weld falls within the range of this virtual triangle.

[0065] As an optional implementation manner of the above embodiment, Figure 2 as shown, the step of obtaining the first relative coordinates of the second marking point P2 relative to the first marking point P1 includes:

[0066] S203, based on the first image, obtain the three first relative coordinates of at least three of the at least three second marking points P2 (such as P21, P22, and P23 shown in the figure) relative to the first marking point P1, and determine the coordinates of the geometric center (P20) of at least one of the virtual triangles relative to the first marking point P1 based on the three first relative coordinates.

[0067] For example, if the coordinates of the i-th second marking point P2 are (x 2i , y 2i , z 2i ), then the first relative coordinates of the i-th second marking point P2 relative to the first marking point P1 are (x 2i - x1, y 2i - y1, z 2i-z1), where i ranges from 1, 2, 3... N. The geometric center coordinates are the coordinates of the incenter or centroid of the virtual triangle in the first coordinate system.

[0068] The step of controlling the second camera 50 to move to a position where it can completely capture the workpiece to be welded 20 includes: S301, controlling the second camera 50 to move above the geometric center coordinates to collect the second image, and the collected second image completely includes the weld seam. That is: the shooting position of the second camera 50 is directly above the incenter or centroid of the virtual triangle, so as to be able to obtain a high-quality second image and facilitate obtaining weld seam information close to the real situation.

[0069] In the technical solution of the present application, the shooting position of the second camera 50 is directly above the geometric center of the virtual triangle. For this reason, in order to be able to clearly capture the weld seam gap, when setting the second marking point P2, the implementer can set the virtual triangle as an equilateral triangle, and its geometric center is located on the weld seam. Therefore, when shooting, the line connecting the focus of the second camera 50 and the geometric center can be perpendicular to the surface of the workpiece to be welded 20, and thus high-quality weld seam information can be obtained.

[0070] In the technical solution of the embodiment of the present application, in order to be able to capture the weld seam of the complete workpiece to be welded 20, the size information of the workpiece to be welded 20 is obtained through the first image; and according to the size information, the focal length of the second camera 50 is adjusted to be able to capture the complete workpiece to be welded 20. Figure 7 As shown, in some alternative embodiments of the present application, after step S100, the control method further includes step S210: obtaining the size information of the workpiece to be welded 20 based on the first image. The size information reflects the shape and size of the workpiece to be welded 20. The size information is mainly used to enable the second camera 50 to capture the complete weld seam gap, so as to model the weld seam. Step S210 can be carried out simultaneously with step S200, or after or before step S200.

[0071] Furthermore, in order to ensure that the virtual welding path of the established weld seam is close to the real value, when the second camera 50 shoots, it can shoot at least two of the second marking points P2 for mutual verification of the virtual welding path. Specifically, Figure 3 As shown, as an alternative embodiment of the above embodiment, the step of planning the virtual welding path relative to the second marking point P2 based on the weld seam information includes:

[0072] S501, randomly select one of at least three of the second marking points P2 as the main reference point; for example, the main reference point is P22.

[0073] S502, randomly select one of at least three of the second marked points P2 as the secondary reference point; for example, the secondary reference point is P21.

[0074] S503, identify the weld seam through image recognition technology to obtain the weld seam trajectory.

[0075] S504, associate the weld seam trajectory with the primary reference point to obtain the first virtual welding path; that is, establish a second coordinate system I with the primary reference point as the origin, model the weld seam trajectory in this second coordinate system I, and obtain the first virtual welding path;

[0076] S505, associate the weld seam trajectory with the secondary reference point to obtain the second virtual welding path;

[0077] That is, establish a second coordinate system II with the secondary reference point as the origin, model the weld seam trajectory in this second coordinate system II, and obtain the second virtual welding path;

[0078] S506, based on the relative positions of the secondary reference point and the primary reference point, transform the second virtual welding path into the third virtual welding path;

[0079] That is, according to the relative positions of the secondary reference point and the primary reference point, rotate the second virtual welding path to the second coordinate system II to obtain the third virtual welding path. Theoretically speaking, the third virtual welding path and the first virtual welding path are completely coincident. However, due to the influence of the resolution of the second image, there are errors in the weld seam recognition algorithm, resulting in the non - coincidence of the third virtual welding path and the first virtual welding path. Therefore, in order to minimize the error of the obtained virtual welding path, the embodiments of the present application propose to reduce the error through the mutual verification of virtual welding paths. Specifically,

[0080] S507, determine the similarity between the third virtual welding path and the first virtual welding path. The similarity can be measured by the distance between the two paths. Usually, there are many classic measurement indexes for the similarity between trajectories, such as: Closest - Pair Distance (CPD), Sum - of - Pairs Distance (SPD), DTW, LCSS, and EDR. The similarity between the third virtual welding path and the first virtual welding path can be calculated by at least one of the above measurement indexes.

[0081] S508, if the similarity is greater than the preset parameter, use the first virtual welding path as the virtual welding path; that is: if the similarity is greater than the preset parameter, the third virtual welding path and the first virtual welding path have a high similarity and pass the mutual verification with each other, and the first virtual welding path can be used as the virtual welding path.

[0082] S509. If the similarity is less than a preset parameter, at least one of the resolution, brightness, and / or focal length of the second camera 50 is adjusted and / or the second camera 50 is controlled to move closer to or away from the welding workpiece in a direction perpendicular to the welding platform 10, so as to re-acquire the second image, and the above steps are repeated until the similarity is greater than or equal to the preset parameter, and the finally determined first virtual welding path is used as the virtual welding path. That is, if the similarity between the third virtual welding path and the first virtual welding path is insufficient and the mutual verification fails, it indicates that the quality of the second image is insufficient at this time. At least one of the resolution, brightness, and / or focal length of the second camera 50 can be adjusted and / or the second camera 50 can be controlled to move closer to or away from the welding workpiece in a direction perpendicular to the welding platform 10 to re-acquire a higher-quality second image, and the above steps are repeated until the mutual verification passes, and then the newly determined first virtual welding path is used as the virtual welding path.

[0083] In order to minimize errors as much as possible, the main reference point and the secondary reference point are respectively located on both sides of the weld.

[0084] As an alternative implementation of the above embodiments, the step of identifying the weld seam through the image technology includes image preprocessing, weld seam identification, and weld seam segmentation. During image preprocessing, generally, color gamut conversion, grayscale conversion, and filtering are required for the second image. Due to various disturbances in the actual process, image edge blurring is one of the problems with a relatively high occurrence frequency. The boundary of the weld seam of the workpiece 20 to be welded is blurred, and the boundary lines are difficult to identify, making the feature extraction process extremely difficult. Therefore, it is particularly important to use image sharpening technology to enhance the blurred area of the weld seam. After image filtering, grayscale conversion, image enhancement, etc. are performed on the image, it is necessary to further filter out irrelevant interference sources, highlight the weld seam area, and then perform weld seam identification and segmentation. In the embodiments of the present application, the weld seam image processing unit separates the region of interest and the non-region of interest of the target image. Further, image segmentation and recognition separate the target object from the background, so as to obtain the position of the weld seam gap in the image coordinate system; then, according to certain feature information sets of the target and corresponding determination criteria, the pixels of the entire image are classified, and the information in the weld seam area is extracted. Generally speaking, image segmentation is performed by threshold segmentation to separate the target object from the background. Threshold segmentation classifies different pixel value ranges in the image by selecting an appropriate threshold, so as to separate the target object from the background in the image. Generally speaking, threshold segmentation is to set an appropriate threshold using prior knowledge or algorithms after completing preprocessing such as image grayscale conversion and filtering, perform algebraic operations on the pixel values of all pixel points and the threshold, and select the set of pixel points that meet the constraint conditions as the contour information of the target object, completing the separation process of the target object and the background. In the actual image segmentation process, the selection of the threshold directly affects the effect of image segmentation. Therefore, in order to ensure the effect during image segmentation, super-resolution reconstruction is introduced on the basis of threshold segmentation operation to effectively eliminate possible multi-threshold situations. Since the ratio between the weld seam area and the background in the image is unbalanced and the image features have multiple peaks, when there is more than one optimal threshold data found by the maximum inter-class variance method for image segmentation processing, a super-resolution reconstruction method is used to obtain a corrected high-resolution image, and then the optimal threshold calculation method for image segmentation is used to obtain the optimal threshold, thereby improving the effect of image segmentation. The weld seam area screening is completed through the super-resolution reconstruction and image threshold segmentation of the scene image in sequence. According to the high-precision point spread function (PSF) as prior knowledge and the accurate color vector information as the constraint condition, the convex set projection algorithm is improved and optimized to complete the super-resolution reconstruction of the scene image. After obtaining the high-resolution image after super-resolution reconstruction, the optimal threshold is obtained using the threshold calculation method, and superpixel segmentation is also introduced in the operation steps of obtaining the weld seam area through the optimal threshold. After obtaining the weld seam area, the center line of the weld seam is extracted, and in the second coordinate systems I and II, a mathematical model of the center line is established to obtain the first virtual welding path and the second virtual welding path.

[0085] In addition, the image recognition of the weld area can also adopt the existing technology. For example, the patent with the publication number CN112238292 discloses a method for tracking the spatial curve trajectory of a friction stir welding robot based on vision. This patent obtains the center coordinates of the weld from the image by means of binarization.

[0086] As an alternative implementation of the above embodiment, Figure 4 As shown, the step of planning the actual welding path based on the virtual welding path and the first relative coordinate includes: S601, based on the first relative coordinate between the main reference point and the first marking point P1, transforming the first virtual welding path that meets the conditions into an actual welding path referring to the first marking point P1. Transforming the first virtual welding path that conforms to the system error into an actual welding path referring to the first marking point P1 can make the movement trajectory of the stirring pin 80 closer to the real weld trajectory, and a deviation correction device is not required during the welding process, which is beneficial to improving the welding speed and welding quality.

[0087] Through the control method provided by the embodiments of the present application, since the operator does not need to adjust the placement position of the workpiece 20 to be welded, the continuous operation of the workpiece 20 to be welded is realized. During continuous operation, the processing process of the workpiece 20 to be welded may include transporting the workpiece 20 to be welded to the welding platform 10 by a manipulator, welding, and transporting the simple workpiece 20 to be welded to the next manufacturing tooling. For this reason, there is a situation where there is no workpiece 20 to be welded on the welding platform 10. Therefore, in order to meet the intelligent production process, as an alternative implementation of the above embodiment, Figure 5 As shown, the step of obtaining the first relative coordinate of the second marking point P2 relative to the first marking point P1 based on the first image specifically includes:

[0088] S201, determining whether there is the workpiece to be welded on the welding platform 10 based on the first image and a preset image;

[0089] S202, if so, preprocessing the first image to improve the resolution of the first image; based on the first image with improved resolution, identifying the workpiece to be welded and the second marking point P2 to obtain the first relative coordinate of the second marking point P2 relative to the first marking point P1.

[0090] That is, based on the first image, when it is determined that there is a workpiece to be welded on the welding platform 10, the first relative coordinates of the second marking point P2 and the first marking point P1 will be calculated. Since the first image is a global image, the pixel positions of the first marking point P1 and the second marking point P2 are relatively blurred in the global image. Therefore, it is necessary to increase the resolution of the first image. Based on the first image with increased resolution, the workpiece to be welded and the second marking point P2 are identified to obtain the first relative coordinates of the second marking point P2 relative to the first marking point P1, which are used to control the movement of the second camera 50.

[0091] As an alternative implementation of the above embodiment, Figure 6 As shown, the steps of controlling the stirring pin 80 to extend into the weld of the workpiece 20 to be welded and performing friction stir welding on the workpiece 20 to be welded along the actual welding path include:

[0092] S701, identifying the label of the workpiece 20 to be welded based on the dimension information; for example, when identifying the first image, the dimension information of the workpiece 20 to be welded is obtained according to the contour information of the workpiece 20 to be welded, and then the type of the workpiece 20 to be welded is obtained through the dimension information.

[0093] S702, based on the label, obtaining the rotation speed and the moving speed of the stirring pin 80; randomly selecting one end of the actual welding path as the starting point, controlling the stirring pin 80 to move directly above the starting point according to the relative position between the starting point and the first marking point P1, controlling the stirring pin 80 to extend into the weld of the workpiece 20 to be welded, and driving the stirring pin 80 to perform friction stir welding on the workpiece 20 to be welded along the actual welding path at the rotation speed and the moving speed.

[0094] Based on the above embodiment, since the welding parameters of each workpiece 20 to be welded are also different. The welding parameters are mainly designed by the material, thickness, weld type, etc. of the workpiece 20 to be welded. The welding parameters include the rotation speed and the moving speed of the stirring pin 80. Before welding, the implementer can associate the rotation speed and the moving speed with the dimension information of the workpiece 20 to be welded. Therefore, on an assembly line, the control system can implement corresponding welding parameters according to the dimension information of the workpiece 20 to be welded to match the type of the workpiece 20 to be welded and obtain welds of corresponding quality.

[0095] The present application also proposes a machine tool, Figure 10 As shown, including:

[0096] A frame 30;

[0097] A welding platform 10, the welding platform 10 is connected to the frame 30 and is used to place the workpiece 20 to be welded; a first marking point P1 is provided on the welding platform 10;

[0098] A first camera 40, which is fixed to the frame 30 and is configured to collect a first image of the workpiece 20 to be welded on the welding platform 10; wherein, a second marking point P2 is provided on the workpiece 20 to be welded.

[0099] A second camera 50, which is movably arranged relative to the frame 30.

[0100] A friction stir welding head 60, which is movably arranged on the frame 30 and is provided with a stirring pin 80; and

[0101] A processor (not shown), which is electrically connected to the first camera 40, the second camera 50 and the friction stir welding head 60 respectively; the processor is configured to: adopt a friction stir welding control method based on visual sensing. Since the control method adopts a part or all of the foregoing embodiments, the machine tool has part or all of the technical advantages of the foregoing embodiments.

[0102] It should be noted that the second camera 50 can be movably arranged on the frame 30, and its driving structure can be a rack and pinion, a worm and worm gear, a crawler, a hydraulic telescopic rod, an electric push rod, etc. The second camera 50 can also be fixedly arranged on the friction stir welding head 60, and the second camera 50 is driven to move by driving the friction stir welding head 60. The friction stir welding head 60 is provided with a stirring pin 80. The driving of the friction stir welding head 60 can be set by using the prior art. The stirring pin 80 is rotatably arranged on the friction stir welding head 60 to friction with the workpiece 20 to be welded at a high speed.

[0103] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. The processor may also include an AI (Artificial Intelligence) processor, which is used to process the operations of the control method of the friction stir welding control system based on visual sensing, so that the control method model of the friction stir welding control system based on visual sensing can be autonomously trained and learned to improve efficiency and accuracy.

[0104] As an alternative implementation of the above embodiment, Figure 11 and Figure 12As shown, a shoulder 70 capable of rotational movement is provided on the friction stir welding head 60, and the stirring pin 80 is connected to the shoulder 70; when the stirring pin 80 extends to the weld seam of the workpiece to be welded 20, the shoulder 70 abuts against the surface of the workpiece to be welded 20; wherein, a groove 70a is provided on the shoulder 70, and the opening of the groove 70a is formed on the surface; the groove 70a extends along the radial direction of the shoulder 70 from the connection between the shoulder 70 and the stirring pin 80 to the outer peripheral wall of the shoulder 70, and the depth of the groove 70a gradually increases in its extending direction. Even in the technical solution of the embodiment of the present application, a part of the systematic error is eliminated through the mutual verification of the welding paths, but the control system still relies on image recognition to model the welding path. Due to the accuracy problem still existing in the image recognition technology, when welding, the stirring pin 80 will still deviate slightly from the center of the welding gap. Therefore, in order to effectively reduce the influence of the systematic error on the welding quality, a groove 70a is provided on the shoulder 70 in the embodiment of the present application, and the opening of the groove 70a is arranged on the surface of the shoulder 70 for abutting against the workpiece to be welded 20. And the groove 70a extends along the radial direction of the shoulder 70 from the connection between the shoulder 70 and the stirring pin 80 to the outer peripheral wall of the shoulder 70, and the depth of the groove 70a gradually increases in its extending direction. Therefore, when the stirring pin 80 rubs against the workpiece to be welded 20 on the side slightly deviating from the center, the excessive molten liquid on this side will flow radially outward from the root of the stirring pin 80 under the action of the centrifugal force, and when the shoulder 70 rotates, the excessive molten liquid will be driven to the other side in the radial direction of the stirring pin 80 to fill the weld gap, facilitating the uniformity of the welds on both sides of the center, and further effectively reducing the influence of the systematic error of the control system on the weld quality.

[0105] Meanwhile, due to the setting of the groove 70a, when judging the similarity between the third virtual welding path and the first virtual welding path, the similarity can be set relatively high, that is, the third virtual welding path and the first virtual welding path are allowed to have a relatively high error. Furthermore, one of the resolution, brightness, and / or focal length of the second camera 50 can be adjusted as little as possible and / or the second camera 50 can be controlled to move closer to or away from the welding workpiece in the direction perpendicular to the welding platform 10 to improve the processing efficiency.

[0106] As Figure 12 shown, there are a plurality of grooves, and the plurality of grooves are arranged at intervals along the circumferential direction of the stirring pin.

[0107] An embodiment of the present application also provides a friction stir welding control system based on visual sensing, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the friction stir welding control method based on visual sensing as described above.

[0108] Generally, the friction stir welding control system based on visual sensing includes: at least one processor, at least one memory, and a control program of the friction stir welding control system stored on the memory and executable on the processor. The control program of the friction stir welding control system is configured to implement the steps of the control method as described above.

[0109] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen. The processor may also include an AI (Artificial Intelligence) processor, which is used to process the operations of the control method of the friction stir welding control system based on visual sensing, so that the control method model of the friction stir welding control system based on visual sensing can be autonomously trained and learned to improve efficiency and accuracy.

[0110] The memory may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one instruction, and the at least one instruction is used to be executed by the processor to implement the friction stir welding control method provided in the method embodiments of the present application.

[0111] The above has introduced in detail a control system and a machine tool for friction stir welding based on visual sensing provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A friction stir welding control method based on visual sensing, used to control the movement of the friction stir welding head. A stirring pin is provided on the friction stir welding head, and it is characterized in that, Including: Collecting a first image by a first camera; Based on the first image, obtaining a first relative coordinate of a second marking point relative to a first marking point; wherein, the second marking point is arranged on a workpiece to be welded, and the first marking point is arranged on a welding platform; Based on the first relative coordinate, controlling a second camera to move to a position where the workpiece to be welded can be completely photographed; Controlling the second camera to collect a second image, and obtaining weld information of the workpiece to be welded based on the second image; Based on the weld information, planning a virtual welding path relative to the second marking point; Based on the virtual welding path and the first relative coordinate, planning an actual welding path; Controlling a stirring pin to extend into a weld of the workpiece to be welded, and performing friction stir welding on the workpiece to be welded along the actual welding path; Wherein, at least three second marking points are determined according to the shape of the weld of the workpiece to be welded, at least three of the second marking points are arranged around the weld, and at least one virtual triangle is defined; The step of obtaining the first relative coordinate of the second marking point relative to the first marking point includes: Based on the first image, obtaining three first relative coordinates of three of at least the three marking points relative to the first marking point, and determining geometric center coordinates of one of at least one of the virtual triangles relative to the first marking point based on the three first relative coordinates; The step of controlling the second camera to move to a position where the workpiece to be welded can be completely photographed includes: Controlling the second camera to move above the geometric center coordinates to collect the second image, and the collected second image completely includes at least two of the weld and the second marking point; wherein, the step of planning a virtual welding path relative to the second marking point based on the weld information includes: Randomly selecting one of at least three of the second marking points as a main reference point; Randomly selecting one of at least three of the second marking points as a secondary reference point, and the main reference point and the secondary reference point are respectively located on two sides of the weld; Identifying the weld by image recognition technology to obtain a weld track; Associating the weld track with the main reference point to obtain a first virtual welding path; Associating the weld track with the secondary reference point to obtain a second virtual welding path; Based on the relative positions of the secondary reference point and the main reference point, transforming the second virtual welding path into a third virtual welding path; Judging the similarity between the third virtual welding path and the first virtual welding path, and if the similarity is greater than a preset parameter, using the first virtual welding path as the virtual welding path; If the similarity is less than a preset parameter, at least one of the resolution, brightness, and / or focal length of the second camera is adjusted and / or the second camera is controlled to move closer to or away from the welding workpiece in a direction perpendicular to the welding platform to re-acquire the second image, and the above steps are repeated until the similarity is greater than or equal to the preset parameter, and the finally determined first virtual welding path is used as the virtual welding path.

2. The control method according to claim 1, characterized in that, The step of identifying the weld seam by the image technique includes image preprocessing, weld seam identification, and weld seam segmentation.

3. The control method according to claim 1, characterized in that, The step of planning the actual welding path based on the virtual welding path and the first relative coordinates includes: Based on the first relative coordinates of the main reference point and the first marking point, the first virtual welding path that meets the conditions is transformed into an actual welding path referring to the first marking point.

4. The control method according to claim 1, characterized in that, The step of obtaining the first relative coordinates of the second marking point relative to the first marking point based on the first image specifically includes: Based on the first image and a preset image, it is determined whether there is a workpiece to be welded on the welding platform; If so, the first image is preprocessed to improve the resolution of the first image; based on the first image with improved resolution, the workpiece to be welded and the second marking point are identified to obtain the first relative coordinates of the second marking point relative to the first marking point.

5. The control method according to claim 1, characterized in that, The step of controlling the stirring pin to extend into the weld seam of the workpiece to be welded and performing friction stir welding on the workpiece to be welded along the actual welding path includes: Identifying the label of the workpiece to be welded based on the dimension information; based on the label, obtaining the rotation speed and moving speed of the stirring pin; wherein, the dimension information is obtained based on the recognition of the first image; Randomly selecting one end of the actual welding path as the starting point, according to the relative position between the starting point and the first marking point, controlling the stirring pin to move directly above the starting point, controlling the stirring pin to extend into the weld seam of the workpiece to be welded, and driving the stirring pin to perform friction stir welding on the workpiece to be welded along the actual welding path at the rotation speed and the moving speed.

6. A machine tool, characterized in that, Includes: A frame; A welding platform, which is connected to the frame and is used to place the workpiece to be welded; The welding platform is provided with a first marking point; A first camera, which is fixed to the frame and is used to collect a first image containing the workpiece to be welded on the welding platform; wherein, the workpiece to be welded is provided with a second marking point; A second camera, which is movably arranged relative to the frame; A friction stir welding head, which is movably arranged on the frame and is provided with a stirring pin; and A processor, which is electrically connected to the first camera, the second camera, and the friction stir welding head respectively; the processor is configured to: adopt the friction stir welding control method based on visual sensing according to any one of claims 1 to 5.

7. The machine tool according to claim 6, characterized in that, The friction stir welding machine head is provided with a rotatable shoulder, and the stirring pin is connected to the shoulder; when the stirring pin extends to the weld seam of the workpiece to be welded, the shoulder abuts against the surface of the workpiece to be welded. Wherein, the shoulder is provided with a groove, and the opening of the groove is formed on the surface; the groove extends along the radial direction of the shoulder from the connection between the shoulder and the stirring pin to the outer peripheral wall of the shoulder, and the groove depth of the groove gradually increases in its extending direction.

8. A friction stir welding control system based on visual sensing, characterized in that, Comprising: One or more processors; A memory; And One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the vision sensing-based friction stir welding control method according to any one of claims 1 to 5.

Images