Welding auxiliary method and system, storage medium and electronic equipment
By obtaining the grayscale map and depth map of the target scene, and using marker codes and point cloud data to estimate and optimize the position of the workpiece, the problem of difficult welding assistance in the prior art is solved, and high-precision welding assistance is achieved.
Patent Information
- Application Number
- CN202311316565.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-11
- Publication Date
- 2025-04-11
AI Technical Summary
The prior art is difficult to achieve effective assistance to the welding process, especially when the workpiece position estimation accuracy is low.
By obtaining the grayscale map and depth map of the target scene, using marker codes to perform rough alignment of the workpiece, and iteratively optimized by observing point clouds and rendering point clouds, the estimated position of the workpiece is obtained, thereby performing intelligent welding assistance.
Accurate estimation of workpiece position is achieved, welding efficiency and accuracy are improved, especially in the alignment ability and precise alignment effect of symmetrical workpieces.
Smart Images
Figure CN119810007A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of welding technology and relates to a welding auxiliary method, and in particular to a welding auxiliary method, system, storage medium and electronic equipment. Background Art
[0002] Intelligent welding technology is an advanced technology applied in the field of welding, which aims to achieve high efficiency, high precision and automation of the welding process through technical means such as automation, computer vision and intelligent control. The rise of this technology has brought far-reaching impacts to the manufacturing and engineering fields, greatly improving the limitations and challenges of traditional welding methods. However, existing technologies are difficult to achieve effective assistance for welding. Summary of the invention
[0003] The purpose of this application is to provide a welding assistance method, system, storage medium and electronic device for realizing intelligent assistance for welding.
[0004] In a first aspect, an embodiment of the present application provides a welding assistance method, which includes: obtaining a grayscale image and a depth map of a target scene, wherein the target scene includes a workpiece and a marking code set on the workpiece; obtaining an initial pose of the workpiece according to the marking code and the grayscale image; obtaining an observation point cloud of the workpiece according to the grayscale image and the depth map; obtaining an estimated pose of the workpiece according to the initial pose, the observation point cloud and the rendering point cloud of the workpiece; and performing assisted welding according to the estimated pose.
[0005] In an implementation of the first aspect, obtaining the initial pose of the workpiece according to the marking code and the grayscale image includes: obtaining a first pose of the workpiece in a marking code coordinate system; obtaining a second pose of the marking code in a camera coordinate system; and obtaining the pose of the workpiece in the camera coordinate system as the initial pose according to the first pose and the second pose.
[0006] In an implementation of the first aspect, obtaining an observation point cloud of the workpiece according to the grayscale image and the depth map includes: obtaining a mask of the workpiece according to the grayscale image; filtering the depth map according to the mask of the workpiece; and obtaining an observation point cloud of the workpiece according to camera intrinsic parameters and the filtered depth map.
[0007] In an implementation manner of the first aspect, filtering the depth map according to the mask of the workpiece includes: determining a position in the depth map corresponding to a mask value of 0 according to the mask of the workpiece, and configuring the depth value of the position to 0.
[0008] In an implementation of the first aspect, obtaining the estimated pose of the workpiece according to the initial pose, the observation point cloud and the rendering point cloud of the workpiece includes: obtaining the estimated pose of the first iteration according to the initial pose, the observation point cloud and the rendering point cloud of the workpiece; and optimizing the estimated pose in an iterative manner according to the observation point cloud and the rendering point cloud of the workpiece and the estimated pose obtained last time.
[0009] In an implementation of the first aspect, the estimated pose obtained in the k+1th iteration is
[0010]
[0011] in, is the estimated pose obtained in the kth iteration, P obs is the observation point cloud, P model For the rendered point cloud, τ k+1 is the maximum matching distance in the k+1th iteration.
[0012] In an implementation manner of the first aspect, the depth map and the grayscale map are obtained by a structured light camera.
[0013] In a second aspect, an embodiment of the present application provides a welding assistance system, which includes: an image acquisition module, used to acquire a grayscale image and a depth map of a target scene, wherein the target scene includes a workpiece and a marking code set on the workpiece; an initial pose acquisition module, used to acquire the initial pose of the workpiece according to the marking code and the grayscale image; an observation point cloud acquisition module, used to acquire the observation point cloud of the workpiece according to the grayscale image and the depth map; an estimated pose acquisition module, used to acquire the estimated pose of the workpiece according to the initial pose, the observation point cloud and the rendering point cloud of the workpiece; and a welding assistance module, used to perform auxiliary welding according to the estimated pose.
[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the welding assistance method described in any one of the first aspects of the embodiments of the present application is implemented.
[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing any welding assistance method described in any one of the first aspects of the embodiment of the present application when the computer program is called.
[0016] An embodiment of the present application provides a welding assistance method, which can accurately obtain the estimated posture of a workpiece and perform intelligent welding assistance based on the estimated posture.
[0017] The embodiment of the present application can perform rough alignment of workpieces through identification codes to improve the alignment capability of symmetrical workpieces and improve the final fine alignment effect, thereby playing a good auxiliary role in welding. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 Shown is a schematic diagram of an application scenario of an embodiment of the present application.
[0019] Figure 2 A flow chart of a welding assistance method provided in an embodiment of the present application is shown.
[0020] Figure 3 Shown is a flow chart for obtaining the initial posture of a workpiece in an embodiment of the present application.
[0021] Figure 4 Shown is a flow chart of obtaining an observation point cloud of a workpiece in an embodiment of the present application.
[0022] Figure 5 Shown is a flowchart for obtaining an estimated pose in an embodiment of the present application.
[0023] Figure 6 Shown is a structural schematic diagram of a welding auxiliary system in an embodiment of the present application.
[0024] Figure 7 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present application.
[0025] Component number description
[0026] 100 Electronic devices
[0027] 101, 107 Processor
[0028] 102 Output Devices
[0029] 103 Input Devices
[0030] 104 Memory Units
[0031] 105 Communication interface
[0032] 106 Storage Media
[0033] 600 Welding Assist System
[0034] 610 Image acquisition module
[0035] 620 Initial pose acquisition module
[0036] 630 Observation point cloud acquisition module
[0037] 640 Estimated pose acquisition module
[0038] 650 Welding Auxiliary Module
[0039] 700 Electronic equipment
[0040] 710 Memory
[0041] 720 processor
[0042] 730 Display
[0043] Steps S21 to S25
[0044] Steps S31 to S33
[0045] Steps S41 to S43
[0046] Steps S51 to S52 DETAILED DESCRIPTION
[0047] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0048] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0049] Intelligent welding technology is an advanced technology applied in the field of welding, and welding based on point cloud alignment is a commonly used method in intelligent welding technology. By using the observed point cloud data to automatically align and intelligently guide the workpiece model during the welding process, the welding efficiency and welding accuracy can be effectively improved. The point cloud is the three-dimensional coordinate data of a large number of discrete points acquired by a 3D camera or other sensors, which can be used to represent the surface shape and geometric features of an object. By estimating the position and posture of the workpiece model in the world coordinate system, the welds marked on the workpiece model can be intelligently positioned and guided. However, the estimation accuracy of the workpiece posture in the related technology is low, making it difficult to achieve effective assistance to welding.
[0050] At least for the above problems, the embodiment of the present application provides a welding assistance method. The method can accurately obtain the estimated posture of the workpiece and perform welding assistance based on the estimated posture. In addition, the embodiment of the present application can perform rough alignment of the workpiece through the identification code to improve the alignment capability of symmetrical workpieces and improve the final fine alignment effect, thereby playing a good auxiliary role in welding.
[0051] The welding auxiliary method provided in the embodiment of the present application can be applied to electronic equipment. Figure 1 The diagram is a schematic diagram of the structure of the electronic device 100 in the embodiment of the present application. Figure 1 As shown, the electronic device 100 includes a processor 101 connected to one or more data storage units. The data storage unit may include a storage medium 106 and a memory unit 104. The storage medium 106 may be read-only, such as a read-only memory (ROM), or readable and writable, such as a hard disk or a flash memory. The memory unit 104 may be a random access memory (RAM). The memory unit 104 may be integrated with the processor 101 or may be an independent component. The processor 101 is the control center of the electronic device 100, and is used to execute program codes to implement functions corresponding to the program instructions. In some possible implementations, the processor 101 includes one or more central processing units (CPU), for example, Figure 1 In some possible implementations, the electronic device 100 includes more than one processor, for example, Figure 1 The processors 101 and 107 are shown. Both the processors 101 and 107 can be single-core processors or multi-core processors. It should be noted that the term "processor" used herein refers to one or more devices, circuits and / or processing cores for processing data such as computer program instructions.
[0052] The CPU of processor 101 and / or 107 stores the executed program code in memory unit 104 or storage medium 106. In some possible implementations, the program code stored in storage medium 106 can be copied to memory unit 104 for processor execution. The processor can control the operation of electronic device 100 by controlling the execution of other programs, controlling the communication with peripheral devices, and controlling the use of resources of electronic device 100 through the kernel.
[0053] The electronic device 100 may further include a communication interface 105 , through which the electronic device 100 may communicate with another device or system directly or through an external network.
[0054] In some possible implementations, the electronic device 100 also includes an output device 102 and an input device 103. The output device 102 is connected to the processor 101 and can display output information in one or more ways. An example of the output device 102 is a visual display device, such as a liquid crystal display (LCD), a light-emitting diode (LED) display, a cathode ray tube (CRT) or a projector. The input device 103 is connected to the processor 101 and can receive user input in one or more ways. Examples of the input device 103 include a mouse, a keyboard, a touch screen device, a sensor device, and the like.
[0055] The above elements of the electronic device 100 may be connected to each other via any one or more combinations of buses such as a data bus, an address bus, a control bus, an extension bus, and a local bus.
[0056] The electronic device 100 may be a general electronic device or an application-specific electronic device. As a practical example, the electronic device 100 may be a storage array, an application server, a supercomputer, a desktop computer, a laptop computer, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a telecommunication device, or a computer having a plurality of such devices. Figure 1 Any other device of similar structure shown. However, the present application is not limited to any specific type of electronic device. The program code with different functions stored in memory 104 is formed into a process after being run by the processor (processor 101 or processor 107). When the process is running, the processor needs to allocate a section of memory space to each process to store the data generated during the process running. In order to facilitate the data communication between each process, the processor (processor 101 or processor 107) will usually divide a section of shared memory in the memory, and allocate the shared memory to multiple processes that need to share data. The process in the embodiment of the present application can be a virtual machine, a container and any other process with data sharing requirements.
[0057] Figure 2 The flowchart of the welding auxiliary method provided by the embodiment of the present application is shown. Figure 2 As shown, the welding auxiliary method provided in the embodiment of the present application includes the following steps S21 to S25.
[0058] S21, obtaining a grayscale image and a depth image of a target scene. The target scene includes a workpiece and a marking code set on the workpiece. Exemplarily, the marking code can be set on the workpiece by pasting or the like.
[0059] In some possible implementations, a 3D camera may be used to shoot the target scene to obtain a depth map and a grayscale map of the target scene.
[0060] In some possible implementations, the marking code may be an ArUco code. The ArUco code is characterized by a unique black and white grid pattern, each code having four distinct corner points and black and white grids. The ArUco code has a simple and effective design, so it can be printed on the surface of the workpiece and is easy to detect and identify. By detecting and identifying the ArUco code, the system can accurately determine the position and posture of the workpiece.
[0061] S22, in the rough alignment stage, the initial pose of the workpiece is obtained according to the marking code and the grayscale image.
[0062] S23, obtaining an observation point cloud of the workpiece according to the grayscale image and the depth image. The observation point cloud is a three-dimensional point set converted from the grayscale image and depth information captured by the camera, and is used to represent the three-dimensional shape of the workpiece.
[0063] S24, in the fine alignment stage, obtaining an estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece, the estimated pose being the finely aligned pose. In some possible implementations, the estimated pose may be obtained by an ICP algorithm.
[0064] S25, performing auxiliary welding according to the estimated posture. For example, the weld seam marked on the workpiece model can be intelligently positioned and guided according to the estimated posture.
[0065] Figure 3 The flowchart of obtaining the initial position and posture of the workpiece according to the marking code and the grayscale image in the embodiment of the present application is shown. Figure 3 As shown, obtaining the initial posture of the workpiece according to the marking code and the grayscale image includes the following steps S31 to S33.
[0066] S31, obtaining the first position of the workpiece in the marking code coordinate system. The marking code coordinate system may be a coordinate system constructed with a feature point in the marking code as the origin. The feature point in the marking code may be, for example, a center point or a corner point of the marking code.
[0067] In some possible implementations, the first pose can be obtained by inverting the pose of the ArUco code in the workpiece coordinate system.
[0068] S32, obtaining a second posture of the marker code in the camera coordinate system. The camera coordinate system may be a coordinate system constructed with a feature point in the camera as the origin.
[0069] In some possible implementations, the second pose may be estimated by an algorithm in the Opencv library.
[0070] S33, acquiring the pose of the workpiece in the camera coordinate system as the initial pose according to the first pose and the second pose.
[0071] In some possible implementations, the pose of the workpiece in the camera coordinate system is It can be obtained by the following formula:
[0072]
[0073] in, represents the second posture, Indicates the first position.
[0074] Figure 4 The flowchart of obtaining the observation point cloud of the workpiece according to the grayscale image and the depth image in the embodiment of the present application is shown. Figure 4 As shown, obtaining the observation point cloud of the workpiece according to the grayscale image and the depth image includes the following steps S41 to S43.
[0075] S41, obtaining a mask of the workpiece according to the grayscale image. Exemplarily, a segmentation algorithm may be used to process the grayscale image to obtain a mask of the workpiece, and the segmentation algorithm may be, for example, threshold segmentation, region growing, edge detection, convolutional neural network (CNN), and the like.
[0076] S42, filtering the depth map according to the mask of the workpiece.
[0077] In some possible implementations, filtering the depth map according to the mask of the workpiece includes: determining positions in the depth map corresponding to mask values of 0 according to the mask of the workpiece, and configuring the depth values of these positions to 0, while keeping the depth values of other positions unchanged.
[0078] S43, obtaining an observation point cloud of the workpiece according to the camera internal parameters and the filtered depth map.
[0079] In some possible implementations, obtaining the observation point cloud of the workpiece based on the camera intrinsic parameters (used to calculate the mapping of pixels to 3D point clouds) and the filtered depth map includes: ignoring pixels with a depth of 0, converting the pixel coordinates (u, v) in the filtered depth map into normalized coordinates (x, y), converting the normalized coordinates (x, y) and the depth value Z (u, v) into point cloud coordinates (X, Y, Z), and obtaining the observation point cloud of the workpiece based on the point cloud coordinates (X, Y, Z), where (X, Y, Z) are coordinates in the camera coordinate system.
[0080] Exemplarily, obtaining the observation point cloud of the workpiece according to the camera intrinsic parameters and the filtered depth map can be expressed by the following formula:
[0081] x = (u - cx) / fx;
[0082] y = (v-cy) / fy;
[0083] X = x × Z (u, v);
[0084] Y = y × Z (u, v);
[0085] Z = Z(u, v);
[0086] Among them, the camera intrinsic parameters include cx, cy, fx, and fy, where cx represents the horizontal coordinate of the optical center point on the image, cy represents the vertical coordinate of the optical center point on the image, fx represents the horizontal focal length, and fy represents the vertical focal length.
[0087] Figure 5 The flowchart of the embodiment of the present application is shown as follows: obtaining the estimated pose of the workpiece according to the initial pose, the observation point cloud of the workpiece and the rendering point cloud. Figure 5 As shown, in the embodiment of the present application, obtaining the estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece includes the following steps S51 and S52.
[0088] S51, obtaining an estimated pose of a first iteration according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece.
[0089] In some possible implementations, the estimated pose of the first iteration can be expressed as get:
[0090]
[0091] in, represents the initial pose, P obs is the observation point cloud of the workpiece, P model To render the point cloud, it can be obtained by model rendering. τ1 is the maximum matching distance in the first iteration, which can be configured according to experience or actual needs.
[0092] S52, optimizing the estimated pose in an iterative manner according to the observed point cloud and the rendered point cloud of the workpiece and the estimated pose obtained last time. The iteration termination condition is, for example, that the number of iterations reaches a preset value.
[0093] In some possible implementations, the estimated pose obtained in the k+1th iteration is
[0094]
[0095] in, is the estimated pose obtained in the kth iteration, where k is a positive integer. τ k+1 is the maximum matching distance in the k+1th iteration. In some possible implementations, τ k+1 It gradually decreases with the increase of iteration number.
[0096] In some possible implementations, the grayscale image and depth image obtained in step S21 are respectively dedistorted grayscale image and depth image obtained by a structured light camera, and the pixels of the two images correspond to each other.
[0097] According to the above description, the embodiment of the present application provides an intelligent welding assistance method based on a marker code. The method can use the marker code to perform rough alignment of the workpiece posture in the rough alignment stage, and can use the ICP algorithm for final alignment in the fine alignment stage. The estimated posture thus obtained has a high accuracy, and thus can well assist the user in welding.
[0098] The protection scope of the welding auxiliary method provided in the embodiment of the present application is not limited to the execution order of the steps listed in the present embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.
[0099] The embodiment of the present application also provides a welding assistance system, which can implement the welding assistance method provided in the embodiment of the present application. However, the implementation device of the welding assistance method described in the present application includes but is not limited to the structure of the welding assistance system listed in the embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.
[0100] Figure 6 The structure diagram of the welding auxiliary system 600 provided in the embodiment of the present application is shown. Figure 6 As shown, the welding auxiliary system 600 includes an image acquisition module 610, an initial pose acquisition module 620, an observation point cloud acquisition module 630, an estimated pose acquisition module 640 and a welding auxiliary module 650. Among them, the image acquisition module 610 is used to acquire a grayscale image and a depth map of the target scene, and the target scene includes a workpiece and a marking code set on the workpiece. The initial pose acquisition module 620 is used to acquire the initial pose of the workpiece according to the marking code and the grayscale image. The observation point cloud acquisition module 630 is used to acquire the observation point cloud of the workpiece according to the grayscale image and the depth map. The estimated pose acquisition module 640 is used to acquire the estimated pose of the workpiece according to the initial pose, the observation point cloud of the workpiece and the rendering point cloud. The welding auxiliary module 650 is used to assist welding according to the estimated pose.
[0101] It should be noted that the various modules included in the welding auxiliary system 600 are Figure 2 Steps S21 to S25 in the welding auxiliary method shown correspond to each other and will not be described in detail here.
[0102] In some possible implementations, the initial pose acquisition module 620 acquires the initial pose of the workpiece according to the marking code and the grayscale image, including: acquiring the first pose of the workpiece in the marking code coordinate system; acquiring the second pose of the marking code in the camera coordinate system; and acquiring the pose of the workpiece in the camera coordinate system as the initial pose according to the first pose and the second pose.
[0103] In some possible implementations, the observation point cloud acquisition module 630 acquires the observation point cloud of the workpiece according to the grayscale image and the depth map, including: acquiring the mask of the workpiece according to the grayscale image; filtering the depth map according to the mask of the workpiece; and acquiring the observation point cloud of the workpiece according to the camera intrinsic parameters and the filtered depth map.
[0104] In some possible implementations, filtering the depth map according to the mask of the workpiece includes: determining positions in the depth map corresponding to mask values of 0 according to the mask of the workpiece, and configuring depth values of these positions to be 0.
[0105] In some possible implementations, the estimated pose acquisition module 640 acquires the estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece, including: acquiring the estimated pose of the first iteration according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece; optimizing the estimated pose in an iterative manner according to the observed point cloud and the rendered point cloud of the workpiece and the estimated pose obtained last time.
[0106] In some possible implementations, the estimated pose obtained in the k+1th iteration is It can be obtained by the following formula:
[0107]
[0108] in, is the estimated pose obtained in the kth iteration, P obs To observe the point cloud, P model To render the point cloud, τ k+1 is the maximum matching distance in the k+1th iteration.
[0109] In some possible implementations, the grayscale image and depth image acquired by the image acquisition module 610 are respectively dedistorted grayscale images and depth images obtained by a structured light camera, and the pixels of the two images correspond to each other.
[0110] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.
[0111] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.
[0112] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0113] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by the processor, the welding auxiliary method provided by the present application embodiment is implemented. It can be understood by a person skilled in the art that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state hard disk (SSD)), etc.
[0114] The present application also provides an electronic device. Figure 7 The structure diagram of the electronic device 700 in one embodiment of the present application is shown. Figure 7 As shown, in this embodiment, the electronic device 700 includes a memory 710 and a processor 720 .
[0115] The memory 710 is used to store computer programs; preferably, the memory 710 includes: ROM, RAM, disk, USB flash drive, memory card or CD-ROM and other media that can store program codes.
[0116] Specifically, the memory 710 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 700 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 710 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.
[0117] The processor 720 is connected to the memory 700 and is used to execute the computer program stored in the memory 710 so that the electronic device 700 performs the welding auxiliary method.
[0118] In some possible implementations, the electronic device 700 may further include a display 730. The display 730 is communicatively connected with the memory 710 and the processor 720, and is used to display a GUI (Graphical User Interface) related to the welding auxiliary method.
[0119] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0120] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A welding auxiliary method, characterized in that: The welding auxiliary method comprises: Acquire a grayscale image and a depth image of a target scene, wherein the target scene includes a workpiece and a marking code set on the workpiece; Acquire the initial posture of the workpiece according to the marking code and the grayscale image; Acquire an observation point cloud of the workpiece according to the grayscale image and the depth image; Acquire an estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece; Auxiliary welding is performed according to the estimated posture.
2. The welding auxiliary method according to claim 1, characterized in that: Acquiring the initial posture of the workpiece according to the marking code and the grayscale image includes: Obtaining the first position of the workpiece in the marking code coordinate system; Obtaining a second pose of the marker code in the camera coordinate system; The pose of the workpiece in the camera coordinate system is acquired as the initial pose according to the first pose and the second pose.
3. The welding auxiliary method according to claim 1, characterized in that: Acquiring the observation point cloud of the workpiece according to the grayscale image and the depth image includes: Acquire a mask of the workpiece according to the grayscale image; filtering the depth map according to a mask of the workpiece; The observation point cloud of the workpiece is obtained according to the camera intrinsic parameters and the filtered depth map.
4. The welding auxiliary method according to claim 3, characterized in that: Filtering the depth map according to the mask of the workpiece includes: determining a position in the depth map corresponding to a mask value of 0 according to the mask of the workpiece, and configuring the depth value of the position to be 0.
5. The welding auxiliary method according to claim 1, characterized in that: Acquiring the estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece comprises: Acquire an estimated pose of a first iteration according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece; The estimated pose is optimized in an iterative manner according to the observed point cloud and the rendered point cloud of the workpiece and the estimated pose obtained last time.
6. The welding auxiliary method according to claim 5, characterized in that: The estimated pose obtained in the k+1th iteration is Where ICP is an iterative nearest neighbor alignment algorithm. is the estimated pose obtained in the kth iteration, P obs is the observation point cloud, P model For the rendered point cloud, τ k+1 is the maximum matching distance in the k+1th iteration.
7. The welding auxiliary method according to claim 1, characterized in that: The depth map and the grayscale map are obtained by a structured light camera.
8. A welding auxiliary system, characterized in that: The welding auxiliary system comprises: An image acquisition module, used to acquire a grayscale image and a depth image of a target scene, wherein the target scene includes a workpiece and a marking code set on the workpiece; An initial posture acquisition module, used to acquire the initial posture of the workpiece according to the marking code and the grayscale image; An observation point cloud acquisition module, used to acquire an observation point cloud of the workpiece according to the grayscale image and the depth image; An estimated pose acquisition module, used for acquiring an estimated pose of the workpiece according to the initial pose, the observed point cloud and the rendered point cloud of the workpiece; A welding auxiliary module is used to perform auxiliary welding according to the estimated posture.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the welding assisting method according to any one of claims 1 to 7 is implemented.
10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes the welding assistance method according to any one of claims 1 to 7 when calling the computer program.