Weld joint identification method, device and equipment and storage medium
The weld seam recognition method based on a dual-stream neural network architecture solves the problem of low efficiency in trajectory information and point cloud matching, and realizes efficient automatic welding operations.
Patent Information
- Application Number
- CN202510770040.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-09-19
AI Technical Summary
The matching efficiency between existing trajectory information and spatial point cloud is low, resulting in low automatic welding efficiency.
A dual-stream neural network architecture of trajectory encoder and point cloud encoder is adopted to extract, discriminate and normalize the weld trajectory and point cloud features, calculate the similarity to determine the matching relationship, and perform welding operations based on the matching relationship.
The matching efficiency between weld trajectory and point cloud features is improved, accurate welding operation is achieved, and automatic welding efficiency is improved.
Smart Images
Figure CN120673148A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of automatic welding technology, and in particular to a weld identification method, device, equipment and storage medium. Background Art
[0002] Welding automation is an important means to improve welding quality and efficiency and reduce labor costs. Weld seam tracking is one of the necessary technologies for achieving welding automation and has been widely used in the field of high-end equipment manufacturing.
[0003] In the field of robotics, spatial equipment is often required to provide discrete trajectory points to the robot body. The robot body's own perception equipment will classify and predict the surrounding point cloud. At this time, it is necessary to match the trajectory information provided by the spatial equipment with the spatial point cloud recognized by the robot body.
[0004] Currently, the matching efficiency of trajectory information and spatial point cloud is low, and the efficiency of automatic welding is low. Summary of the Invention
[0005] The present disclosure provides a weld identification method, device, equipment and storage medium to at least solve the problems of low matching efficiency between existing trajectory information and spatial point clouds and low automatic welding efficiency.
[0006] The technical solutions disclosed in this disclosure are as follows:
[0007] The present disclosure provides a weld identification method, comprising:
[0008] Scanning the welds of the object to be welded to obtain weld trajectories and weld point clouds corresponding to each weld;
[0009] Inputting each weld track into a track encoder to obtain track features of each weld track;
[0010] Inputting each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud;
[0011] Determining a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory;
[0012] According to the matching relationship and each of the weld seam trajectories, a welding operation is performed on the object to be welded.
[0013] Optionally, scanning the welds of the object to be welded to obtain weld trajectories and weld point clouds corresponding to each weld includes:
[0014] Scanning the welds of the object to be welded using a space mouse connected to the robot network to obtain the weld trajectory corresponding to each weld; and
[0015] The radar sensor installed on the robot is used to scan the welds of the object to be welded to obtain the weld point cloud corresponding to each weld.
[0016] Optionally, the trajectory encoder includes: an LSTM network, a first feature discrimination network, and a first normalization network. Inputting each weld trajectory into the trajectory encoder to obtain the trajectory features of each weld trajectory includes:
[0017] Inputting each weld trajectory into the LSTM network for feature extraction to obtain a first preliminary feature;
[0018] Inputting the first preliminary feature into the first feature discrimination network for feature discrimination to obtain a first discriminant feature;
[0019] The first discriminant feature is input into the first normalization network for feature normalization to obtain the trajectory feature of each weld trajectory.
[0020] Optionally, the point cloud encoder includes: a PointNet++ network, a second feature discrimination network, and a second normalization network. Each weld point cloud is input into the point cloud encoder to obtain point cloud features of each weld point cloud, including:
[0021] Inputting each weld point cloud into the PointNet++ network for feature extraction to obtain a second preliminary feature;
[0022] Inputting the second preliminary feature into the second feature discriminant network for feature discrimination to obtain a second discriminant feature;
[0023] The second discriminant feature is input into the second normalization network for feature normalization to obtain the point cloud feature of each weld point cloud.
[0024] Optionally, determining a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory includes:
[0025] For a target point cloud feature, respectively calculating the similarity between the target point cloud feature and the trajectory features corresponding to the plurality of weld trajectories; the target point cloud feature is a point cloud feature corresponding to a target weld point cloud, and the target weld point cloud is any one of the plurality of weld point clouds;
[0026] Selecting the target weld trajectory with the greatest similarity from the multiple weld trajectories;
[0027] It is determined that there is a matching relationship between the target weld point cloud and the target weld trajectory.
[0028] Optionally, respectively calculating the similarity between the target point cloud feature and the trajectory features corresponding to the plurality of weld trajectories includes:
[0029] For any trajectory feature, matrix multiplication is used to calculate the similarity between the target point cloud feature and the any trajectory feature.
[0030] The present disclosure also provides a weld identification device, comprising:
[0031] A scanning module is used to scan the welds of the object to be welded and obtain a weld trajectory and a weld point cloud corresponding to each weld;
[0032] A trajectory module, used for inputting each weld trajectory into a trajectory encoder to obtain a trajectory feature of each weld trajectory;
[0033] A point cloud module is used to input each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud;
[0034] a matching module, configured to determine a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory;
[0035] The welding module is used to perform welding operations on the objects to be welded according to the matching relationship and each of the weld seam trajectories.
[0036] The present disclosure also provides an electronic device, including:
[0037] processor;
[0038] a memory for storing instructions executable by the processor;
[0039] The processor is configured to execute the instructions to implement the steps in the above method.
[0040] The embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step in the above method is implemented.
[0041] The present disclosure also provides a computer program product, including a computer program / instruction, which implements each step of the above method when executed by a processor.
[0042] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:
[0043] In some embodiments of the present disclosure, the welds of the object to be welded are scanned to obtain the weld trajectory and weld point cloud corresponding to each weld; each weld trajectory is input into a trajectory encoder to obtain the trajectory features of each weld trajectory; each weld point cloud is input into a point cloud encoder to obtain the point cloud features of each weld point cloud; based on the trajectory features and point cloud features of each weld trajectory, the matching relationship between the weld trajectory and the weld point cloud is determined; based on the matching relationship and each weld trajectory, the welding operation is performed on the object to be welded; the present disclosure uses a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder, and the trajectory features and point cloud features obtained can quickly determine the matching relationship between the weld trajectory and the weld point cloud; thereby, the welding operation can be performed accurately, thereby improving the matching efficiency of the trajectory features and the point cloud features, and improving the efficiency of automatic welding.
[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.
[0046] Figure 1 A schematic flow chart of a weld identification method provided by an exemplary embodiment of the present disclosure;
[0047] Figure 2 A schematic diagram of a two-stream neural network architecture provided in an embodiment of the present disclosure;
[0048] Figure 3 A schematic structural diagram of a weld identification device provided by an exemplary embodiment of the present disclosure;
[0049] Figure 4 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0050] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0051] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.
[0052] It should be noted that the user information involved in this disclosure includes but is not limited to: user device information and user personal information; the collection, storage, use, processing, transmission, provision and disclosure of user information in this disclosure comply with the relevant laws and regulations and do not violate public order and good morals.
[0053] In response to the above technical problems, in some embodiments of the present disclosure, the welds of the object to be welded are scanned to obtain the weld trajectory and weld point cloud corresponding to each weld; each weld trajectory is input into a trajectory encoder to obtain the trajectory features of each weld trajectory; each weld point cloud is input into a point cloud encoder to obtain the point cloud features of each weld point cloud; according to the trajectory features and point cloud features of each weld trajectory, the matching relationship between the weld trajectory and the weld point cloud is determined; according to the matching relationship and each weld trajectory, the welding operation is performed on the object to be welded; the present disclosure uses a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder, and the trajectory features and point cloud features obtained can quickly determine the matching relationship between the weld trajectory and the weld point cloud; thereby, the welding operation can be performed accurately; the matching efficiency of the trajectory features and the point cloud features is improved, and the efficiency of automatic welding is improved.
[0054] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0055] Figure 1 A schematic flow chart of a weld identification method provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method includes:
[0056] S101: Scan the welds of the object to be welded to obtain weld trajectories and weld point clouds corresponding to each weld;
[0057] S102: inputting each weld track into a track encoder to obtain track features of each weld track;
[0058] S103: Input each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud;
[0059] S104: Determine a matching relationship between the weld trajectory and the weld point cloud based on the trajectory characteristics and point cloud characteristics of each weld trajectory;
[0060] S105: Perform welding operations on the object to be welded according to the matching relationship and each weld trajectory.
[0061] In this embodiment, the execution subject of the above method may be a terminal device or a server. Optionally, the execution subject of the above method may be a robot.
[0062] Among them, terminal devices include but are not limited to mobile stations (MS), mobile terminals, mobile phones, handsets, and portable equipment. The terminal devices can communicate with one or more core networks via a radio access network (RAN). For example, the terminal devices can be mobile phones (or "cellular" phones), computers with wireless communication capabilities, etc. The terminal devices can also be computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, AR terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical care, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc., and the operating systems installed on the terminal devices include but are not limited to: iOS, Android, Windows, Linux, Mac OS, etc. In different networks, a terminal may be called by different names, such as user equipment, mobile station, subscriber unit, station, cellular phone, personal digital assistant, wireless modem, wireless communication device, handheld device, laptop computer, cordless phone, wireless local loop station, television, etc. For the convenience of description, the terminal is referred to as terminal equipment in this embodiment.
[0063] In this embodiment, the server implementation is not limited. For example, the server can be a conventional server, a cloud server, a cloud host, a virtual center, or other server devices. The server components primarily include a processor, a hard disk, memory, a system bus, and other common computer architecture types.
[0064] In this embodiment, the weld of the object to be welded is scanned to obtain the weld trajectory and weld point cloud corresponding to each weld; each weld trajectory is input into a trajectory encoder to obtain the trajectory feature of each weld trajectory; each weld point cloud is input into a point cloud encoder to obtain the point cloud feature of each weld point cloud; according to the trajectory feature and point cloud feature of each weld trajectory, the matching relationship between the weld trajectory and the weld point cloud is determined; according to the matching relationship and each weld trajectory, the welding operation is performed on the object to be welded; the present disclosure uses a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder, and the trajectory features and point cloud features obtained can quickly determine the matching relationship between the weld trajectory and the weld point cloud; thereby, the welding operation can be performed accurately, thereby improving the matching efficiency of the trajectory features and the point cloud features, and improving the efficiency of automatic welding.
[0065] It's important to note that a Space Mouse is an input device optimized for 3D design, modeling, and visualization. It's primarily used to precisely control objects or perspectives in 3D space. Unlike a traditional 2D mouse, it utilizes six degrees of freedom (6DOF) technology to enable more natural spatial interaction.
[0066] In some embodiments of the present disclosure, the welds of the object to be welded are scanned to obtain weld trajectories and weld point clouds corresponding to each weld. One achievable method is to use a spatial mouse connected to a robot network to scan the welds of the object to be welded to obtain weld trajectories corresponding to each weld; and to use a radar sensor installed on the robot to scan the welds of the object to be welded to obtain weld point clouds corresponding to each weld. Among them, the robot needs to use a spatial mouse to scan the welds of the object to be welded to obtain high-quality weld trajectories for more accurate welding; the robot needs to identify which weld the weld trajectory corresponds to, and therefore, it is necessary to obtain the weld point cloud and the weld trajectory for matching to accurately determine the weld trajectory corresponding to the weld. It should be noted that the present disclosure does not limit the type of radar sensor, which can be a lidar.
[0067] In this embodiment, a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder is adopted. The model of each modality outputs a feature, a point cloud feature or a trajectory feature, and the point cloud feature and the trajectory feature are normalized so that the normalized features can be directly subjected to matrix multiplication operations.
[0068] In a feasible embodiment, after obtaining the weld trajectory and weld point cloud, the vector trajectory is parameterized into a Bézier curve for encoding; the point cloud data is voxelized, and features are extracted using 3DCNN.
[0069] It should be noted that the trajectory encoder includes: an LSTM network, a first feature discrimination network, and a first normalization network.
[0070] In some embodiments of the present disclosure, each weld seam trajectory is input into a trajectory encoder to obtain a trajectory feature for each weld seam trajectory. One achievable method is to input each weld seam trajectory into an LSTM network for feature extraction to obtain a first preliminary feature; input the first preliminary feature into a first feature discriminant network for feature discrimination to obtain a first discriminant feature; and input the first discriminant feature into a first normalization network for feature normalization to obtain a trajectory feature for each weld seam trajectory.
[0071] In an optional embodiment, the trajectory encoder can use an LSTM network to encode the vector trajectory (including the temporal position sequence) input by the user into a feature vector through the LSTM network. The trajectory encoder can also be a CNN network or a graph neural network.
[0072] It should be noted that the point cloud encoder includes: PointNet++ network, the second feature discrimination network and the second normalization network.
[0073] In some embodiments of the present disclosure, each weld point cloud is input into a point cloud encoder to obtain point cloud features for each weld point cloud. One implementation method is to input each weld point cloud into a PointNet++ network for feature extraction to obtain a second preliminary feature; input the second preliminary feature into a second feature discriminant network for feature discrimination to obtain a second discriminant feature; and input the second discriminant feature into a second normalization network for feature normalization to obtain the point cloud features for each weld point cloud.
[0074] In some embodiments of the present disclosure, the matching relationship between the weld trajectory and the weld point cloud is determined based on the trajectory features and point cloud features of each weld trajectory. One achievable method is to calculate the similarity between the target point cloud feature and the trajectory features corresponding to multiple weld trajectories, respectively, for the target point cloud feature; the target point cloud feature is the point cloud feature corresponding to the target weld point cloud, and the target weld point cloud is any one of the multiple weld point clouds; the target weld trajectory with the greatest similarity is selected from the multiple weld trajectories; and it is determined that the target weld point cloud has a matching relationship with the target weld trajectory. The similarity between any point cloud feature and all trajectory features is calculated one by one to obtain the matching relationship between all weld trajectories and weld point clouds.
[0075] In the above embodiment, the similarity between the target point cloud feature and the trajectory features corresponding to multiple weld trajectories is calculated separately. One possible implementation is to use matrix multiplication to calculate the similarity between the target point cloud feature and any trajectory feature for any trajectory feature; the similarity value range is in the interval (0, 1). The present disclosure performs matrix multiplication on the normalized features to directly obtain the similarity value, thereby improving the matching efficiency of trajectory features and point cloud features.
[0076] After obtaining the matching relationship between the weld trajectory and the weld point cloud, the robot performs welding operations on the object to be welded based on the matching relationship and each weld trajectory.
[0077] Figure 2 A schematic diagram of a dual-stream neural network architecture provided by an embodiment of the present disclosure. Figure 2 As shown in the figure, the weld trajectory and weld point cloud are respectively input into the LSTM network and the PointNet++ network for feature extraction to obtain preliminary trajectory features and preliminary point cloud features; then, the trajectory features and point cloud features are respectively input into the feature discrimination network and the normalization network in turn to output point cloud features and trajectory features; the normalized point cloud features and trajectory features can be directly used for cosine similarity calculation to obtain similarity results.
[0078] In an embodiment of the method disclosed herein, the weld of the object to be welded is scanned to obtain a weld trajectory and a weld point cloud corresponding to each weld; each weld trajectory is input into a trajectory encoder to obtain a trajectory feature of each weld trajectory; each weld point cloud is input into a point cloud encoder to obtain a point cloud feature of each weld point cloud; based on the trajectory feature and point cloud feature of each weld trajectory, a matching relationship between the weld trajectory and the weld point cloud is determined; based on the matching relationship and each weld trajectory, a welding operation is performed on the object to be welded; the present invention uses a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder, and the trajectory features and point cloud features obtained can quickly determine the matching relationship between the weld trajectory and the weld point cloud; thereby, the welding operation can be performed accurately; the matching efficiency of the trajectory features and the point cloud features is improved, and the efficiency of automatic welding is improved.
[0079] Figure 3 FIG. 3 is a structural diagram of a weld identification device 30 provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the weld identification device 30 includes: a scanning module 31, a trajectory module 32, a point cloud module 33, a matching module 34 and a welding module 35.
[0080] The scanning module 31 is used to scan the welds of the object to be welded and obtain the weld trajectory and weld point cloud corresponding to each weld;
[0081] The track module 32 is used to input each weld track into a track encoder to obtain the track characteristics of each weld track;
[0082] The point cloud module 33 is used to input each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud;
[0083] A matching module 34 is used to determine the matching relationship between the weld trajectory and the weld point cloud according to the trajectory characteristics and point cloud characteristics of each weld trajectory;
[0084] The welding module 35 is used to perform welding operations on the welding object according to the matching relationship and each weld trajectory.
[0085] Optionally, when scanning the weld of the object to be welded and obtaining the weld trajectory and weld point cloud corresponding to each weld, the scanning module 31 is used to:
[0086] Scanning the weld seams of the object to be welded using a spatial mouse connected to the robot network to obtain weld seam trajectories corresponding to each weld seam; and
[0087] The radar sensor installed on the robot is used to scan the welds of the object to be welded, and the weld point cloud corresponding to each weld is obtained.
[0088] Optionally, the trajectory encoder includes: an LSTM network, a first feature discrimination network, and a first normalization network. When the trajectory module 32 inputs each weld trajectory into the trajectory encoder and obtains the trajectory features of each weld trajectory, it is used to:
[0089] Each weld trajectory is input into the LSTM network for feature extraction to obtain the first preliminary features;
[0090] Inputting the first preliminary feature into the first feature discrimination network for feature discrimination to obtain a first discriminant feature;
[0091] The first discriminant feature is input into the first normalization network for feature normalization to obtain the trajectory feature of each weld trajectory.
[0092] Optionally, the point cloud encoder includes: a PointNet++ network, a second feature discrimination network, and a second normalization network. When the point cloud module 33 inputs each weld point cloud into the point cloud encoder and obtains the point cloud features of each weld point cloud, it is used to:
[0093] Each weld point cloud is input into the PointNet++ network for feature extraction to obtain the second preliminary feature;
[0094] Inputting the second preliminary feature into the second feature discrimination network for feature discrimination to obtain the second discriminant feature;
[0095] The second discriminant feature is input into the second normalization network for feature normalization to obtain the point cloud features of each weld point cloud.
[0096] Optionally, when determining the matching relationship between the weld trajectory and the weld point cloud based on the trajectory features and point cloud features of each weld trajectory, the matching module 34 is configured to:
[0097] For the target point cloud feature, the similarity between the target point cloud feature and the trajectory features corresponding to the multiple weld trajectories is calculated respectively; the target point cloud feature is the point cloud feature corresponding to the target weld point cloud, and the target weld point cloud is any weld point cloud among the multiple weld point clouds;
[0098] Select the target weld trajectory with the greatest similarity from multiple weld trajectories;
[0099] Determine whether there is a matching relationship between the target weld point cloud and the target weld trajectory.
[0100] Optionally, when respectively calculating the similarity between the target point cloud feature and the trajectory features corresponding to the plurality of weld trajectories, the matching module 34 is configured to:
[0101] For any trajectory feature, matrix multiplication is used to calculate the similarity between the target point cloud feature and any trajectory feature; the value range of the similarity is in the interval (0, 1).
[0102] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0103] Figure 4 The present disclosure provides a schematic diagram of the structure of an electronic device. Figure 4 As shown, the electronic device includes: a memory 41 and a processor 42. In addition, the electronic device also includes a power supply component 43 and a communication component 44.
[0104] The memory 41 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device.
[0105] The memory 41 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0106] The communication component 44 is used for data transmission with other devices.
[0107] The processor 42 can execute computer instructions stored in the memory 41 to: scan the weld of the object to be welded to obtain the weld trajectory and weld point cloud corresponding to each weld; input each weld trajectory into a trajectory encoder to obtain the trajectory characteristics of each weld trajectory; input each weld point cloud into a point cloud encoder to obtain the point cloud characteristics of each weld point cloud; determine the matching relationship between the weld trajectory and the weld point cloud based on the trajectory characteristics and point cloud characteristics of each weld trajectory; and perform welding operations on the object to be welded based on the matching relationship and each weld trajectory.
[0108] Optionally, when the processor 42 scans the weld of the object to be welded and obtains the weld trajectory and weld point cloud corresponding to each weld, it is configured to:
[0109] Scanning the weld seams of the object to be welded using a spatial mouse connected to the robot network to obtain weld seam trajectories corresponding to each weld seam; and
[0110] The radar sensor installed on the robot is used to scan the welds of the object to be welded, and the weld point cloud corresponding to each weld is obtained.
[0111] Optionally, the trajectory encoder includes: an LSTM network, a first feature discrimination network, and a first normalization network. When the processor 42 inputs each weld trajectory into the trajectory encoder and obtains the trajectory features of each weld trajectory, it is used to:
[0112] Each weld trajectory is input into the LSTM network for feature extraction to obtain the first preliminary features;
[0113] Inputting the first preliminary feature into the first feature discrimination network for feature discrimination to obtain a first discriminant feature;
[0114] The first discriminant feature is input into the first normalization network for feature normalization to obtain the trajectory feature of each weld trajectory.
[0115] Optionally, the point cloud encoder includes: a PointNet++ network, a second feature discrimination network, and a second normalization network. When the processor 42 inputs each weld point cloud into the point cloud encoder and obtains the point cloud features of each weld point cloud, it is used to:
[0116] Each weld point cloud is input into the PointNet++ network for feature extraction to obtain the second preliminary feature;
[0117] Inputting the second preliminary feature into the second feature discrimination network for feature discrimination to obtain the second discriminant feature;
[0118] The second discriminant feature is input into the second normalization network for feature normalization to obtain the point cloud features of each weld point cloud.
[0119] Optionally, when determining the matching relationship between the weld trajectory and the weld point cloud based on the trajectory features and point cloud features of each weld trajectory, the processor 42 is configured to:
[0120] For the target point cloud feature, the similarity between the target point cloud feature and the trajectory features corresponding to the multiple weld trajectories is calculated respectively; the target point cloud feature is the point cloud feature corresponding to the target weld point cloud, and the target weld point cloud is any weld point cloud among the multiple weld point clouds;
[0121] Select the target weld trajectory with the greatest similarity from multiple weld trajectories;
[0122] Determine whether there is a matching relationship between the target weld point cloud and the target weld trajectory.
[0123] Optionally, when respectively calculating the similarity between the target point cloud feature and the trajectory features corresponding to the plurality of weld trajectories, the processor 42 is configured to:
[0124] For any trajectory feature, matrix multiplication is used to calculate the similarity between the target point cloud feature and any trajectory feature; the value range of the similarity is in the interval (0, 1).
[0125] Accordingly, the embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores the computer program and the computer program is executed by one or more processors, the one or more processors are caused to execute Figure 1 Each step in the embodiment of the method.
[0126] Accordingly, the present disclosure also provides a computer program product, which includes a computer program / instruction, and the computer program / instruction is executed by a processor. Figure 1 Each step in the method embodiment.
[0127] above Figure 4 The communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 4G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0128] above Figure 4The power supply component in a device provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.
[0129] The electronic device also includes a display screen and an audio component.
[0130] The display screen includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0131] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.
[0132] In the embodiments of the apparatus, device, storage medium and computer program product disclosed in the present invention, the weld of the object to be welded is scanned to obtain the weld trajectory and weld point cloud corresponding to each weld; each weld trajectory is input into a trajectory encoder to obtain the trajectory feature of each weld trajectory; each weld point cloud is input into a point cloud encoder to obtain the point cloud feature of each weld point cloud; based on the trajectory feature and point cloud feature of each weld trajectory, the matching relationship between the weld trajectory and the weld point cloud is determined; based on the matching relationship and each weld trajectory, the welding operation is performed on the object to be welded; the present invention uses a dual-stream neural network architecture of a trajectory encoder and a point cloud encoder, and the trajectory features and point cloud features obtained can quickly determine the matching relationship between the weld trajectory and the weld point cloud; thereby, the welding operation can be performed accurately; the matching efficiency of the trajectory features and the point cloud features is improved, and the efficiency of automatic welding is improved.
[0133] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0134] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0135] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0137] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.
[0138] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0139] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.
[0141] The above are merely specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments, but is to be construed in the broadest manner consistent with the principles and novel features disclosed herein.
Claims
1. A weld identification method, characterized in that: include: Scanning the welds of the object to be welded to obtain weld trajectories and weld point clouds corresponding to each weld; Inputting each weld track into a track encoder to obtain track features of each weld track; Inputting each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud; Determining a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory; According to the matching relationship and each of the weld seam trajectories, a welding operation is performed on the object to be welded.
2. The method according to claim 1, characterized in that Scanning the welds of the object to be welded to obtain weld trajectories and weld point clouds corresponding to each weld includes: Scanning the welds of the object to be welded using a space mouse connected to the robot network to obtain the weld trajectory corresponding to each weld; and The radar sensor installed on the robot is used to scan the welds of the object to be welded to obtain the weld point cloud corresponding to each weld.
3. The method according to claim 1, characterized in that The trajectory encoder includes: an LSTM network, a first feature discrimination network and a first normalization network. Each weld trajectory is input into the trajectory encoder to obtain the trajectory features of each weld trajectory, including: Inputting each weld trajectory into the LSTM network for feature extraction to obtain a first preliminary feature; Inputting the first preliminary feature into the first feature discrimination network for feature discrimination to obtain a first discriminant feature; The first discriminant feature is input into the first normalization network for feature normalization to obtain the trajectory feature of each weld trajectory.
4. The method according to claim 1, wherein The point cloud encoder includes: a PointNet++ network, a second feature discrimination network, and a second normalization network. Each weld point cloud is input into the point cloud encoder to obtain point cloud features of each weld point cloud, including: Inputting each weld point cloud into the PointNet++ network for feature extraction to obtain a second preliminary feature; Inputting the second preliminary feature into the second feature discriminant network for feature discrimination to obtain a second discriminant feature; The second discriminant feature is input into the second normalization network for feature normalization to obtain the point cloud feature of each weld point cloud.
5. The method according to claim 1, wherein Determining a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory includes: For a target point cloud feature, respectively calculating the similarity between the target point cloud feature and the trajectory features corresponding to the plurality of weld trajectories; the target point cloud feature is a point cloud feature corresponding to a target weld point cloud, and the target weld point cloud is any one of the plurality of weld point clouds; Selecting the target weld trajectory with the greatest similarity from the plurality of weld trajectories; It is determined that there is a matching relationship between the target weld point cloud and the target weld trajectory.
6. The method according to claim 5, characterized in that The respectively calculating similarities between the target point cloud features and the trajectory features corresponding to the plurality of weld trajectories includes: For any trajectory feature, matrix multiplication is used to calculate the similarity between the target point cloud feature and the any trajectory feature.
7. A weld identification device, characterized in that: include: A scanning module is used to scan the welds of the object to be welded and obtain a weld trajectory and a weld point cloud corresponding to each weld; A trajectory module, used for inputting each weld trajectory into a trajectory encoder to obtain a trajectory feature of each weld trajectory; A point cloud module is used to input each weld point cloud into a point cloud encoder to obtain point cloud features of each weld point cloud; a matching module, configured to determine a matching relationship between the weld trajectory and the weld point cloud according to the trajectory features and the point cloud features of each weld trajectory; The welding module is used to perform welding operations on the objects to be welded according to the matching relationship and each of the weld seam trajectories.
8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement each step in the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.