Method and system for image segmentation and bonding of materials
Through deep learning technology and neural networks, image segmentation in computer-assisted welding is solved, the problem of joint identification in the existing technology is improved, the accuracy and efficiency of welding is improved, and a more efficient welding process is achieved.
Patent Information
- Application Number
- CN202410173962.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2024-02-07
- Publication Date
- 2024-11-26
AI Technical Summary
When identifying and segmenting joints in digital images, existing computer-assisted welding technology is affected by factors such as lighting changes, uneven backgrounds and material deviations, making it difficult for image processing to accurately find suitable joints, affecting welding quality and efficiency.
The trained neural network is used to convert digital images into tensors, and through deep learning technology, image segmentation is performed using convolutional neural networks, etc., to generate a segmentation mask to determine the junction point, and automatic junction is achieved by combining imaging sensors and image processing computers.
Improves the accuracy of joint identification and welding quality, increases production efficiency, reduces material waste and energy consumption, and achieves more accurate dimensional consistency.
Smart Images

Figure CN119027498A_ABST
Abstract
Description
Background Art
[0001] Computer-aided welding (CAW) uses computers to improve welding quality and productivity. Computer vision (CV) is an aspect of CAW that uses imaging and digital image processing to automate various welding tasks such as seam tracking and quality inspection. CV methods can improve consistency, accuracy, reduce rework, and enable real-time monitoring of the welding process. Joining two materials by computer-aided welding (CAW) requires determining the appropriate points (joining points) where the materials will be joined together. CV is required to analyze, identify, and separate corresponding areas in a digital image. The task of identifying and separating corresponding areas is called "segmentation" in CV.
[0002] The subject matter of the present application relates to a method for joining materials. Summary of the invention
[0003] Disclosed herein are methods and systems for segmenting and joining materials. In one aspect of the present disclosure, a method for joining materials includes: providing two materials; placing a first portion of a first material adjacent to a second portion of a second material; acquiring digital images of the first portion and the second portion through an imaging sensor; converting the digital image into a tensor, the tensor including at least a first dimension, a second dimension, and a third dimension, wherein the first dimension includes the height of the digital image, the second dimension includes the width of the digital image, and the third dimension includes the number of digital channels of the imaging sensor; inputting the tensor into a trained neural network (NN); outputting a segmentation mask by the NN; determining a joining point using the segmentation mask; and joining the first material and the second material at the joining point. In another aspect of the present disclosure, the first material and / or the second material include a metal. In another aspect of the present disclosure, the joining includes welding, brazing, and / or soldering. In another aspect of the present disclosure, welding includes gas welding, arc welding, resistance welding, energy beam welding, or ultrasonic welding. In another aspect of the present disclosure, the first material and / or the second material include a wire. In another aspect of the disclosure, the junction is determined by thresholding, connected components, contours, morphological segmentation, or Gaussian mixture. In another aspect of the disclosure, thresholding includes histogram shape-based thresholding, clustering-based thresholding, entropy-based thresholding, object attribute-based thresholding, and spatial thresholding. In yet another aspect of the disclosure, the digital image is segmented by at least one of semantic segmentation, panoptic segmentation, and instance segmentation.
[0004] In one aspect of the disclosure, the junction is determined by determining at least one of a distance (gap), a convexity, a boundary contour, and a centroid (center point) of the first portion and the second portion. In another aspect of the disclosure, deep learning is used to train an artificial neural network. In another aspect of the disclosure, the NN includes training at least one of a convolutional NN, a fully convolutional neural network (FCN), a visual transformer (ViT), and a SNN. In another aspect of the disclosure, training includes reducing an error associated with a training set.
[0005] In one aspect of the present disclosure, the first material and the second material are conductive hairpins, and the joining includes hairpin welding for manufacturing a stator, wherein placing a first portion of the first material adjacent to a second portion of the second material includes introducing the hairpin into the stator so that portions of adjacent hairpins are placed adjacent to each other, determining a joining point includes determining the shape and orientation of the portions placed adjacent to each other based on the segmentation mask, and joining the first material and the second material includes welding the joining point to form a stator winding from the welded hairpins. In another aspect of the present disclosure, a digital image is acquired from a cross-section of the portions placed adjacent to each other. In another aspect of the present disclosure, the method may include preprocessing the digital image by normalizing, cropping and / or scaling the digital image. In another aspect of the present disclosure, the NN is an artificial NN or a spiking neural network.
[0006] A system for joining materials is disclosed herein. In one aspect of the disclosure, a system includes: a holder, the holder being adapted to hold at least a first material and a second material so that a first portion of the first material is placed adjacent to a second portion of the second material; a camera, the camera being adapted to acquire a digital image of the first portion and the second portion; an image processing computer, the image processing computer being adapted to process the digital image into a segmentation mask using a neural network (NN) and determine a joining point of the materials; and a joining device, the joining device being adapted to join the first material and the second material at the joining point. In another aspect of the disclosure, the joining device is a welding device, a brazing device, or a soldering device. In another aspect of the disclosure, the welding device is a gas welding device, an arc welding device, a resistance welding device, an energy beam welding device, or an ultrasonic welding device. In another aspect of the disclosure, the holder is a stator having a hole, the hole holding the conductive hairpins parallel to each other so that portions of adjacent hairpins are placed adjacent to each other, and the joining device is a hairpin welding device. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 An automated joining system according to one or more embodiments is shown.
[0008] Figure 2 A flow chart illustrating method steps for joining materials according to one or more embodiments is shown.
[0009] Figure 3A A hair clip according to one or more embodiments is shown.
[0010] Figure 3B Two hair clips are shown positioned adjacent to each other according to one or more embodiments.
[0011] Figure 4A Digital images of two cross-sections of portions of two adjacent hairpins are shown according to one or more embodiments.
[0012] Figure 4B According to one or more embodiments, Figure 4A Segmentation mask of a digital image.
[0013] Figure 5A A stator of an electric motor is shown according to one or more embodiments.
[0014] Figure 5B According to one or more embodiments, Figure 5A of the stator and parts of two adjacent hairpins.
[0015] Figure 5C shows placed next to each other according to one or more embodiments Figure 5B part. DETAILED DESCRIPTION
[0016] To find suitable joints, digital images of possible joints are acquired and such digital images may be processed by digital image processing, which uses digital computers and computer vision (CV)-based algorithms that have historically been manually programmed to analyze the digital images and identify joints. However, in real-world conditions, imaging algorithms may be limited due to lighting variations, uneven backgrounds, material deviations, and blurring of actual images, which may make it difficult for the image processing computer to find suitable joints. Disclosed herein are methods and systems for improving joint identification and, therefore, material joining, through segmentation performed using a neural network.
[0017] Figure 1 A computer-aided joining system 100 is shown. The system 100 consists of a fixture 102 that holds a first material 104 and a second material 106. The first material 104 and the second material 106 can be components of a clamping device, a suction device, a conveying element, or an extraction device. Figure 1The fixture 102 is schematically depicted as a chuck that may or may not rotate on its axis, but any suitable workpiece holding device may be used for a given application. The first material 104 and the second material 106 may be any suitable material for computer-assisted joining, such as a battery electrode, a motor stator hairpin. Although further discussion may only focus on one or more of these examples, the present disclosure is not limited thereto.
[0018] The system 100 may also include an optical assembly 110. The optical assembly 110 may include a light 112, a camera 114, an image processing computer 116, and a controller 118. The light 112 may be used to provide illumination for the camera 114. The camera 114, in conjunction with the image processing computer 116, constructs a digital image of the possible joint points 122. The image processing computer 116 is adapted to provide image segmentation, identify edges or regions of the first material 104 and the second material 106 in the digital image, and determine a suitable joint point from the possible joint points 122 in the digital image and send axis coordinates of the suitable joint point to the controller 118 that controls the joint device 124.
[0019] In some embodiments, the digital image is acquired by one or more image sensors, such as the sensor of camera 114, wherein the image sensor converts electromagnetic waves from the first material and the second material into electrical signals. In some embodiments, the image sensor includes one or more of a charge coupled device (CCD), an active pixel sensor (APS), an optical coherence tomography (OCT) sensor, a line scanner, a stereo and / or trifocal camera system, a time of flight camera, a light detection and ranging (LIDAR) sensor, and a radar (radio direction finding and ranging) sensor. In some embodiments, the APS is an image sensor with pixels, including: a photodetector and an active transistor. The APS can be an N-type metal oxide semiconductor (NMOS) APS, a metal oxide semiconductor (MOS) APS, a complementary metal oxide semiconductor (CMOS) APS, or a dynamic vision sensor (if the camera is an event camera), the dynamic vision sensor including an imaging sensor that responds to local brightness changes. In another example, one or more image sensors may include a time of flight camera to obtain distance information to the first material 104 and the second material 106 or other reference objects.
[0020] The joining device 124 may include the necessary mechanical devices for the degrees of freedom required for the mechanical joining process. For example, the joining device 124 is moved in the X, Y and Z directions and / or the fixture 102 is moved, for example, along a rotation axis (not shown) to the coordinates of the appropriate joining point. This may include appropriate belts, gears, motors, stepper motors, actuators, etc., which are generally schematically represented as motors 126. The joining head 128 is suitable for joining the first material 104 and the second material 106, and may include appropriate joining devices based on the type of materials, for example, the joining head 128 may include one or more of a laser welder, an arc welder, an ultrasonic welder, a solvent dispenser, an actuation syringe, a solenoid, etc. The controller 118 is suitable for moving the joining head 128 to the determined appropriate joining point, for example, using the motor 126, and controlling the activation time and / or power of the joining head 128. The image processing computer 116 of the joining system 100 is suitable for performing image segmentation and determining the appropriate joining point based on the digital image acquired by the camera 114, which will be further discussed below.
[0021] Methods for joining materials, e.g. Figure 2 The method 200 shown and described below enables quick and easy finding of suitable joining points, thereby improving the efficiency and increasing the throughput of the system 100. In addition, the method allows for a faster production process with more precise dimensional and material consistency, the production process using only the required amount of joining material, thus minimizing waste while reducing energy consumption.
[0022] Figure 2 A method 200 of exemplary method steps for material joining is shown. The method includes the following steps, which may each be used alone or in combination with one or more other steps in the method 200. Although the method 200 below is described as an overview, additional details and optional features are further described below.
[0023] In a first step 202, two materials are provided, such as a first material 104 and a second material 106. The two materials can be different or the same. In one example, the material can be a metal, including a wire or strip. In another example, the material can be a plastic. As will be discussed below, the specific type of joining, such as welding, solvent welding, brazing, gluing, soldering, etc., will depend on the materials to be joined. For example, in one particular embodiment, the first material 104 and the second material 106 are hairpins, and the joining is hairpin welding. The term "hairpin" is used in the industry for a U-shaped conductive element that is very much like a hairpin, which is typically used to join windings in an electric motor. Joining hairpins will be discussed as an exemplary application. Figure 3A , Figure 3B , Figure 4A and Figure 4Band the remainder of method 200. However, as previously mentioned, additional materials and configurations may also be used. Figure 3A An exemplary hairpin 304 is shown. The hairpin 304 comprises a wire, typically comprising copper. Each hairpin 304 comprises two portions 304A, 304B to be joined to an adjacent hairpin.
[0024] In step 204, a first portion of a first material (e.g., Figure 1 The first material 104, which may be embodied as a first hairpin) and the second portion of the second material (eg, Figure 1 A second material 106, which may be embodied as a second hairpin, is placed adjacent thereto. Figure 3B An example of a first hair clip 304 and a second hair clip 306 is shown. A second portion 304B of the first hair clip 304 is positioned adjacent to a first portion 306A of the second hair clip 306.
[0025] In step 206 , digital images of the ends of the first portion and the second portion are acquired by an imaging sensor (eg, the imaging sensor described above with camera 114 ). Figure 4A An exemplary digital image of a cross section of a second portion 304B of a first hairpin 304 and a top view of a cross section of a first portion 306A of a second hairpin 306 in a position where the first hairpin 304 and the second hairpin 306 are to be joined is shown. As can be seen from the photographs, the edges of the cross section may be blurred or difficult to distinguish from the background due to camera depth of field, optical distortion, background non-uniformity, or the cross section may be rough or have streaks caused by cutting of the material, and / or the material may reflect light inconsistently due to tilted surfaces, each of which typically causes inaccuracies in prior art methods and systems.
[0026] A digital image comprises a plurality of pixels arranged in rows and columns. Each pixel holds a value representing the brightness of a given color. A digital image may be a raster image or a bitmap image. A digital image may also be pre-processed, which may include, for example, normalizing, cropping, and / or scaling the digital image.
[0027] In step 208, the digital image is converted into a tensor with the height of the digital image as the first dimension of the tensor, the width of the digital image as the second dimension of the tensor, and the number of communication paths (digital channels) of the imaging sensor for transmitting digital signals as the third dimension of the tensor. The height of the digital image is represented by rows of pixels, the width of the digital image is represented by columns of pixels, and the number of digital channels is represented by pixel values. In some embodiments, the digital channels are communication paths for a single sensor, multiple sensors, and all sensors of the optical assembly 110. In some embodiments, the dimensionality of the tensor may be higher, for example, for a batch of digital images. In addition, the specific order of dimensions discussed herein is arbitrary and can be modified. The tensor structure and design should be selected to be compatible with the input structure of the NN, and vice versa.
[0028] A tensor is established from the output of each imaging sensor, i.e., sensor_1, ... sensor_N. The tensor has multiple dimensions T(H, W, C, ...). For example, in a three-dimensional tensor T(H, W, C), H is the image height, W is the image width, and C is the total number of digital channels of all N imaging sensors. The digital channels contain digital image information about the detection area obtained from one or more of the aforementioned imaging methods, which may include, for example, a combination of the first material and the second material with any other background items (such as a clamping device or other parts of the workpiece that will eventually be distinguished). To this end, it is preferred that the detection area is aligned with the working area in which the material will be joined so that the operating environment is captured. In addition, if multiple imaging sensors are used, the imaging sensors can also be recorded, linked and processed simultaneously. For example, the first imaging sensor may include a camera, and the second imaging sensor may include a time-of-flight camera / sensor and a combination of their data.
[0029] In step 210, the tensor is input into a trained neural network (NN). The NN can be, for example, an artificial NN (ANN), a fully convolutional neural network (FCN), a visual transformer (ViT), and / or a spiking neural network (SNN). The NN extracts features from the digital image to classify each pixel into multiple categories or classes, and as described below, predicts a segmentation mask.
[0030] In step 212, the NN outputs a segmentation mask. The segmentation mask is determined by segmenting the digital image using a trained NN, the configuration and training of which will be discussed further below. An exemplary output of the segmentation process, a segmentation mask 400, is shown in FIG. Figure 4B, where the region of the segmentation mask 400 is segmented between target regions 402A, 402B (collectively “402”) and “separated” from background 404, resulting in a “mask” of the segmented region. That is, pixels are grouped by category, and pixels within the same category will be represented by the same color or value. Adjacent segments have different colors, intensities, or textures than other segments or categories. In addition, depending on the application, the generated segmentation mask can be further processed (post-processed). As described below, such post-processing can include, for example, improving the boundaries of the segmented regions, correcting small errors.
[0031] During the forward pass, e.g., steps 208, 210, and 212, the NN is fed with data in the form of input tensors from the image sensor, and an output tensor is produced again as a segmentation mask. The task of the NN is to find a mapping of the input tensor to the output tensor. In one example, the mapping is a series of stacked simple nonlinear transformations stored in the layers of the NN. The layers in turn transform the input tensor into a new representation, extract features, and make final decisions and predictions. The transformation is a function consisting of a matrix (weights), a vector (bias), and a nonlinear activation function.
[0032] During inference, e.g., steps 210 and 212, the trained NN may predict / generate an output tensor of new data representing the segmentation mask. For post-processing, the tensor may be processed channel-by-channel and / or converted back to a digital image. Regions of the same class (e.g., segment) may have the same color or value in the output tensor. Segmentation may be pixel-by-pixel classification.
[0033] In one configuration, the NN outputs the class probabilities or probability densities of pixels belonging to a particular class (segment), some pixels may also be misclassified. Classical image processing such as median filtering and morphological operators can also be used to help clean up the regions (turn off, delete, etc.) in the output image or in the corresponding channels of the output sensor. The images thus cleaned can be analyzed using connected components. The regions thus found can be further analyzed by determining and sorting them based on their size and shape as well as their contours and their respective properties (such as convexity).
[0034] Once this is done, for example, the predicted target areas 402 (e.g., hairpins) found can be counted, and if necessary, the missing hairpins can be notified to a higher-level programmable logic controller, image processing computer 116, and / or controller 118. If a certain number of expected predicted target areas 402 are located, for example, a hairpin pair is detected for the case of welding two hairpins together, the corresponding areas (e.g., target areas 402) and their properties are passed to the final algorithm, which determines the exact location of the weld.
[0035] Although Figure 4BOnly two segment types are shown (target region 402 and background 404), but additional segments may also be determined. For example, additional segments identifying workpiece holding devices or reference surfaces. One or more digital channels are assigned to each point of the segmentation mask, where each digital channel m=1 to M represents a different segment or region that the segmentation mask contains.
[0036] In some embodiments, the digital image is segmented by semantic segmentation, instance segmentation, or panoptic segmentation. Semantic segmentation detects the class to which each pixel belongs. The class can be the background or foreground of the segmentation mask.
[0037] Instance segmentation identifies the instance to which each pixel belongs to its neighboring parts. Instance segmentation detects each pair of distinct neighboring parts in a digital image. Panoptic segmentation combines semantic segmentation and instance segmentation. Panoptic segmentation identifies the class to which each pixel belongs and distinguishes between different instances of the same class.
[0038] The NN used in step 208 may reside in Figure 1 The NN may be in software or other machine instructions in an image processing computer 116, or reside in a remote computer, server, or cloud infrastructure communicatively connected to the image processing computer 116. Regardless of its location, the NN performs image segmentation of the digital image.
[0039] The NN may be formed of connected nodes (neurons) arranged in multiple layers, where the input to the NN is the first layer and the output is the last layer. In the above example, the input to the NN may include a digital image from a camera, such as numerical representations of pixels associated with the image (e.g., the input image tensor discussed above), and the output of the NN may include numerical representations of the segmented regions. Figure 4BA graphical representation of such output is shown, which may include individual pixel values representing a segmentation mask 400 segmentation mask 400. In a NN, each neuron transmits a signal to another neuron, which processes the signal according to its trained weights or biases and transmits the signal to another connected neuron. The signal is a real number, and the output of each neuron is calculated by a nonlinear function of the sum of its input signals. In some embodiments, neurons have weights that are adjusted as NN learning proceeds. The weight increases or decreases the strength of the signal at the connection, such as the output of any particular neuron, and thus increases or decreases its contribution to the sum. Neurons may have a threshold so that signals are sent only when the aggregate signal exceeds the threshold. Neurons can be aggregated into layers connected to previous or subsequent layers. In addition, the number of neurons in each layer closer to the output is more abstract than the previous layer, for example, with fewer neurons. Therefore, in one example, the first layer will include neurons for each input channel, such as digital pixel information from the input, and the output neurons (the last layer) will be reduced to the number of neurons required to represent the segmentation mask of the output.
[0040] The NN model architecture can be a convolutional network with an architecture similar to that described, for example, in Ronneberger et al., Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, Vol. 9351: 234--241, 2015 ("U-Net"); J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, doi: 10.1109 / CVPR.2015.7298965; and E. Shelhamer, J. Long, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, doi: 10.1109 / CVPR.2015.7298965. Segmentation,” IEEE Journal of Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, pp. 640-651, April 1, 2017, doi:10.1109 / TPAMI.2016.2572683 (“FCN”), the entire text of each of which is incorporated herein by reference.
[0041] The disclosed methods and systems may also include training the NN to specialize the NN for segmenting two materials to be joined. For example, training, otherwise known as "learning" or "deep learning," may be performed prior to starting method 200 to generate the NN used in step 208. In some embodiments, training may include utilizing a plurality of matched input photo sets (similar to Figure 4A ) and its corresponding segmentation mask generated previously (similar to Figure 4B ) as input and output (training set) of the NN architecture, so that the NN updates weights and biases associated with multiple neurons to minimize one or more errors associated with the training set, for example, the difference between a known segmentation mask and the output of the NN. The training set can be a supervised training set, in which a human reviews the input photo and manually creates a segmentation mask associated with the input photo. When the segmentation mask is manually created, the digital image from the manual segmentation mask will then be converted to an expected tensor (output tensor) with the same width and height as the digital image, where the number of channels is equal to the number of categories. Training is performed by adjusting the NN weights and biases to find a mapping from the input tensor to the output tensor to repeatedly minimize the error between the actual output tensor and the predicted output tensor.
[0042] Providing the NN with a sufficient number of training sets (the number of which will depend on the specific materials being joined and their orientation and installation environment) will result in a dedicated NN trained to be used in step 208 for the appropriate material to be joined.
[0043] In addition, the training matching set (i.e., training set, or training data) may include multiple data subsets. For example, if there are a sufficient number of training sets, the available data may be divided into one or more of a training subset, a validation data set, and a test data set. In one example, the optimizer may use the training data set to minimize the error associated with the training data set by adjusting its weights to correctly classify each pixel in the digital image (fit the model) as the initial training of the NN. The validation data set may be used to evaluate the model during training and to check whether the model is generalized according to the training data or whether it is just "memorized". For example, by comparing the output of the validation data set (generated by passing its input to the NN) with a known matching set segmentation mask. The goal is to train the model until the error associated with the validation data set reaches a minimum. When a large number of experiments (hyperparameter tuning) are performed with slightly different models, the validation data set may be overfitted. Therefore, the test data set is used to finally evaluate the model and check its generalization ability to reduce the possibility of overfitting the validation data set when a large number of experiments (hyperparameter tuning) are performed with slightly different models.
[0044] In one example, the NN is trained based on one or more sets of matched input photos and their corresponding segmentation masks previously generated. Or using unmatched inputs using automatic or manual feedback. In other examples, the NN is trained manually as described below.
[0045] The first step in training a NN may include acquiring a digital image from the coverage area.
[0046] The second step in training a NN is to label the digital image: determine the regions of interest (eg, joints) in the digital image, either manually or with the help of an annotation program.
[0047] The third step in training a NN may involve converting labeled digit images into tensors.
[0048] The fourth step of training the NN may include inputting the tensor into an untrained NN, which outputs a true segmentation mask created by a suitable encoding. The training of the NN produces a trained NN that transforms the coverage area into corresponding segments of the region of interest. Transforming the coverage area into corresponding segments includes taking each pixel (all unique) and subdividing them into fewer segments (m to M) based on the number of segments to be identified.
[0049] The trained NN transforms the unlabeled tensor into a segmentation mask, i.e., the output of the NN.
[0050] In the fifth step, the error between the true segmentation mask and the estimated segmentation mask is minimized by changing the parameters of the NN (e.g., the biases / weights of the nodes) using a training optimization process. For example, minimizing the error may include a calculus chain rule that tracks the error signal from the output to the input of the NN and calculates the corresponding gradient, and an optimization algorithm (e.g., gradient descent) that is used to iteratively improve the mapping.
[0051] As noted above, the disclosed embodiments are described with reference to joining hairpins as the first exemplary material 104 and the second exemplary material 106 . Figure 4B Shows Figure 4A A segmentation mask of the digital image is obtained as the output of the NN from step 208 . Figure 4B It is shown that the portion of the hairpin that is the target region 402 in the segmentation mask is well located by its shape and orientation. The end of the hairpin representing the foreground is clearly distinguished from the background 304. Identifying the target region 402 from the background 404 improves the accuracy of subsequent steps (e.g., step 212) discussed further below.
[0052] In step 214, the segmentation mask is used to determine the join point. The output of the NN is used as input to determine the join point, that is, the segmentation mask is used to determine where the join between the two materials will occur, which will be translated by the system 100 (e.g., the image processing computer 116 and / or the controller 118) into appropriate axis coordinates or other appropriate machine instructions, such as G-code, RS-274, etc.
[0053] The segmentation mask divides the coverage area into regions belonging to the first material and the second material for bonding, and / or other regions. Once the segmentation mask is output, it can be further evaluated to determine the desired bonding location. For example, if the centroid method is used, the regions belonging to the first material and the second material are evaluated by calculating the centroid of each region or the contour of each region.
[0054] The information about the centroids and contours of the individual areas is then processed to find exemptions for certain areas, for example cross sections of the first material and the second material, for calculating a corresponding center and / or a corresponding contour.
[0055] Based on the information obtained about the center of gravity and the contour of each area, certain process parameters such as the positioning of the laser beam can be adjusted. Specifically, these process parameters are based on the positioning of the laser beam based on the center of gravity, the center point and the descent along the contour. Other process parameters include the line energy to be introduced, the feed speed, the laser power, the beam deflection and the movement frequency.
[0056] In step 216, the first material and the second material are joined at the determined joining point. In some embodiments, the first material and the second material can be joined by laser welding. The laser welding can be performed using a laser welder.
[0057] In some embodiments, the first material and the second material are hairpins placed in the stator, and the joints of the hairpins are welded to form the stator winding, for example by performing steps 206, 208, 210, and 212 of method 200. After the joints are welded, a mechanical connection and / or an electrically conductive connection is formed between the parts. The welding of the hairpin parts next to each other forms a continuous stator winding or part thereof that is mechanically and electrically interconnected. The joints can be welded by gas welding, arc welding, resistance welding, energy beam welding, ultrasonic welding, solvent welding, gluing, etc.
[0058] For example, gas welding can include oxygen-fuel welding (oxyacetylene welding). For gas welding, acetylene burns in oxygen to produce a flame with a temperature of about 3100°C (5600°F) for gas welding. Arc welding can, for example, use a power supply to generate and maintain an arc between an electrode and the portion of the material forming the joint. The power supply uses direct current (DC) or alternating current (AC) and consumable or non-consumable electrodes. The welding area can be shielded with an inert or semi-inert gas, which is called a shielding gas. In addition, a filler material can be used.
[0059] Resistance welding may include, for example, welding in which the contact between two parts forms a resistor and current flows through the resistor to generate heat. A high current (1000 to 100,000A) is then passed through the resistor, forming a small pool of molten metal at the joint.
[0060] Energy beam welding may include, for example, laser beam welding and electron beam welding. Laser beam welding uses a highly focused laser beam. Electron beam welding uses an electron beam and is done in a vacuum.
[0061] Ultrasonic welding may involve, for example, joining materials by vibrating the materials at high frequencies and high pressures.
[0062] In some embodiments, the two materials (e.g., the first material 104 and the second material 106) are hairpins used to make a stator. FIG. 5 shows an example of adding two or more hairpins to a stator 502. The stator 502 is a fixed part of a generator, motor, siren, mud motor, or bio-rotor. The stator consists of an insulating material and a stator winding. To make the stator winding, hairpins are introduced into the stator to join adjacent conductors or winding conductors.
[0063] For example, the conductive hairpins are introduced parallel to each other into the holes of the stator so that parts of adjacent hairpins are placed adjacent to each other, as e.g. Figure 5A , Figure 5B and Figure 5C as shown in .
[0064] Figure 5A The stator 502 of the electric motor is shown. The stator 502 comprises a hole extending parallel to the axis of the stator 502 for inserting a hairpin 304. The hairpin 304 is shaped like a clamp comprising two parts. The hairpin 304 is made of a conductive material. The hairpin 304 is introduced into the hole of the stator 502 so that a part of the hairpin 304 protrudes outside the hole of the stator 502.
[0065] Figure 5B Shows Figure 5AThe stator 502 and the second portion 304B of the first hairpin and the first portion 306A of the second hairpin, wherein the first hairpin and the second hairpin are placed adjacent to each other. The portions 304B, 306A are thus placed next to each other. If the portions 304B, 306A are not in the intended engagement position, the portions can be physically manipulated or adjusted so that the cross-sections of the portions 304B, 306A are next to each other and, for example, in the same plane, as indicated by the arrows.
[0066] After alignment, such as by performing steps 206, 208, 210, and 212 of method 200, joints 308 are welded to form stator windings. After welding joints 308, mechanical and electrical connections are formed between segments 304A, 304B. Welding segments 304B, 306A of hairpin 304 next to each other forms a mechanically and electrically interconnected continuous stator winding or portion thereof.
[0067] The method steps in any embodiment described herein are not limited to being performed in any particular order. In addition, the structures mentioned in any method embodiment can utilize the structures mentioned in any device embodiment. Such structures may only be described in detail for the device embodiment, but are applicable to any method embodiment.
[0068] Unless the specific arrangements described herein are mutually exclusive, the various implementations described herein may be combined in whole or in part to enhance system functionality or generate supplementary functionality. Similarly, various aspects of the implementation may be implemented in independent arrangements. Therefore, the above description is given by way of example only, and modifications in detail may be made within the scope of the present invention.
[0069] With respect to the use of substantially any plural or singular terminology herein, those skilled in the art may translate the plural to the singular or from the singular to the plural, depending on the context or application. For clarity, various singular / plural arrangements may be expressly set forth herein. Unless otherwise specified, reference to an element in the singular is not intended to mean "one and only one," but rather "one or more." Furthermore, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recited in the above description.
[0070] In general, the terms used herein, especially in the appended claims (e.g., the body of the appended claims), are generally intended to be "open" terms (e.g., the term "including" should be interpreted as "including but not limited to", the term "having" should be interpreted as "having at least", the term "including" should be interpreted as "including but not limited to", etc.). In addition, in those cases where conventional terms like "at least one of A, B, and C, etc." are used, generally, such construction is intended to have the meaning understood by those skilled in the art for the conventional terms (e.g., "a system having at least one of A, B, and C" will include but is not limited to a system including only A, only B, only C, including A and B, including A and C, including B and C, or including A, B, and C, etc.). Similarly, whether in the specification, claims or drawings, a statement presenting two or more alternative terms should be understood to include one of these terms, any one of these terms, or both of these terms. For example, the statement "A or B" should be understood to include the possibility of "A" or "B" or "A and B".
[0071] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments should be considered to be illustrative in all respects only and not restrictive. The scope of the present invention is therefore indicated by the appended claims rather than by the foregoing description. All changes within the meaning and scope of the equivalents of the claims are included within the scope of the claims.
Claims
1. A method for joining materials, the method comprising: Two materials are provided; placing a first portion of a first material adjacent to a second portion of a second material; Acquire digital images of the first portion and the second portion by an imaging sensor; converting the digital image into a tensor, the tensor comprising at least a first dimension, a second dimension, and a third dimension, wherein the first dimension comprises a height of the digital image, the second dimension comprises a width of the digital image, and the third dimension comprises a number of digital channels of the imaging sensor, Inputting the tensor into a trained neural network (NN); The segmentation mask output by the NN, determining a junction point using the segmentation mask; as well as The first material and the second material are joined at the joining point. 2 . The method of claim 1 , wherein the first material and / or the second material comprises a metal.
3. The method of claim 2, wherein the joining comprises welding, brazing and / or soldering.
4. The method of claim 3, wherein the welding comprises gas welding, arc welding, resistance welding, energy beam welding or ultrasonic welding.
5. The method of claim 1, wherein the first material and / or the second material comprises a wire.
6. The method of claim 1, wherein the junctions are determined by thresholding, connected components, contours, morphological segmentation, or Gaussian mixtures.
7. The method of claim 6, wherein the thresholding comprises histogram shape-based thresholding, clustering-based thresholding, entropy-based thresholding, object attribute-based thresholding, and spatial thresholding.
8. The method of claim 1, wherein the digital image is segmented by at least one of semantic segmentation, panoptic segmentation, and instance segmentation.
9. The method of claim 1, wherein the joining point is determined by determining at least one of a distance (gap), a convexity, a boundary contour, and a centroid (center point) of the first portion and the second portion.
10. The method of claim 1, wherein the neural network is trained using deep learning.
11. The method of claim 1, wherein the training of the NN comprises training at least one of a convolutional NN, a fully convolutional neural network (FCN), a visual transformer (ViT), and a SNN.
12. The method of claim 1, wherein the training comprises reducing an error associated with a training set.
13. The method of claim 1, wherein the first material and the second material are electrically conductive hairpins, and the joining comprises hairpin welding for manufacturing a stator, wherein: Placing a first portion of the first material adjacent to a second portion of the second material includes introducing the hairpins into the stator such that portions of adjacent hairpins are placed adjacent to each other; Determining the joining point includes determining the shape and orientation of the portions placed adjacent to each other according to the segmentation mask; and Joining the first material and the second material includes welding the join to form a stator winding from the welded hairpins.
14. The method of claim 13, wherein the digital image is acquired from a cross section of the portions placed adjacent to each other.
15. The method of claim 1, further comprising preprocessing the digital image by normalizing, cropping, and / or scaling the digital image.
16. The method of claim 1, wherein the NN is an artificial NN or a spiking neural network.
17. A system for joining materials, the system comprising: a holder adapted to hold at least a first material and a second material such that a first portion of the first material is positioned adjacent to a second portion of the second material; a camera adapted to acquire digital images of the first portion and the second portion; an image processing computer adapted to process the digital image into a segmentation mask using a neural network (NN) and determine the junction points of the material; as well as A joining device is adapted to join the first material and the second material at the joining point.
18. The system of claim 17, wherein the joining device is a welding device, a brazing device, or a soldering device.
19. The system of claim 17, wherein the welding device is a gas welding device, an arc welding device, a resistance welding device, an energy beam welding device, or an ultrasonic welding device.
20. The system of claim 17, wherein The holder is a stator having holes that hold the conductive hairpins parallel to each other so that portions of adjacent hairpins are placed adjacent to each other, and The joining device is a hairpin welding device.
