Image processing device to support a qualitative and / or quantitative assessment of the quality of a crimp connection, image evaluation device and production release system for a crimp device

Description

Technical field

[0001] The invention relates to an image processing device for supporting a qualitative and / or quantitative evaluation of the quality of a crimp connection. Furthermore, the invention relates to an image evaluation device for the qualitative and / or quantitative evaluation of the quality of a crimp connection, as well as a production release system for a crimping device for manufacturing a crimp connection. State of the art

[0002] In cable manufacturing, it is necessary, especially before or during the processing of a production order, to ensure that a crimp connection produced accordingly for a cable, for example a data cable, is free of defects.

[0003] For this purpose, a so-called micrograph of the manufactured crimp connection is produced before or during the production of a manufacturing order. Typically, the crimp connection is cut perpendicular to its longitudinal direction, ground, and the cross-section of the cut crimp connection is examined with regard to specific quality parameters. The micrograph is therefore a cross-sectional image of the crimp connection.

[0004] Micrographs help in the development of a crimp connection by determining the crimp dimensions and checking the crimp quality of crimping tools as part of crimping devices.

[0005] For this purpose, a cross-section of the crimp connection is recorded, typically in digital and magnified form, for example using a microscope, in order to examine it for the presence of crimp defects.

[0006] It is known that classic image analysis algorithms are used to evaluate such cross-sectional images of a crimp connection in order to determine corresponding crimping defects or quality parameters of the crimp connection. Typically, the results of the classic image analysis algorithms are checked manually, and inaccurate results from the image analysis software are corrected manually.

[0007] Alternatively, the crimp quality of crimp connections can be checked entirely manually based on the captured cross-sectional images of the crimp connection.

[0008] US 2021295487 A1 discloses a method for assessing the crimp condition of a crimp connection in a wiring harness. An image of a portion of the crimp connection is captured. From this image, initial data related to a void in that portion of the crimp connection is determined. The crimp condition of that portion is then calculated from this initial data. Cross-sectional images of a crimp connection are used for this purpose; see [reference]. Fig 2 , Fig 4 , Fig 6 .

[0009] CN 111 665 267 A discloses a visual detection method for the crimp quality of a contact piece and solves the problems that the crimp quality of an existing contact body and wire cannot be visually and thoroughly detected and that crimp quality problems often remain hidden.

[0010] EP 3 109 624 A1 discloses a pipe inspection system. The pipe inspection system comprises a mirror arrangement with an odd number of sides, arranged to form a pyramid-like structure surrounding a pipe segment. A camera captures a multitude of images of the pipe segment reflected by the mirrors, each image showing a different side of the pipe segment.

[0011] US 2023 245299 A1 discloses a terminal inspection system for a crimping machine. The terminal inspection system comprises an image processing device that captures a terminal to be inspected and generates a digital image of the terminal. The terminal inspection system further comprises a terminal inspection module that communicates with the image processing device to receive the digital image of the terminal as an input image. The terminal inspection module has a reference image. The terminal inspection module compares the input image with the reference image and performs semantic segmentation between the input image and the reference image to generate an output image. The output image shows differences between the input image and the reference image to identify potential defects.

[0012] EP 2 173 015 A1 discloses a method for determining the quality of a crimp connection between a conductor and a contact, in which a crimping force is first applied to the conductor and the contact using a crimping device. A normalized force-displacement crimp force curve is derived from the crimp force curve generated during crimping, and a compression area is determined that lies below a reference crimp force curve. The crimp force curve and the reference crimp force curve are divided into several zones, the division taking into account the size of the compression area. A further area lying below the crimp force curve is determined and used to infer the quality of the crimp connection.

[0013] US 7 174 324 B2 discloses a system in which estimation units that have learned in advance a relationship between known connection data belonging to the connection setup and unknown connection data belonging to the connection setup for the known connection data, calculate the unknown connection data for the known connection data according to an input of the known connection data based on the learning result.

[0014] "Deep learning-based automated optical inspection system for crimp connections" by Giang Nguyen Huong et al. (XP033892793) discloses a computer vision system for automating the final inspection of crimp connections. The image processing chain and the deep learning-based model for analyzing image data of crimp connections are described with respect to various defect classes.

[0015] Regarding the required documentation of corresponding results of the examination of the crimp connection within the framework of quality assurance, this is usually created manually and occasionally even handwritten and stored and archived in a designated database.

[0016] The current method for determining the quality of crimp connections is costly because it is labor-intensive, and often leads to non-reproducible results due to the manual and individual handling by the testing personnel when determining the quality parameters, especially quantitative quality parameters, of the crimp connection.

[0017] Furthermore, the manual documentation of the evaluation of the micrographs makes it more difficult to trace and find already archived results of the micrograph evaluation, as these are usually not machine-readable and therefore not searchable. Description of the invention

[0018] The object of the invention is to provide a solution by which the determination of quality parameters of crimp connections can be carried out in a more robust and reliable manner, and by which the production of corresponding crimp connections can be made more reliable and less costly.

[0019] The problem is solved by the subject matter of the independent claims. Advantageous embodiments of the invention are specified in the dependent claims, the description, and the accompanying figures. In particular, the independent claims of one claim category may also be further developed analogously to the dependent claims of another claim category. Further embodiments and developments will become apparent from the dependent claims and from the description with reference to the figures.

[0020] The invention particularly comprises an image processing device for supporting a qualitative and / or quantitative evaluation of the quality of a crimp connection, with a receiving unit for receiving a cross-sectional representation of an image, in particular designed as a digital microscope image in the visible spectral range, of a crimp connection, with a processing unit which is designed to generate a raster image of the received image from the received image by means of a deep neural network, in that at least the pixels of a relevant image area of ​​the received image can be assigned to a predetermined class by means of the trained deep neural network, in particular each pixel of the received image can be assigned to a predetermined class.Furthermore, the processing unit is designed to generate at least one vector contour from the generated raster image and to generate and output a signal based on the determined vector contour, from which at least one qualitative and / or quantitative quality parameter attributable to the crimp connection can be determined.

[0021] Such an image processing device provides a robust and reliable way to process the cross-sectional image of the crimp connection in such a way that a reliable, objective, reproducible evaluation of the image with regard to qualitative and / or quantitative quality parameters can be carried out with high reliability.

[0022] In particular, such an image processing device makes it possible to handle defects during the image acquisition step in such a way that they do not negatively affect subsequent image evaluation. For example, this can compensate for an inaccurately adjusted focus, unfavorable lighting conditions, reflections on the section of interest in the cross-sectional image, or contamination on the polished surface, etc.

[0023] In particular, these defects can include contamination of the cross-section of the crimp connection, polished edges or unusual burrs, which can then lead to errors in the evaluation of the image with classic image evaluation algorithms.

[0024] Classical image evaluation algorithms are understood to be image evaluation methods that perform image evaluation independently of machine-learned models.

[0025] Such errors in the micrograph occur particularly in production-related environments. This leads to limitations in the automation and accuracy of determining the quality parameters of crimp connections using classic image evaluation algorithms in manufacturing environments.

[0026] This disadvantage can be eliminated by using a suitable image processing device, as this is robust against corresponding disturbances in image acquisition and therefore enables reliable automation.

[0027] Such an image processing device therefore also allows for a cost-effective, reliable and standardized approach with regard to any image evaluation that is to be carried out.

[0028] Assigning at least the pixels of a relevant area of ​​the received image to a predefined class, and in particular assigning each pixel of the received image to a predefined class, can also be described as semantic segmentation of the received image. The different classes thus form a semantically segmented raster image of the crimp connection, typically a schematic representation of the crimp connection's cross-section.

[0029] By means of such semantic segmentation, at least of the relevant image area of ​​the cross-sectional image of the crimp connection, in particular of the entire cross-sectional image, defects present in the original image that have a detrimental effect on evaluation using conventional evaluation algorithms can be eliminated.

[0030] The relevant image area is that part of the cross-sectional image of the crimp connection in which the crimp connection is depicted. The surrounding area within a predefined minimum distance around the outer boundary of the crimp connection does not necessarily require image processing and / or evaluation, as it is generally not relevant to the quality parameters of the crimp connection.

[0031] Preferably, the cross-sectional image is digitally captured, for example using a CCD camera behind the eyepiece of a microscope to generate images in a spectral range visible to the human eye. It can be advantageous to enlarge the imaging area of ​​the device so that the cross-section of the crimp connection is completely imaged and simultaneously provides a sufficiently high magnification to analyze the structure of the cross-section.

[0032] Both qualitative and quantitative quality parameters can be considered. A qualitative quality parameter might, for example, state whether the crimp connection or a specific quality parameter is acceptable or not, i.e., defective or flawless according to the manufacturing specifications. Quantitative quality parameters include comprehensive quantitative values ​​that measurably characterize the crimp connection, such as crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, base thickness, voids between strands, and cracks. A quality parameter of the crimp connection can be determined both qualitatively and quantitatively.

[0033] The evaluation process is supported by a suitably designed image processing device. The evaluation process can, for example, still be carried out using classic image evaluation algorithms based on the processed micrograph of the crimp connection, in particular based on the at least one determined vector contour.

[0034] The image processing device comprises a receiving unit for receiving a cross-sectional image of the crimp connection, in particular a photographic cross-sectional image of the crimp connection in digital form. This cross-sectional image advantageously, but not necessarily, shows the entire cross-section of the crimp connection, particularly at its maximum size within the scope of the image acquisition device used. The image can be captured in such a way that the outer radial boundary of the crimp sleeve is completely depicted.

[0035] The image acquisition device can optionally be included in the image processing device. The image acquisition device can, for example, be a microscope installed close to the production area. The cross-sectional images of the crimp connection captured by the image acquisition device can be fed to the receiving unit of the image processing device. The image processing device is preferably arranged in a monitoring area and connected to the image acquisition device so that an image captured by the image acquisition device can be fed to the receiving unit.

[0036] The received cross-sectional image of the crimp connection can be further processed, in particular enhanced, by means of a processing unit. For this purpose, the processing unit comprises a trained deep neural network with which a semantic segmentation of the received image can be performed. This means that, using this trained deep neural network, a class can be assigned to at least the pixels of the relevant image area, i.e., the area showing the cross-section of the crimp connection, and in particular to each pixel of the received cross-sectional image of the crimp connection.

[0037] In particular, such semantic segmentation allows the relevant image content of the cross-sectional image of the crimp connection to be identified with regard to quality parameters. The classes into which the pixels can be assigned depend on the training data and the training of the deep neural network.

[0038] In this context, all deep neural network topologies suitable for performing semantic segmentation of an image can be used.

[0039] The segmented image can then be converted from a raster image into a vector contour using a specially designed processing unit. A vector contour is a graphically representable structure defined by mathematical expressions of lines, curves, and shapes. Unlike a raster image, which consists of pixels and can lose quality when enlarged, vector graphics retain their clarity and quality at any size change due to the mathematical expressions that describe the relationship between different points, lines, and curves of the representable structure. This allows for increased accuracy in subsequent analysis, particularly when determining at least one actual value for quantitative quality parameters.

[0040] Converting the raster image into a vector contour is particularly advantageous with regard to the accuracy of subsequent analysis. This means that the subsequent analysis is no longer dependent on the pixel count of the received image, thus improving the accuracy in determining quality parameters, especially quantitative ones. The vector contour can, for example, be solid and / or linear.

[0041] The processing unit is designed to generate and output a signal based on the vector contour, from which quality parameters attributable to the crimp connection can be determined, both qualitatively and / or quantitatively. This can be, in particular, the generated vector contour itself, which is then available for further image analysis, for example, using classic image analysis algorithms or manually. Specifically, the vector contour can be output as a graphical representation.

[0042] The conversion of the resulting raster image with at least three classes into a vector contour can be performed conventionally, i.e., without using a trained deep neural network. Appropriate procedures for this are known to those skilled in the art. Similarly, an output signal can also be generated and output outside of the trained deep neural network.

[0043] Alternatively, the trained deep neural network can be configured to generate the vector contour from the raster image. Furthermore, the trained deep neural network can also be configured to generate and output the output signal.

[0044] The output signal can be designed in such a way that it can be processed by a technical device to which the output signal can be fed. In particular, the output signal can be designed in such a way that the result of the image processing can be reproduced on an image display device.

[0045] However, the output signal can also be provided in such a way that the output signal, from which at least one qualitative and / or quantitative quality parameter can be determined, can be fed to an evaluation unit and an automated determination of at least one qualitative and / or quantitative quality parameter can be carried out by means of the evaluation unit.

[0046] In another embodiment, the trained deep neural network comprises an image transformer network, also known as a vision transformer or viT, and / or a convolutional neural network, also known as a CNN. These are particularly suitable deep neural networks for the intended image processing.

[0047] In particular, the trained deep neural network can be configured as an image transformer network or a convolutional network. Using an image transformer network is especially advantageous because it requires significantly less training effort compared to other suitable network topologies. Image transformer networks are encoder-decoder networks suitable for classifying images.

[0048] The trained deep neural network, in particular the deep convolutional network, is trained using appropriate training datasets and trained for the semantic segmentation of a cross-sectional image of a crimp connection.

[0049] As part of such training, the trainable neural network can be presented with appropriate annotated training data, which contains the desired classes that should later be recognized by the trained network.

[0050] Such a training dataset can be created manually, for example, from existing historical data, such as error-free and error-prone cross-sectional images from past production orders. These cross-sectional images of crimp connections are readily available because cable production is routinely monitored, and therefore cross-sectional images are taken at regular intervals, making the data available for quality assurance purposes over an extended period.

[0051] The following section describes, using an example, the training of a deep neural convolutional network. It should be understood that the creation of the training data and the training itself are separate processes.

[0052] A corresponding set of training images is generated for training the deep neural network. These images include a cross-sectional view of the crimp connection and a desired pixel class assignment.

[0053] For example, all pixels showing an inner conductor are assigned to one class, all pixels showing the crimp sleeve are assigned to another class, and all pixels lying radially outside the crimp sleeve are assigned to yet another additional class.

[0054] The training data includes not only fault-free crimp connections, but also faulty crimp connections, so that the pixels can be assigned to the classes regardless of their relative arrangement on the cross-sectional image.

[0055] The acquisition and qualification of relevant images can be performed during normal production or testing operations, allowing for the generation of numerous images with varying crimping, size, viewing direction, rotation, contrast, lighting, occlusion, etc., which can then be supplemented with the classes typically determined offline. This image qualification can be performed manually or at least partially by a conventional image processing system, with the option of manual rework. The pre-qualified training images can then be further pre-processed for training purposes.

[0056] For example, during further preprocessing, the size of the training images can be adjusted to a predefined size. Specifically, the number of pixels in the training images can be adjusted to match the number of inputs to an input layer of the neural network. Preprocessing the training images can also include normalizing them.

[0057] In one embodiment, the aforementioned preparation and preprocessing of the images does not take place, so that the trained neural network can handle corresponding raw data, which increases the subsequent speed of semantic segmentation using the trained deep neural network.

[0058] To improve the results of the neural network and make it more robust, the training images can be subjected to random image manipulations. Such manipulations can include, for example, rotation, enlargement, reduction, and / or distortion. It goes without saying that appropriate magnitudes can be specified for each image manipulation.

[0059] For example, a maximum enlargement or reduction can be specified as a percentage, such as 110% or 90%. For rotation, a maximum or minimum rotation angle can be specified, such as + / - 10°, 20°, or 30°. Similarly, corresponding limits can be defined for distortion. It goes without saying that different algorithms can be used for image distortion, each with different parameters.

[0060] The image manipulations serve to introduce greater variability into the training data. Consequently, the intended classes will be present in different locations and sizes within the image. This prevents, for example, the neural network from learning to identify the specified feature only in a small section of an image and incorrectly concluding that the feature is not present, even though it is simply located outside that section.

[0061] After preprocessing the training images, the neural network is trained using a portion of these images. The remaining training images can be used to verify the learning progress by feeding them back into the trained neural network and comparing its output to the known or expected output for that particular training image. This portion of the training images can therefore also be referred to as test data.

[0062] After the training is complete, its quality can be verified, as mentioned above, by qualifying the test data with the trained neural network. If the results meet the desired quality, the training can be terminated. If the results are of the desired quality, or if the neural network is to be further developed, the training can be continued with appropriate training data or repeated with modified parameters.

[0063] It goes without saying that the training can be carried out differently depending on the type of neural network used.

[0064] Typically, the weights of the neural network are adjusted during each training iteration to minimize the error in the network's output compared to the known result from the training dataset. This is usually achieved through a process called backpropagation, error feedback, or backpropagation.

[0065] For training, a so-called epoch number and a termination criterion can also be specified. The epoch number indicates the number of training iterations. For each training iteration, a predetermined number of training data points can be used, for example, all training data or only a selection of the training data. The termination criterion specifies how far the result of the trainable neural network may deviate from the ideal result in order to consider the training successfully completed, and thus the neural network sufficiently trained.

[0066] Convolutional neural networks, especially deep convolutional neural networks (dCNNs), deliver good to very good results, particularly for classifying objects in image data. It goes without saying that other suitable neural networks from the field of machine learning are also possible.

[0067] Such a dCNN can have an input layer, a multitude of hidden layers, and an output layer. The hidden layers can be at least partially identical or repeating.

[0068] The input layer can have one input for each pixel of the captured images. The images can be transmitted to the input layer, for example, as an array or vector with the corresponding number of elements. Furthermore, the size of the captured images can be the same for all images. For example, the cross-sectional images can be captured with 1024 x 1024 pixels, covering a capture area of ​​2 cm x 2 cm, 1 cm x 1 cm, or 2.5 cm x 2.5 cm.

[0069] The detection area can be selected depending on the crimp connection being analyzed. The detection area is preferably selected such that the entire cross-section of the crimp connection is captured.

[0070] It goes without saying that these specifications are merely examples and other image parameters can be used.

[0071] For semantic image segmentation, there are several simple convolutional network architectures that serve as a basis for more sophisticated models or are suitable for simpler use cases.

[0072] For example, a so-called Fully Convolutional Network, abbreviated FCN, can be used, which was developed for semantic segmentation. It consists of a series of convolutional layers that have been modified for image classification to generate pixel-accurate output.

[0073] Unlike traditional convolutional neural networks (CNNs), which are typically used for classifying entire images, a free-form neural network (FCN) replaces the fully connected layers at the end of the network with convolutional operations to produce pixel-accurate output. The FCN further uses upsampling operations to scale the output back to the original image size. FCNs are trainable to integrate information from different scales, allowing them to incorporate both contextual and detailed information for accurate segmentation. This can be achieved through different layers or modules operating at different scales.

[0074] The output layer of an FCN generates a probability distribution for each class for each pixel in the input image. This output represents the predicted classes for each pixel, thus enabling pixel-accurate segmentation.

[0075] FCNs are flexible and can be trained for various input image sizes. They are effective for semantic segmentation because they are able to generate precise pixel-to-pixel mappings for different classes or objects within an image.

[0076] Advantageously, an image transformer network can be used to perform the semantic segmentation of the cross-sectional image of the crimp connection.

[0077] In this application, the term "image transformer network" refers to neural networks based on the transformer architecture, i.e., those comprising an encoder, a decoder, or both. These are also frequently referred to as vision transformers (ViT). These are transformer networks specifically designed for image processing.

[0078] An image transformer network comprises, for example, the following components. First, the network can include a patch processing unit, specifically a patch embedding unit. A patch is understood to be a partial image of the image to be processed. The patch processing unit divides the received image into patches, i.e., sub-images or partial images, and treats each patch as a token that serves as input for the subsequent transformer. In particular, a vector representation of the input data can be provided. Furthermore, the patch processing unit retains the positional information of the partial images.

[0079] The image, or partial image, is thus converted into a numerical representation. These patches are then considered a sequence of tokens, similar to words in a sentence in natural language processing. Typically, the patches or partial images overlap to better capture the local and global relationship between image features.

[0080] Furthermore, the image transformer network typically comprises multiple transformer blocks. These include so-called attention mechanisms, such as self-attention, attention matrices, or mechanisms for modeling the relationships between the tokens or patches. These transformer blocks process the patch sequences to extract global and local features from the image and to model relationships between the patches.

[0081] In self-attention, the relationship between each patch or token and other patches or tokens is recorded and made available to the model.

[0082] For each patch or token, the attention mechanism calculates a weighting or attention distribution across all other tokens or patches in the image. This weighting indicates how relevant the other parts of the image are to the current token or patch.

[0083] Attention matrices describe how each token or patch interacts with other tokens or patches. These attention matrices indicate which parts of the image are considered more important and which are considered less important.

[0084] By capturing these relationships between the tokens in the form of self-attention and attention matrices, the model can incorporate contextual information and capture relevant visual relationships within the image. This allows for the effective processing of both global and local contextual information.

[0085] The transformer blocks process the patch sequences to extract global and local features based on the attention mechanisms described above and to model relationships between the patches. Furthermore, these attention mechanisms are combined with feedforward layers to update and improve the representation of the patches.

[0086] Furthermore, an image transformer network can typically be designed to preserve the spatial position information of the patch in the image, for example by inserting position encodings or other mechanisms, especially in the area of ​​patch embedding, so that the statements provided by the image transformer network can be reproduced in a positionally correct manner.

[0087] One advantage of such an image transformer network is its good scalability, which allows it to be applied to images of varying sizes. Unlike many CNN-based approaches, this network does not require fixed input parameters. Through its transformer mechanism, the image transformer network can capture and utilize global contextual information across the entire image.

[0088] Furthermore, an image transformer network can be easily adapted to various tasks and configured for different applications such as classification, object detection, image segmentation, and more. This can be achieved in particular through a multi-layer perceptron encompassed by the image transformer network and placed downstream of the encoder.

[0089] In particular, an image transformer network can be trained in two phases. First, pre-training can be performed based on a large dataset to learn the task of semantic segmentation. Second, the image transformer network can be fine-tuned, for example, by training it with application-specific data provided by the user.

[0090] Furthermore, the image transformer network allows for parallel processing of data, which reduces the resources required for inference.

[0091] In particular, an image transformer network configured as a SegFormer network can be used for the semantic segmentation of the cross-sectional images of the crimp connection. However, other transformer architectures can also be used for semantic segmentation, such as the Detection Transformer (DETR), the Vision and Language Bidirectional Encoder Representations from Transformers (VilBERT), and others.

[0092] The training of an image transformer network, especially a SegFormer network, can be done analogously to the procedure described above for training a CNN.

[0093] In one exemplary embodiment, the SegFormer architecture can be selected as follows to achieve a robust and reliable result with regard to image processing. However, the architecture described below is merely one possible configuration, and it is understood that it is not binding on those skilled in the art.

[0094] The SegFormer network, as an image transformer network, comprises an encoder side and a decoder side as a transformer network.

[0095] On the encoder side, for example, four transformer blocks connected in series can be provided, with each transformer block preceded by a patch processing unit.

[0096] Each transformer block has an output through which data, especially image data or representations of image data, is provided that has been processed in the respective transformer block. The first transformer block in the series processes the image in relatively few patches; that is, the output image is divided into a comparatively small number of patches, usually overlapping, for example, 4 patches (2x2).

[0097] Each patch is processed within the transformer block according to the attention mechanisms described above and fed into a feedforward network. In a further step, new image data is assembled from this, for example using overlap patch merging, and made available at the output of the first transformer block.

[0098] This initial image data from the first transformer block is then provided to the subsequent unit for patch processing, which in turn provides the subsequent, second transformer block in the series with a larger number of patches, usually overlapping, for example, 16 patches (4x4). These are then processed in the second transformer block and made available at its output.

[0099] The image data at the output of the second transformer block of the four series-connected transformer blocks is provided to the subsequent unit for patch processing, which then provides the following, third transformer block with, for example, 64 patches (8x8), usually overlapping. These are processed in the third transformer block and made available at its output.

[0100] The image data at the output of the third transformer block of the four transformer blocks connected in series is then provided to the subsequent unit for patch processing, which then provides the subsequent, fourth transformer block with, for example, 256 patches (16x16), usually overlapping, which are processed in the fourth transformer block and made available at the output of the fourth transformer block.

[0101] The image data at the output of the fourth transformer block is then provided to a subsequent unit for patch processing and subsequently fed to a multi-layer perceptron (MLP). Furthermore, all output data from each transformer block, for example, all four transformer blocks, is also fed to the multi-layer perceptron.

[0102] A multi-layer perceptron (MLP) refers to a specific type of network layer used in the SegFormer architecture. A multi-layer perceptron is a simple form of neural network consisting of at least three layers, though in typical use cases it usually includes more than three.

[0103] There is an input layer to which data, for example from the transformer blocks, is fed. This layer represents the input characteristics and passes them on to the next layer, which is at least one hidden layer.

[0104] Typically, a plurality of hidden layers, but at least one hidden layer, are located downstream of the input layer. This at least one hidden layer is called "hidden" because it lies between the input and output layers and is not directly visible from the outside. A multi-layer perceptron can contain several hidden layers, in particular a large number of hidden layers. The weights generated during training process the data fed to the at least one hidden layer in the desired manner.

[0105] Furthermore, there is an output layer of the multi-layer perceptron, into which the processed data is output and usually further processed in the decoder area.

[0106] The decoder section of the SegFormer network typically has the task of scaling up the determined relationships to the original image size using the preceding blocks and modules, also called upsampling, in order to obtain segmentation results that are as pixel-accurate as possible.

[0107] In a SegFormer, the decoder section includes upsampling layers and corresponding transformations. For example, bilinear upsampling can be used to scale the results to the desired size. This produces a smooth upscaling of the predictions.

[0108] The decoder integrates upscaling with local and global information captured in the encoder domain by the transformer blocks. In this example, skip connections are also used to incorporate detailed features from earlier layers into the processing by the multi-layer perceptron and the decoder process. The decoder can further include attention mechanisms that are applied to the data being processed.

[0109] The output of the decoder of the image transformer network are the reconstructed segmentation masks, which represent the predicted class or category for each pixel in the image.

[0110] The decoder, like the encoder, can be implemented in various ways. Examples of decoder implementations can be found on relevant platforms, such as huggingface.co or GitHub.

[0111] However, the decoder's function is typically designed to reduce the upscaled results or predictions to the original image size and to deliver detailed, pixel-accurate segmentation results.

[0112] In a further embodiment, the deep neural network is designed as an image transformer network, wherein the image transformer network comprises a publicly available pre-trained image transformer network and at least one network layer, in particular an additional network layer, especially one subsequently added by the user, is present, which is trained, in particular subsequently, with application-specific training data for image processing of a cross-sectional image of a crimp connection.

[0113] Application-specific training data refers to data provided by the specific application for which the image transformer network is to be used; in this case, for example, the application of segmenting and / or evaluating cross-sectional images of crimp connections. Suitable pre-trained image transformer networks for semantic segmentation are available, for example, via huggingface.co or GitHub.

[0114] Pre-trained image transformer networks have the advantage of being easily adaptable, requiring only minimal additional training to customize them for a specific task or application. Therefore, it is possible to utilize publicly available pre-trained image transformer networks and easily adapt them to the specific application afterward.

[0115] Subsequent adaptation of the pre-trained model to the specific application can be achieved by adding at least one new layer or modifying at least one existing layer of the model through appropriate training. Training with the application-specific data then leads to a corresponding adaptation of the model based on the training data, resulting in improved outcomes.

[0116] Firstly, pre-trained image transformer networks can generally be easily adapted to the specific application of cross-sectional images of crimp connections.

[0117] Furthermore, the characteristics of a crimp connection can vary from crimping device to crimping device. Therefore, application-specific data may be crimping device-specific or crimping tool-specific. This means that, due to the minimal training effort required, the image transformer network can be trained separately for each crimping device or crimping tool and the crimp connection produced with it. This provides even further improved results with regard to image processing and subsequent image analysis.

[0118] In a further advantageous embodiment of the image processing device, the trained deep neural network is configured to generate a raster image with at least three classes from the received image, wherein a first class corresponds to an inner component of the crimp connection, in particular a conductor element, a second class corresponds to an outer component of the crimp connection, in particular a crimp sleeve, and a third class corresponds to the environment of the crimp connection.

[0119] It has been shown that three classes are sufficient for robust and reliable image processing. However, additional classes can also be provided within the framework of image segmentation.

[0120] The first class, corresponding to an internal component of the crimp connection, does not necessarily have to be a conductor element, for example in the form of strands or a solid inner conductor, but can also be a more complex configuration. For example, in the case of a sheath crimp, this could be the cable configuration enclosed by the crimp sleeve, including the cable sheath.

[0121] The second of the at least three classes can correspond to the outer component of the crimp connection. In particular, this class can be selected to correspond to the crimp sleeve of the crimp connection. Specifically, it can be a crimp sleeve for a sheath crimp or for an inner conductor crimp.

[0122] The third of the at least three classes can correspond to the environment of the crimp connection. This environment is not part of the crimp connection itself. The environment can be air or include other mechanical components, such as components that guide the crimp connection during image acquisition or other cable structures that are not intended for analysis.

[0123] By including such three classes during image processing, subsequent image analysis can be significantly improved. In particular, the precise classification into internal component of the crimp connection, crimp sleeve, and their demarcation from the surroundings provides a robust, reliable, and accurate basis for subsequent analysis.

[0124] At least three classes can be assigned corresponding, distinct values. These can be any distinguishable range of values, with each range being assigned to one class. The respective range of values ​​can also be a single value for a class. The assignment of a pixel to a corresponding value can be probability-based. A pixel is assigned the value that most likely describes it.

[0125] In one embodiment of the image processing device, the trained deep neural network is configured to assign at least one color to each of the first, second and third classes, wherein adjacent classes in the raster image and / or vector contour can be reproduced as, in particular, contrasting color areas of different colors, and this color assignment is included in the output signal.

[0126] This embodiment provides a suitable basis for both manual evaluation and conventional image evaluation software to determine at least one qualitative and / or quantitative quality parameter of the crimp connection.

[0127] By means of the different color schemes of at least the first, second and third classes, the contours and extents of the respective areas of the crimp connection, i.e. for example inner component, outer component and surroundings of the crimp connection, can be quickly and unambiguously recorded by a manual evaluator and / or image evaluation software, which significantly increases the reliability of an evaluation.

[0128] This is particularly easy when adjacent classes have colors that provide a sufficiently good contrast, making them easily distinguishable for the image evaluation software and / or the manual evaluator.

[0129] In particular, this makes it easy to recognize the boundary contours of adjacent color areas, i.e., the boundary line between first class and second class or between second and third class.

[0130] In particular, exactly three classes can be provided, comprising the first, second and third grades.

[0131] The output signal can be designed to display the cross-sectional image of the crimp connection as a three-color image with different colors. This provides a particularly simple and clear representation of the crimp connection's cross-sectional image, enabling reliable assessment of its quality parameters, either manually or using image analysis software.

[0132] In a further advantageous embodiment, the trained deep neural network is configured to determine the boundary contour between the first and second classes and / or between the second and third classes. The boundary contour can, in particular, be designed as a line, especially with a predefinable thickness, which encompasses or exhibits the boundary area of ​​adjacent classes or colors. The trained deep neural network can furthermore be configured to generate an output signal by means of which the determined boundary contour can be graphically represented.

[0133] For example, at least one boundary contour can be approximated as a polygonal path during evaluation using classical image algorithms, and qualitative and / or quantitative quality parameters of the crimp connection can be determined on this basis.

[0134] Training such a network to determine boundary contours can be carried out, for example, using cross-sectional images of crimp connections into which the corresponding boundary contours have been added, for example, manually. The training data can be augmented and varied in the usual way, especially as described above, to improve the network training.

[0135] The image processing device can be designed in particular to identify at least one point of the boundary contour, on the basis of which a determination of a quality parameter, in particular a quantitative one, is to be carried out, for example for the determination of at least one of the following quality parameters: crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance crimp flank ends, burr height, burr width, bottom thickness, voids between strands, cracks.

[0136] Such points are called marker points because they mark the reference points for any subsequent measurement of quality parameters. Marker points can also be provided to specifically highlight defective structures in the cross-sectional view of the crimp connection, such as voids between strands. If such voids were present, they would be expected, for example, in the inner area of ​​the crimp connection, corresponding to the first class.

[0137] These marker points can therefore identify those points of the boundary contour that are required for determining the desired quality parameters, and thus in particular for the quantitative evaluation of the cross-sectional image.

[0138] The marker points can be determined, for example, using simple logic operations, classical image evaluation algorithms, or a deep neural network trained for this purpose.

[0139] The boundary contour, in particular the line representing the boundary contour, can preferably be superimposed on the received cross-sectional image of the crimp connection. This provides a means of verification by making the boundary contour visible in the received cross-sectional image of the crimp connection. Based on the boundary contour, quantitative and qualitative quality parameters of the crimp connection can be determined. In particular, the relevant marker points for the respective quality parameters of the crimp connection can also be displayed.

[0140] In a further embodiment of the image processing device, the output signal is configured to include a superimposed representation based on the received image and the vector contour, in particular at least one boundary contour, wherein the measurement points used to determine the respective quality parameter are included as separately visible marker points within the superimposed representation. This makes it particularly easy to perform an evaluation based on this representation. In particular, the marker points can be designed to be distinguishable, for example, by different colors, so that they can be directly assigned to a specific, especially quantitative, quality parameter. This facilitates a reliable and virtually error-free evaluation. The marker points can be determined from the vector contour using a corresponding logic operation.

[0141] The invention further comprises, in particular, an image evaluation device for the qualitative and / or quantitative assessment of the quality of a crimp connection. This device comprises an image processing device according to any one of claims 1 to 6, wherein, based on the generated vector contour, a defect classification can be performed to determine predetermined qualitative and / or quantitative quality parameters of the crimp connection, and the image evaluation device is configured to generate and output an evaluation signal depending on the defect classification performed, in particular which includes the defect classification performed to determine predetermined qualitative and / or quantitative quality parameters of the crimp connection.

[0142] The image evaluation device thus serves not only for image processing but also for the evaluation of the cross-sectional image of the crimp connection. This allows for a reliable, objective, and robust determination of the qualitative and / or quantitative quality parameters of the crimp connection, as manual evaluation is no longer necessary.

[0143] The error classification for determining predefined qualitative and / or quantitative quality parameters of the crimp connection can be performed using classic image analysis software not based on neural networks, or in another way. In particular, this can be done using a separate evaluation unit to which the output signal of the image processing device can be fed.

[0144] The term "defect classification" is to be understood broadly in the context of the disclosed invention. Defect classification refers not only to the presence of defects in a cross-sectional view of the crimp connection, but also to the absence of defects. The defect classification can be qualitative in nature, for example, crimp connection "OK" or crimp connection "not OK". Furthermore, the defect classification can be designed to classify different defect patterns qualitatively and / or quantitatively. Quantitative determination is also to be understood as classification in this context. The defect classification thus includes, in particular, the categorization of qualitative and quantitative quality parameters of the crimp connection represented as a cross-sectional image, as well as the measurement of quantitative quality parameters, for example, in the form of actual values, from the vector contour.

[0145] During image analysis, a quantitative quality parameter can be assigned an actual value, which can be understood as a measured value for the respective quality parameter. This value can then be compared with a predefined target value for the respective quality parameter.

[0146] To determine the actual values, the image evaluation device is advantageously provided with a reference scale in addition to the cross-sectional image of the crimp connection. This reference scale allows the conversion of image features into SI units or other suitable scales, such as test bench-specific conversion values ​​like pixels to millimeters. This enables easy comparison with any target values, which can be specified in SI units, for example.

[0147] In an advantageous embodiment, the image evaluation device comprises a trained deep neural network, by means of which the error classification can be carried out to determine predetermined qualitative and / or quantitative quality parameters of the crimp connection.

[0148] Error classification can be performed efficiently and reliably using a trained deep neural network. The image processing device can, in particular, include a separate trained deep neural network, which is trained to perform the evaluation based on the output signal of the image processing device. Thus, the image processing device can, for example, include two separate, cascaded deep neural networks, where a first trained deep neural network is trained to perform image processing and a second trained deep neural network is trained to perform image evaluation based on the output signal of the image processing.

[0149] The deep neural network is trained for the corresponding classification task, as described above, using training data that represents the relevant error classifications. Error classification to determine predefined qualitative and / or quantitative quality parameters of the crimp connection can be performed, in particular, using a trained image transformer network or a trained deep convolutional neural network.

[0150] In another embodiment, the image evaluation device is designed such that image processing and defect classification for determining predefined qualitative and / or quantitative quality parameters of the crimp connection can be performed using a common deep neural network. Image processing and image evaluation can thus be combined. The image processing, as the basis for evaluating the micrograph of the crimp connection, is directly linked to the image evaluation.

[0151] This has the advantage that only one training process is required for image processing and subsequent error classification, while simultaneously providing a robust, reliable and objective evaluation of the quality parameters of the crimp connection.

[0152] If an image transformer network pre-trained for semantic segmentation is used, this can easily be extended with regard to error classification and determination of actual values ​​for quantitative quality parameters.

[0153] In a further embodiment, the evaluation output signal comprises at least one quality parameter from the following group: defect-free crimp connection, crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands. In particular, the evaluation output signal can comprise at least one measured value or actual value for at least one of the aforementioned quality parameters.

[0154] As a basis for this, a defect classification can be carried out for at least one of the defect classes from the following group: defect-free crimp connection, crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands. Using such quality parameters, preferably all of them, the respective crimp connection can be characterized, in particular completely.

[0155] The following quality parameters can be determined quantitatively and qualitatively: crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands.

[0156] In a further embodiment of the image evaluation device, the evaluation output signal is designed to include a superimposed representation based on the received image and the vector contour, in particular at least one boundary contour, wherein the measurement points used for determining the respective quality parameter are included in the superimposed representation as separately identified, in particular separately visible, marker points.

[0157] Marker points are markings within an image that identify specific points associated with a particular quality parameter. These markers, often placed along the edge of a boundary contour, allow for the calculation of distances, such as in pixels. These distances can then be converted into units of length using a conversion scale.

[0158] In embodiments that use marker points as reference points for determining actual values ​​of the crimp connection, it is not necessary for the marker points to be perceptible to a user. It is generally sufficient for determining the actual values ​​of quality parameters of the crimp connection if these marker points are present and can be used for this purpose.

[0159] However, it is advantageous if the marker points are clearly marked so that a viewer of the superimposed image can quickly and easily determine their location. This allows for a rapid, possibly random, review of the automated evaluation by monitoring personnel. This is particularly useful for verifying whether the image processing device is functioning correctly.

[0160] In particular, the different marker points for different quality parameters can be identified separately and distinguishably. Specifically, the trained deep neural network can be configured to locate and mark the relevant marker points in the image. Furthermore, the trained deep neural network can be configured to generate an output signal that allows the relevant marker points in the image to be displayed.

[0161] Distinguishable marker points can be used for one or more of the following quality parameters: defect-free crimp connection, crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands, and cracks. This allows each marker point to be easily assigned as a reference point to a corresponding measurement.

[0162] In a further embodiment, the image evaluation device is configured to feed the evaluation output signal to a central database, in particular a manufacturing database. This can be done, in particular, by means of a data interface included in the image evaluation device.

[0163] This allows the determined quality parameters to be stored and documented in a traceable manner. The results of the image analysis can thus be automatically assigned to the respective crimp connections produced, documented, retrieved, and traceable. By eliminating the need for manual recording of qualitative and / or quantitative quality parameters of the crimp connection, the consistency of the stored data is ensured, which in turn contributes to improved data analysis and management.

[0164] Furthermore, the image evaluation device can include an evaluation unit in which the quantitatively determined quality parameters are compared with target values ​​for the manufactured crimp connection. Depending on the result of the comparison, a control signal can be generated that influences the usability of similar crimp connections and / or the production of further similar crimp connections.

[0165] The target values ​​can be provided to the image evaluation device, in particular an evaluation unit of the image evaluation device, by means of a central database, in particular a manufacturing database, which is connected to the image evaluation device via data technology.

[0166] The central database, particularly the production database, can contain the target values ​​for each crimp connection production order. When connected to the image analysis device, these target values ​​can be transmitted to the device for comparison with the actual values, requested from the device, or received by the device, depending on the production order.

[0167] The invention also relates to a production release system for a crimping device, comprising an image evaluation device according to one of claims 7 to 12, a data interface to a database in which production order-dependent target values ​​for crimp connections are stored, and a release unit which is designed to provide, after a comparison between at least one qualitative and / or quantitative quality parameter of the crimp connection with an associated target value, a release or a refusal of release for the production of the fault-classified crimp connection.

[0168] The comparison is therefore made using the evaluation output signal provided by the image evaluation device, which includes at least one qualitative and / or quantitative quality parameter. Depending on this comparison, production is either approved or rejected.

[0169] In a further embodiment, the manufacturing release system comprises a release unit which is further configured to provide a release or a refusal of release of a delivery of the classified crimp connection based on the evaluation output signal.

[0170] The release unit is therefore not only designed to release a production run, but also to release crimp connections for shipment. This may be necessary if a manufacturing defect is only discovered subsequently, for example, through random sampling. In particular, a shipment stop can be recorded in the central database, thus preventing the defective crimp connection from being delivered to the customer.

[0171] In a further embodiment, the manufacturing release system comprises a database by means of which the evaluation output signal, comprising at least one quality parameter from the following group, can be stored: defect-free crimp connection, crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance crimp flank ends, burr height, burr width, bottom thickness, voids between strands, cracks. Brief character description

[0172] Advantageous embodiments of the invention are explained below with reference to the accompanying figures. These show: Figure 1 is a schematic representation of an embodiment of an image processing device, Figure 2 is a schematic representation of the structure of an exemplary image transformer network designed as a SegFormer, Figure 3 is a schematic representation of a received image, a semantically segmented 3-color raster image and a superimposed image from the received image and determined boundary contours, in particular identifiable by means of an image processing device according to Figure 1Figure 4 is a schematic representation of a first embodiment of an image evaluation device, Figure 5 is a schematic representation of a second embodiment of an image evaluation device, Figure 6 is a schematic representation of a received image, a semantically segmented 3-color raster image, a superimposed image of a superimposed image from a received image and a determined boundary contour, and quantitative quality parameters, which can be generated by means of an image evaluation device according to Figure 4 or Figure 5 Figure 7 shows a schematic embodiment of an embodiment of a manufacturing release device, and Figure 8 shows a schematic cross-sectional view of a crimp connection.

[0173] The figures are merely schematic representations and serve only to illustrate the invention. Identical or equivalent elements are consistently identified by the same reference numerals. Detailed description

[0174] Figure 1Figure 1 shows a schematic representation of an image processing device 100. This includes a receiving unit 101 for receiving a recorded cross-sectional image of a crimp connection.

[0175] This can be, in particular, a digitally recorded cross-sectional image of the crimp connection, especially a photographic microscope image of the cross-section of the crimp connection. The plane of the cross-sectional image of the crimp connection is, in particular, perpendicular to a longitudinal center axis of a cable on which the crimp connection is located. However, another cross-sectional plane can also be chosen if this appears helpful for determining quality parameters of the crimp connection.

[0176] Such a cross-sectional image of the crimp connection is generated, for example, by cutting the crimp connection perpendicular to the longitudinal axis of the cable. The cross-sectional image is usually prepared by grinding. Therefore, such a cross-sectional image is also referred to as a micrograph of the crimp connection.

[0177] The image processing device 100 further comprises a processing unit 102. The processing unit 102 serves to process the received cross-sectional image. Processing the received image enables improved image evaluation, as it eliminates undesirable deviations from a cross-sectional image taken under ideal conditions, deviations that frequently occur in practice.

[0178] For this purpose, processing unit 102 features a trained deep neural network configured as an image transformer network. This image transformer network is trained to process the image in the desired manner.

[0179] The image transformer network is specifically designed as a SegFormer and is configured to generate a semantically segmented raster image from the received image. Each pixel of an area of ​​interest in the received image, and in particular each pixel of the received image, can be assigned to a predefined class. This assignment is based on the training data, in which the respective classes were defined by appropriate annotations.

[0180] It is advantageous to use a publicly available, pre-trained image transformer network and to customize it for the segmentation task of cross-sectional images of crimp connections through fine-tuning, i.e., subsequent training tailored to the specific task. This significantly reduces the training effort for the user of the image transformer network. For this purpose, a pre-trained SegFormer can be used, which is then further trained by the user of the pre-trained SegFormer network using appropriately annotated training data.

[0181] A vector contour can be generated from the raster image produced by SegFormer using processing unit 102. This means the generated raster image is converted into a vector contour. This increases the accuracy for subsequent evaluation of the processed image. Based on this, processing unit 102 then generates an output signal, which can be used for further, improved evaluation.

[0182] Such image processing can eliminate artifacts and inaccuracies, thus preventing disruption of the evaluation by such artifacts and inaccuracies.

[0183] The output signal is designed such that it includes at least one qualitative and / or quantitative quality parameter attributable to the crimp connection, such that this parameter can be determined from the output signal. This at least one quality parameter can then be determined using standard image analysis software and / or manually.

[0184] In one approach, the conversion of the generated raster images into a vector contour can be done classically, i.e., without using a neural network. This is possible, for example, using known image processing techniques and algorithms. A common method for this is the application of the "canny edge detector" technique in combination with Hough transforms.

[0185] However, it is also possible to not only generate the raster image using the image transformer network, but – after appropriate training – also the vector contour itself, and optionally the corresponding output signal. In this context, the image transformer network can further include a residual network for generating the vector contour from the raster image.

[0186] Therefore, after receiving the cross-sectional image of the crimp connection, all processing steps can be carried out using a suitably trained deep neural network, in particular an image transformer network.

[0187] The raster view generated by the trained deep neural network typically comprises image areas assigned to a specific class. In a simple form, this can be a first class corresponding to an inner region of the crimp connection, such as the inner conductor area. Furthermore, a second class can be provided, corresponding to the crimp sleeve, and a third class, corresponding to the surrounding area of ​​the crimp connection.

[0188] A fourth class may also be provided, which is assigned, for example, to cavities within the first class, i.e., in the area of ​​the inner conductor.

[0189] If necessary, at least one further class may also be provided, which corresponds, for example, to a specific sub-area of ​​the inner component, such as individual strands, or to a specific sub-area of ​​the outer component, such as a burr of the crimp sleeve.

[0190] The corresponding classes are typically two-dimensional areas within the cross-sectional view of the crimp connection. The vector contour derived from the raster view can also be defined as an two-dimensional area within the cross-sectional view, representing the determined classes.

[0191] The vector contour can also only affect a portion of the raster image, in particular at least a boundary region between adjacent classes. This boundary region or regions are generally of considerable importance for determining the quantitative and / or qualitative quality parameters of the crimp connection.

[0192] The output signal provided by the image processing device can, in particular, be designed in such a way that a graphical reproduction of the planar and / or linear vector contour is possible.

[0193] The graphical representation can also be superimposed on the received cross-sectional image of the crimp connection, especially when displaying at least one boundary contour.

[0194] Furthermore, marker points can be determined which mark significant structures of the cross-section of the crimp connection, for example the reference points for the crimp width (average width) or the measurable crimp width (maximum crimp width), etc.

[0195] Figure 2Figure 1 shows a schematic representation of the structure of an image transformer network (BTN) configured as a SegFormer. Such an image transformer network (BTN) comprises an encoder section (E) and a decoder section (D). The encoder section (E) and its associated multi-layer perceptron (MLP) of the image transformer network (BTN) are generally of particular importance for achieving sufficiently good semantic segmentation of the cross-sectional image. The decoder section (D) serves, among other things, to scale the output of the encoder section (E) to the original image size, specifically the number of pixels. However, the decoder section (D) may also have other or additional functionalities.

[0196] The cross-sectional image received by the receiver unit is fed to encoder section E of the image transformer network. This image can be processed using Overlap Patch Embedding (OPE). With Overlap Patch Embedding (OPE), a large number of image sub-areas can be provided in multiple overlapping patches. The overlap area can, for example, be 50%.

[0197] Overlapping patches allows for better capture of global and local contextual information, leading to improved context representation for prediction and segmentation. Furthermore, the image or sub-images are transformed into a machine-readable format for the transformer blocks. Overlap Patch Embedding (OPE) particularly helps minimize artifacts that can arise due to the discrete nature of patch-based processing by supporting the continuity of features that span neighboring patches.

[0198] The overlapping patches of the received cross-sectional image of the crimp connection can be fed to a first transformer block T1, for example, in vector form. The depicted structure of the image transformer network BTN comprises four transformer blocks T1, T2, T3, and T4, which are structurally essentially identical and arranged in series. Furthermore, each of the transformer blocks is preceded by an overlap patch embedding unit (OPE).

[0199] Each of the four transformer blocks T1, T2, T3, and T4 comprises so-called attention mechanisms, such as self-attention, attention matrices, or mechanisms for modeling the relationships between the tokens or patches. These transformer blocks T1, T2, T3, and T4 process the supplied patch sequences using these attention mechanisms to extract global and local features from the image and to model relationships between the patches.

[0200] For this purpose, the four transformer blocks T1, T2, T3, and T4 can each contain a module for Efficient Self-Attention (ESA). This is a special form of self-attention characterized by the use of alternative approaches to reduce the computational complexity of self-attention.

[0201] This approach uses approximate attention mechanisms and compressed representations to reduce the number of elements to be considered. The aim is to optimize self-attention calculations without significantly reducing the model's performance.

[0202] Efficient self-attention (ESA) therefore facilitates the scalability of the SegFormer to large datasets and more complex tasks, making it easy to further train it for additional tasks. Furthermore, the demands on computing resources are lower than with traditional self-attention calculations.

[0203] After a patch has undergone the Efficient Self-Attention (ESA) calculations, the resulting data is fed into a Mix-Feed-Forward Network (MFFN), also referred to as Mix-FFN. The Mix-FNN (MFFN) performs mixing operations based on the features of each individual patch. The Mix-FNN (MFFN) attempts to improve the representations within each patch by combining or transforming different features or information.

[0204] The operations of the Mix-FNN MFFN help to emphasize or enhance specific features that are relevant for the semantic segmentation of images by better highlighting or combining these features.

[0205] Subsequently, an Overlap Patch Merging (OPM) process is performed, in which information from overlapping processed patches is fused or combined to achieve a more consistent and integrated representation of the image information. This enables improved integration of local and global contextual information.

[0206] Furthermore, these overlaps help to reduce artifacts or discontinuities at the patch boundaries that could otherwise occur if patch-based processing were to consider non-overlapping areas of the image. Overlap Patch Merging OPM also helps to reduce artifacts or discontinuities at the patch boundaries that could otherwise occur if patch-based processing were to consider non-overlapping areas of the image.

[0207] Each transformer block T1, T2, T3, T4 has an output through which the data generated by the respective transformer block T1, T2, T3, T4 is provided. This data is then fed to another overlap patch embedding OPE, which, with a modified overlapping patch layout (especially overlapping, smaller patches), is fed to the next transformer block, for example, T2, in machine-readable form, e.g., as a vector. In this transformer block T2, the same structural steps then take place as described above. However, each transformer block T1, T2, T3, or T4 can also have a different structure.

[0208] In this way, the data from the received cross-sectional images successively pass through the four transformer blocks T1, T2, T3 and T4 with overlap patch embedding in between.

[0209] Not only the result processed by all four transformer blocks T1, T2, T3, and T4 is fed to a multi-layer perceptron (MLP), but also all intermediate results. Therefore, so-called shortcuts, also known as skip connections, exist that lead from the respective OPE steps to the multi-layer perceptron (MLP) and through which the corresponding data can be fed to the MLP.

[0210] The multi-layer perceptron (MLP) complements the transformation functions of the transformer blocks T1, T2, T3, and T4 by providing an additional layer of feature modeling and processing specifically designed to improve segmentation accuracy. It allows the model to tailor features to the specific requirements of the segmentation task and to selectively transform them to achieve more precise segmentation.

[0211] The multi-layer perceptron (MLP) is therefore also the part of the model that can be adapted to the specific segmentation task, in this case the segmentation of the cross-sectional images of a crimp connection, by adding a layer or changing an existing layer through retraining, also known as fine-tuning.

[0212] The multi-layer perceptron (MLP) and the image transformer network (BTN) can be further trained to enable at least one fault classification. This means that a more detailed analysis of the cross-sectional image of the crimp connection is possible, going beyond simple image processing.

[0213] The multi-layer perceptron (MLP), which follows the transformer blocks T1, T2, T3, and T4 in the SegFormer, is at least one additional layer applied to the output of the transformer blocks T1, T2, T3, and T4 to improve segmentation results and further optimize model performance. This makes it possible to further refine and adapt the cross-sectional representations of the crimp connection derived from the transformer blocks T1, T2, T3, and T4.

[0214] Decoder D typically includes a pixel class decoding layer that converts the extracted feature representations into pixel class predictions. Here, for example, individual pixels are assigned to a class, such as class one, class two, or class three. This layer is therefore crucial for generating the desired raster view. It can be part of the multi-layer perceptron or implemented separately.

[0215] Furthermore, decoder side D typically contains upsampling operations and / or convolution layers to increase feature resolution and adapt the size of predictions to the original input size of the image.

[0216] Typically, there is also a classification layer that predicts the probabilities or labels for each pixel class in an image, leading to the creation of a complete semantic segmentation map or segmentation mask.

[0217] The decoding layer, the upsampling layer, and the classification layer are collectively referred to in Figure 2 schematically represented as a DM decoder module.

[0218] Such a pre-trained SegFormer network can be accessed, for example, via the website https: / / huggingface.co / docs / transformers / model_doc / segformer.

[0219] This model can then be adapted or customized with regard to the specific application of semantic segmentation of cross-sectional images of crimp connections, possibly of crimp connections that were manufactured on a specific crimping device.

[0220] Using such a model, a semantically segmented raster image of the cross-sectional image of the crimp connection can be provided.

[0221] It is understood that a person skilled in the art may also use other methods to perform semantic segmentation. In particular, a person skilled in the art may also use a deep neural convolutional network that is trained for the relevant task.

[0222] Figure 3 shows three exemplary images of a cross-section of a crimp connection C, each in a view as a photograph and as a line drawing.

[0223] A first image B1 shows a reproduction of the received image, which is subject to image processing by means of an image processing device, for example the image processing device according to Figure 1 , is.

[0224] The second image, B2, shows a semantically segmented raster image. The raster image comprises a first class K1, corresponding to the inner conductor; a second class K2, corresponding to the crimp sleeve; and a third class K3, corresponding to the area surrounding the crimp connection. Each class K1, K2, and K3 is assigned a different color or shade in the photograph. This clearly illustrates the relative boundary between adjacent classes K1, K2, and K3.

[0225] The first class K1 has a first color F1, the second class K2 a second color F2, and the third class K3 a third color F3. Preferably, immediately adjacent color areas have a high contrast ratio so that they are easily distinguishable visually.

[0226] Furthermore, the second image B2 has already been converted from a pixel-based raster view into a vector contour to increase the accuracy of a potential measurement on the second image B2.

[0227] The third image, B3, shows the received image, which is superimposed with a first boundary contour G1 and a second boundary contour G2. The first boundary contour G1 traces the boundary between the first class K1 and the second class K2.

[0228] This allows, for example, the inner conductor area to be distinguished from the crimp sleeve. The second boundary contour G2 traces the boundary between the second class K2 and the third class K3, thus allowing, for example, the crimp sleeve to be distinguished from the surroundings of the crimp connection.

[0229] The boundary contours G1 and G2 may, for example, have been determined from the vector contour of the second image B2.

[0230] Furthermore, the third image, B3, shows a number of marker points, in particular marker points M1, M2, M3, M4, and M5, which can be used to determine quantitative quality parameters of the crimp connection. Marker points M1, M2, M3, M4, and M5 can be conventionally determined from the vector contour, for example, by logic operations or by a suitably trained deep neural network.

[0231] In Figure B3, exemplary marker points M1, M2, M3, M4, and M5, as well as other marker points, are each marked with a white dot. However, different colors or other distinguishing features can be used for each marker point M1, M2, M3, M4, and M5. In particular, all reference points can be marked with marker points M1, M2, M3, M4, and M5, which are necessary for determining quantitative quality parameters; see details below. Figure 8 .

[0232] The (average) crimp width can be determined using marker points M1 and M2. The crimp height of the crimp connection can be determined using marker points M3, M4, and M5.

[0233] Based on the respective marker points, a traceable, objective measurement of quantitative quality parameters of the crimp connection C recorded in cross-section can be carried out.

[0234] Figure 4 shows an image evaluation device 200, which includes an image processing device 100 according to Figure 1 includes. The explanations regarding the image processing device 100 in the context of the Figure 1 apply accordingly Figure 4 .

[0235] The output signal provided by the image processing device 100 is evaluated by means of an evaluation unit 201 of the image evaluation device 200.

[0236] The evaluation unit 201 determines, for example, the qualitative and / or quantitative quality parameters based on the provided marker points of the vector contour, such as the absence of defects in the crimp connection, crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands, and / or cracks. The evaluation unit 201 can also determine the marker points itself and not receive them from the image processing device.

[0237] For quantitative quality parameters, measurements for the respective quality parameters can first be determined from the vector contour. These can then be converted into real units of measurement, such as degrees and lengths, for example in SI units, using a corresponding reference.

[0238] In particular, this reference can be test bench-specific and thus takes into account the individual measurement setup for capturing the cross-sectional image of the crimp connection. The evaluation output signal can, for example, include quantitative quality parameters that are evaluated in a subsequent comparison.

[0239] Furthermore, the evaluation unit can also be designed in such a way that it provides an evaluation output signal which includes a statement about the presence or absence of errors with regard to the analyzed crimp connection.

[0240] In particular, qualitative quality parameters and / or quantitative quality parameters of the crimp connection determined with corresponding target values ​​can be used to determine whether the crimp connection is "OK", i.e., free of defects within the specified tolerances, or "not OK", i.e., exceeds the specified tolerances.

[0241] In such a configuration, the evaluation unit 201 is advantageously designed to access target values ​​for predefined quality parameters. In particular, a Figure 4An interface to a database (not shown) is provided in which relevant manufacturing data, such as target values ​​for specific quantitative quality parameters, are stored. This data can then be used by the evaluation unit 201 for comparisons with actual values. Furthermore, the actual values ​​determined by the image evaluation device 200 can be fed into the database and stored there for retrieval.

[0242] Figure 5 showed another embodiment of an image evaluation device 200. This is characterized in that the image processing unit 102 and the evaluation unit 201 are combined in a combined image analysis unit 201'.

[0243] The combined image analysis unit 201' is designed to include a trained deep neural network, in particular an image transformer network, which is trained to perform both image processing and evaluation.

[0244] For this purpose, the deep neural network is trained accordingly; that is, a raster image is generated by semantic segmentation of at least the relevant image area of ​​the cross-sectional image, in particular the entire cross-sectional image of the crimp connection. Furthermore, the deep neural network is trained to generate at least one vector contour from the raster image and, based on the generated vector contour, to perform a defect classification to determine predefined qualitative and / or quantitative quality parameters of the crimp connection.

[0245] Furthermore, the trained deep neural network is designed to output an evaluation signal depending on the error classification performed, in particular such that the evaluation output signal includes the error classification performed to determine specified qualitative and / or quantitative quality parameters of the crimp connection.

[0246] Such a trained deep neural network can [achieve the following]: Figure 2 The setup shown is as follows, with the multi-layer perceptron trained accordingly to perform the aforementioned tasks. Additional layers can be added to the multi-layer perceptron for this purpose, which are trained using error-classified training data.

[0247] Both by means of an exemplary configuration of an image evaluation device 200 according to Figure 4 as well as after Figure 5This enables an essentially error-free, automated determination of quantitative and / or qualitative quality parameters, so that the measurement of the properties of a crimp connection can be objectified.

[0248] Figure 6 shows a possible result of an image evaluation device 200. Figure 6 This includes, to that extent, the image display of images B1, B2 and B3 generated by an image processing device, accordingly. Figure 3 .

[0249] However, it also includes Figure 6 a fourth image B4, which, for example, shows the actual values ​​of the examined crimp connection C determined by means of the image transformer network and / or qualitative quality parameters such as "OK" or "not OK".

[0250] In particular, the fourth image B4 may, for example, contain the following content. Cross-section measurement / quantitative quality parameter Actual value / Quantity [cm] Quality Crimp height 0,6641 In order Crimp width 1,0002 In order Measurable crimp width 1,0339 Not ok Support angle 3,6285 In order Support height 0,1742 In order soil thickness 0,1164 In order cavity between strands available Not ok Overall result crimp connection Not ok

[0251] The actual values ​​in the preceding table are determined using the trained deep neural network, which provides a corresponding evaluation output signal that includes the corresponding actual values ​​or statements.

[0252] The evaluation can further include a comparison to determine whether the measured actual values ​​lie within a specified tolerance range around a target value for the respective quality parameter of the crimp connection. This comparative evaluation can be performed separately from the trained deep neural network, for example, using a separate comparison unit, or it can also be integrated into the trained deep neural network itself. This integration can be carried out by the image processing device.

[0253] Figure 7 shows a schematic representation of a manufacturing release system 300 for a crimping device for manufacturing a crimp connection.

[0254] In this embodiment, the manufacturing release system 300 comprises a cutting system 301 in which a cross-section of a crimp connection can be produced by cutting and grinding the crimp connection perpendicular to the longitudinal direction of the cable comprising the crimp connection.

[0255] Furthermore, the manufacturing release system 300 includes an image acquisition device 302 designed as a microscope for capturing a digital image of the cross-section of the crimp connection. The cut crimp connection is thus fed to the microscope 302 and imaged by means of the microscope 302, in particular as accurately as possible and without image acquisition errors.

[0256] Furthermore, the manufacturing release system 300 includes an image evaluation device 200. This can, for example, be configured according to Figure 4 or Figure 5 be trained. This includes a receiving unit 101 for receiving the cross-sectional image of the crimp connection.

[0257] Furthermore, this includes a trained deep neural network configured as an image transformer network, which is trained for both image processing and image analysis. This is made available by means of a combined image analysis unit 201'.

[0258] The combined image analysis unit 201' is designed to determine desired quantitative and qualitative quality parameters of the crimp connection within the framework of a defect classification. The quantitative quality parameters can include, in particular: crimp height, crimp width, measurable crimp width, support angle, support height, flank end distance, distance between crimp flank ends, burr height, burr width, bottom thickness, voids between strands, and cracks.

[0259] The combination image analysis unit 201' is used, among other things, to quantitatively determine quality parameters of the crimp connection, i.e., to measure distances and / or angles based on the recorded and processed cross-sectional image of the crimp connection.

[0260] When determining quantitative quality parameters, any marker points used for quantitative analysis do not necessarily need to be visually represented. However, having visual representations of the marker points improves the traceability of the measured values.

[0261] The determined quantitative quality parameters are fed to a comparison unit 202, which according to Figure 7The image evaluation device 200 is included. However, this is not mandatory. The comparison unit 202 is connected to a production database 304, which is also included in the production release system 300, via a database interface 303. The production database 304 can be designed, for example, as a database, e.g., as an ERP database, in particular as an SAP database.

[0262] The production database 304 contains relevant production data for order A to be manufactured, and thus also corresponding target values ​​for a crimp connection according to order A. Therefore, target values ​​for the desired quantitative quality parameters can be provided via production database 304 and transmitted to the comparison unit 202.

[0263] The comparison unit 202 compares the determined quantitative quality parameters with the corresponding target values. This comparison reveals whether the manufactured and analyzed crimp connection is defective or not. Furthermore, the determined quantitative quality parameters, i.e., the corresponding actual values, are transferred to and stored in the production database 304.

[0264] The comparison unit 202 is connected to a release unit 305. The release unit 305 determines whether the production of the crimp connection should be released or not.

[0265] If the comparison reveals that the crimp connection exhibits quantitative quality parameters within the target value tolerances, production of the crimp connection is released. If the comparison reveals that the crimp connection exhibits quantitative quality parameters outside the target value tolerances, production of the crimp connection is not released. This can also apply to qualitative, i.e., unmeasured, quality parameters, whereby release via release unit 305 is not granted if a given quality parameter is qualitatively classified as "not OK".

[0266] Furthermore, the release unit 305 can be connected to the delivery system. If a subsequent analysis of a cross-sectional image of a crimp connection reveals that a completed production order is likely to be faulty, the release unit 305 can be configured to provide a signal that stops the delivery of the production order to the customer.

[0267] It is understood that the person skilled in the art can also divide the functionalities of the aforementioned units for the manufacturing release system 300 among corresponding units in other ways or, if necessary, provide them by means of a single technical unit.

[0268] Figure 8 shows a schematic cross-sectional view of a crimp connection, which illustrates the reference points for measuring quantitative quality parameters.

[0269] Reference numeral 1 identifies the reference points of the crimp connection for measuring the crimp height. Reference numeral 2 identifies the reference points of the crimp connection for measuring the crimp width. Reference numeral 3 identifies the reference points of the crimp connection for measuring the measurable crimp width. Reference numeral 4 identifies the reference points of the crimp connection for measuring the support angle. Reference numeral 5 identifies the reference points of the crimp connection for measuring the support height. Reference numeral 6 identifies the reference points of the crimp connection for measuring the flank end distance. Reference numeral 7 identifies the reference points of the crimp connection for measuring the distance between the crimp flank ends. Reference numeral 8 identifies the reference points of the crimp connection for measuring the burr height. Reference numeral 9 identifies the reference points of the crimp connection for measuring the burr width.Reference numeral 10 clarifies the reference points of the crimp connection for measuring the base thickness. Reference numeral 11 clarifies the reference points of the crimp connection for measuring a crack. While not shown, a person skilled in the art can readily perform a corresponding measurement of the cavity, possibly in several directions, using marker points for cavities, based on their expertise. This applies analogously to crack detection.

[0270] Since the devices and methods described in detail above are exemplary embodiments, they can be modified extensively by a person skilled in the art without departing from the scope of the invention. In particular, the mechanical arrangements and the relative sizes of the individual elements are merely exemplary. REFERENCE MARK LIST

[0271] 100 Image processing device 101 Receiving unit 102 Processing unit 200 Image evaluation device 201 Image evaluation unit 201' Image evaluation unit with a trained deep neural network for image processing and image evaluation, configured as a unit: Combination image analysis unit 202 Comparison unit 300 Production release system 301 Cutting unit 302 Image acquisition device 303 Data interface 304 Database 305 Release unit K1 First class: inner component of the crimp connection: conductor element K2 Second class: outer component of the crimp connection: crimp sleeve K3 Third class: surroundings of the crimp connection B1 Received cross-sectional image B2 Raster image in 3 colors B3 Superimposed image from received image and boundary contour F1 Color 1 F2 Color 2 F3 Color 3 G1 Boundary contour between first and second class G2 Boundary contour between second and third class C Cross-section of crimp connection M1 Marker point 1 M2 Marker point 2 M3 Marker point 3 M4 Marker point 4 M5 Marker point 5 A Order number T1 Transformer block, first T2 Transformer block, second T3 Transformer block, third T4 Transformer block, fourth OPE Overlap Patch Embedding MLP Multi-Layer Perceptron ES A Efficient Self Attention MFF N Mix-Feed Forward Network OP M Overlap Patch Merging E Encoder D Decoder DM Decoder Module BTN Image Transformer Network 1 Crimp height 2 Crimp width 3 Measurable crimp width 4 Support angle 5 Support height 6 Flank end distance 7 Distance crimp flank ends 8 Burr height 9 Burr width 10. Floor thickness 11. Crack

Claims

1. Image processing device (100) for supporting qualitative and / or quantitative assessment of the quality of a crimp connection (C), - having a receiving unit (101) for receiving a cross-sectional representation of an image, in particular in the form of a digital microscope image in the visible spectral range, of a crimp connection (C), - having a processing unit (102) which is designed to generate a raster image of the received image from the received image by means of a trained deep neural network by virtue of the fact that at least the pixels of a relevant image area of the received image can be assigned to a predefined class (K1, K2, K3) by means of the trained deep neural network, to generate at least one vector contour (G1, G2) from the generated raster image, to generate and output an output signal based on the determined vector contour (G1, G2), from which at least one qualitative and / or quantitative quality parameter (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) that can be assigned to the crimp connection (C) can be determined.

2. Image processing device according to Claim 1, wherein the trained deep neural network comprises an image transformer network (BTN) and / or a convolutional network.

3. Image processing device according to Claim 2, wherein the deep neural network is in the form of an image transformer network (BTN), wherein the image transformer network (BTN) comprises a publicly available pre-trained image transformer network (BTN) and at least one network layer which is trained, in particular subsequently, with application-specific training data for image processing.

4. Image processing device according to one of the preceding claims, wherein the trained deep neural network is designed to generate from the received image a raster image having at least three classes (K1, K2, K3), wherein a first class (K1) corresponds to an inner component of the crimp connection (C), in particular a conductor element, a second class (K2) corresponds to an outer component of the crimp connection (C), in particular a crimp sleeve, and a third class (K3) corresponds to the environment of the crimp sleeve, in particular the environment of the crimp connection (C).

5. Image processing device according to Claim 4, wherein the trained deep neural network is designed to assign a colour (F1, F2, F3) in each case to at least the first, the second and the third class (K1, K2, K3), wherein adjacent classes (K1, K2, K3) in the raster image and / or in the vector contour can be reproduced as, in particular contrasting, colour areas of different colours (F1, F2, F3) and this colour assignment is included in the output signal.

6. Image processing device according to Claim 4 or 5, wherein the trained deep neural network is designed to determine a boundary contour (G1, G2) between the first class (K1) and the second class (K2) and / or the second class (K2) and the third class (K3).

7. Image evaluation device (200) for qualitatively and / or quantitatively assessing the quality of a crimp connection (C), - having an image processing device (100) according to one of the preceding claims, - wherein an error classification for determining predefined qualitative and / or quantitative quality parameters (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) of the crimp connection (C) can be carried out on the basis of the vector contour generated, - wherein an evaluation output signal can be generated and output depending on the error classification carried out, in particular which comprises the error classification carried out for determining predefined qualitative and / or quantitative quality parameters (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) of the crimp connection (C).

8. Image evaluation device according to Claim 7, comprising a trained deep neural network, in particular a trained deep image transformer network (BTN), by means of which the error classification for determining predefined qualitative and / or quantitative quality parameters (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) of the crimp connection (C) can be carried out.

9. Image evaluation device according to Claim 8, wherein the image processing and the error classification for determining predefined qualitative and / or quantitative quality parameters (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) of the crimp connection (C) can be carried out by means of a common deep neural network, in particular by means of an image transformer network (BTN).

10. Image evaluation device according to one of Claims 7 to 9, wherein the evaluation output signal comprises at least one quality parameter (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) from the following group, in particular at least one actual value of a quality parameter (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) from the following group: correctness of the crimp connection, crimp height (1), crimp width (2), measurable crimp width (3), support angle (4), support height (5), flank end distance (6), crimp flank end distance (7), burr height (8), burr width (9), base thickness (10), cavities between strands, cracks.

11. Image evaluation device according to one of Claims 7 to 10, wherein the evaluation output signal is designed to comprise a superimposed representation based on the received image and the vector contour (G1, G2), wherein the measurement points used for determining the respective quality parameter are included in the superimposed representation as separately identified marker points (M1, M2, M3, M4, M5).

12. Image evaluation device according to one of Claims 7 to 11, which is designed to supply the evaluation output signal to a database (304), in particular a production database.

13. Production release system (300) for a crimping device, having an image evaluation device (200) according to one of Claims 7 to 12, having a data interface (303) to a database (304), in which database (304) production order-dependent target values for crimp connections (C) are stored, having a release unit (305) which is designed, after a comparison between at least one qualitative and / or quantitative quality parameter (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) and an associated target value, to provide a release or a refusal of release for the production of the error-classified crimp connection (C).

14. Production release system according to Claim 13, having a release unit (305) which is further designed to provide a release or a refusal of release for delivery of the classified crimp connection (C) on the basis of the evaluation output signal.

15. Production release system according to either of Claims 13 and 14, having a database (304) that can be used to store the output signal comprising at least one parameter from the following group: correctness of the crimp connection, crimp height (1), crimp width (2), measurable crimp width (3), support angle (4), support height (5), flank end distance (6), crimp flank end distance (7), burr height (8), burr width (9), base thickness (10), cavities between strands, cracks.